-1

I have such directory/file structure:

myPath\20140511\0002.xml
               \0003.xml
      \20140513\0004.xml
               \0006.xml
               \0008.xml
      \20140515\0009.xml
      \20140516\0010.xml
               \0011.xml
               \0012.xml

... where myPath is base directory of data, then directories named by dates in "yyyyMMdd" format.

How would I get sorted directory listing with sorted file listing from that?
For example, if I need files from 12.05.2014. to 15.05.2014. files 0004.xml, 0006.xml, 0008.xml, 0009.xml should be listed. Note that some number of files is missing and some dates is missing too.
Directories should be read in alphabetical order.

How to get that?

2
  • Stack Overflow is not a code-writing service. If you show us the code you have tried and explain why it isn't working, we can help you with that. Nov 25, 2014 at 19:50
  • Who said that Stack Overflow is code-writing service? My code may turn possible solution to wrong direction so sometimes is better to see independent point of view to common problem such is mine. It seem you didn't think enough before negative voting.
    – Wine Too
    Nov 25, 2014 at 21:11

3 Answers 3

1

Okay. Linq-lovers will be happy with this very basic code :)

Imports System.Linq
Imports System.IO

The core functions :

Private Function GetSubDirs( _
    ByVal myPath As String, _
    ByVal DateStart As String, ByVal DateEnd As String) _
    As IEnumerable(Of String)

    DateStart = myPath + "\" + DateStart
    DateEnd = myPath + "\" + DateEnd
    ' weird isn't it ? but heh ! it works...

    Dim SelectedSubDirs As IEnumerable(Of String) = _
        From InRangeDir In Directory.GetDirectories(myPath) Order By InRangeDir
        Where (InRangeDir >= DateStart) AndAlso (InRangeDir <= DateEnd)

    Return SelectedSubDirs
End Function

Private Function GetSubDirFiles(ByVal CurrentDir As String) _
    As IEnumerable(Of String)
    ' Gets the Files of one Sub Dir sorted in ascending order.
    Return From AnyFile In Directory.GetFiles(CurrentDir) Order By AnyFile
End Function

Then the ones you'll have to call :

Public Function GetSubDirsFilesRange( _
    ByVal myPath As String, _
    ByVal DateStart As String, ByVal DateEnd As String) _
    As List(Of String)
    ' Gets all the Files in a range of Directory Dates.

    Dim AllFiles As New List(Of String)()
    Dim SelectedSubDirs As IEnumerable(Of String) = _
        GetSubDirs(myPath, DateStart, DateEnd)

    For Each CurrentDir As String In SelectedSubDirs
        AllFiles.AddRange(GetSubDirFiles(CurrentDir))
    Next

    Return AllFiles
End Function

Public Function GetSubDirsRange( _
    ByVal myPath As String, _
    ByVal DateStart As String, ByVal DateEnd As String) _
    As List(Of String)

    Return GetSubDirs(myPath, DateStart, DateEnd).ToList()
End Function

.

GetSubDirsFilesRange(myPath, "20140512", "20140515")
    ' myPath\20140513\0004.xml
    ' myPath\20140513\0006.xml
    ' myPath\20140513\0008.xml
    ' myPath\20140515\0009.xml

Why I don't like Linq much is the difficulty to debug codes much more complex than this.

The other answer came from an AddonManager of mine that handles thousands of zip files and directories as ressources for a game. My folder scheme is type :

yyyyMMdd\*.*
yyyyMMdd_Saturday\*.*
yyyyMMdd_Monday_ToDo_Series\*.*

and can contain the following subdirectories :

\_Installed
\_Rejected
\_Revised

Int32 was my best choice for performance issues (several thousands of queries per user action, and other eye candy things like drawing GDI+ or (un)packing zip files)

4
  • Hm, also complicated. Would you advise me to use DirectoryInfo.GetFiles to Array(Of dirInfo+fileinfo) and sort it by usage of relevant IComparer. Not so fast but with less coding I suppose? Anyway, your examples are educational so I will accept your answer. Thanks.
    – Wine Too
    Nov 25, 2014 at 21:12
  • @user973238 - You could use use an array(of Anything1 + Anything2) if your 1 and 2 are the same type. DirInfo and FileInfo are not the same type. You may use the paths (which are string), but you addressed a specific task : to select a set of directories and/or files providing a starting date and an ending date. Sorting is very easy, and your array will do just fine; then, think of the code to get the files in range.. you'll end up with as much lines as the Linq example above. :) Nov 26, 2014 at 6:56
  • @user973238 - the key part is not sorting your collection. Each sorting is usually one single line of code ("Order By InRangeDir" with Linq; SortedList(Of String...); Array.Sort(..)) The key part is selecting the items that are in range. That task makes the code longer... If you sort by subdir "yyyyMMdd\xxxx.xml", you loose the ability to extract "yyyyMMdd" so you must write more code to extract that anyway. That's why it's better to combine directory parsing along with direct item access (using a key for example) in a variable or Class (the other answer) This answer just bypass all that. Nov 26, 2014 at 7:06
  • @user973238 - But thanks for the upvote and accept :) "complicated" is a matter of taste. It would be great if something like this existed : Dim Files() As String = Array.Join(Directory.SelectIn(myPath Where DateStart* <= .Name <= DateEnd*).ForEach(.GetFiles().Sort()) - and you're done with just one line of code... Maybe in vb.Net 17.. XD Nov 26, 2014 at 7:23
1

Simple and straightforward :

  • Your dates are already in yyyyMMdd format, which are basically Integers : easy to sort -> Use SortedList(Of Int32, Anything)
  • I'm assuming your FileSystem structure immutable : You'll always have an ordered list of dates (folder names), and for each date a list of File(name)s.

Disclaimer : was too long to test it in an IDE.. could contain typos or not working things that I'll try to fix later (kind of a hurry now..) Especially when I'm unsure whether I'm allowed to safely access the .Keys Property of a SortedList (remembering an exception like "IList altered while accessing collection...")


I would create a class (while you could also do it without)

Imports System.IO

Private Class CustomDateTypeDirectory
    Private p_BaseDirectory As String = ""
    Private p_FileSystem As SortedList(Of Int32, List(Of String)) = Nothing

    Public Sub New(ByVal NewDirectoryPath As String)
        ' where NewDirectory stands for myPath
        If Directory.Exists(NewDirectoryPath) Then
            p_FileSystem = New SortedList(Of Int32, List(Of String))
            ParseDirectory(NewDirectoryPath) ' -> below...
        Else
            ' ... Throw New Exception()
        End If
    End Sub

    ' Called by New() -> Parse your myPath Directory SubFolders
    Private Sub ParseDirectory(ByVal NewDirectoryPath As String)
        Dim AllDirectories As String() = Directory.GetDirectories(NewDirecoryPath)
        Dim SubDirName As Int32 = -1

        p_FileSystem.Clear()
        For Each SubDir As String In AllDirectories
            SubDirName = IsDateFolder(Path.GetDirectoryName(SubDir))
            If SubDirName > -1 Then
                p_FileSystem.Add(SubDirName, GetFilesOf(SubDir))
            End If
        Next
    End Sub

    ' Called by ParseDirectory() -> Validate a SubDir Name as expression of Date
    Private Function IsDateFolder(ByVal DirName As String) As Int32
        ' Here you'll define how you'll consider the name as a valid "yyyyMMdd"
        If DirName.Length = 8 Then
            Dim IntValidator As Int32 = -1
            If Integer.TryParse(DirName, IntValidator) Then
                ' You could do more checks like if IntValidator > 20000101...
                ' Or simply try parsing the represented Date...
                ' I sometimes use "20140300" dit names when I don't know the day,
                ' so while using Date would throw an exception, 
                ' my function just simply don't care using Integers...
                Return IntValidator
            Else
                Return -1
            End If
        Else
            Return -1
        End If
    End Function

    ' Called by ParseDirectory() -> Gather Files of Date SubFolder
    Private Function GetFilesOf(ByVal DirPath As String) As List(Of String)
        ' I'm assuming you make sure to properly sort the Files
        Dim AllFiles As String() = Directory.GetFiles(DirPath)
        Dim FilesList As New List(Of String)

        For Each CurrentFile As String In AllFiles
            ' Use condition validation if your subDir can contain other files..
            FilesList.Add(Path.GetFileName(CurrentFile))
        Next

        Return FilesList
    End Function

End Class

Then create some Functions to use your Class

This is the basic declaration :

    Private Function GetDirectoryRange( _
        ByVal DateStart As Int32, _
        ByVal DateEnd As Int32) As List(Of String)
        ' ...
    End Function

This is an overload :

    Public Function GetDirectoryRange( _
        ByVal DateStart As Int32, _
        ByVal DateEnd As Int32, _
        ByVal IncludeStartingDate As Boolean, _
        ByVal IncludeEndingDate As Boolean) As List(Of String)
        ' ...
    End Function

Note that the basic declaration is Private while the othe is Public.

Two other Private Function are required :

    Private Function GetNearestStartingDate(ByVal DateStart As Int32) As Int32
        If p_FileSystem.Count > 0 Then
            If p_FileSystem.ContainKey(DateStart) Then
                Return DateStart
            Else
                ' Find next available Date...
                If p_FileSystem.Keys.Item(0) > DateStart Then
                    Return p_FileSystem.Keys.Item(0)
                ElseIf p_FileSystem.Keys.Item(p_FileSystem.Keys.Count - 1) < DateStart Then
                    Return - 1 ' DateStart greater than last folder..
                Else
                    ' Okay -_- ! Let's find an existing starting Date...
                    ' Simpliest way is iterating the integers 
                    ' until it's greater than DateStart
                    Dim i As Int32 = 1 ' we've already checked for i = 0
                    While DateStart > p_FileSystem.Keys.Item(i)
                        i += 1
                    End While

                    Return p_FileSystem.Keys.Item(i)
                End If
            End If
        Else
            Return -1 ' No StartingDate
        End If
    End Function

I'm assuming you're able to create the other GetNearestEndingDate() Function.. (looking at your reputation score)


Wonder why the basic declaration above is Private ? Because it's working ONLY with valid DateStart and DateEnd inputs.

Here is the Public declaration :

    Public Function GetDirectoryRange(DateStart, DateEnd, IncludeStartingDate, IncludeEndingDate) As List(Of String)

        ' First, fix some possible issues :
        If DateStart > DateEnd Then
            Dim TempDate As Int32 = DateStart
            DateStart = DateEnd
            DateEnd = TempDate
        End If ' obvious

        If Not IncludeStartingDate Then DateStart = DateStart + 1
        If Not IncludeEndingDate Then DateEnd = DateEnd - 1

        ' Now let's find actual values of DateStart and DateEnd...
        If DateStart <= DateEnd Then ' Okay, let's look for available dates...
            DateStart = GetNearestStartingDate(DateStart)
            If DateStart > -1 Then
                DateEnd = GetNearestEndingDate(DateEnd)
                If DateEnd >= DateStart Then
                    Return GetDirectoryRange(DateStart, DateEnd)
                Else
                    Return Nothing ' Or Return New List(Of String) - empty !
                End If
            Else
                Return Nothing
            End If
        Else
            Return Nothing
        End If
    End Function

Then here's the basic function :

    Private Function GetDirectoryRange(DateStart, DateEnd) As List(Of String)
        Dim NewList As New List(Of String)
        Dim DateIndex As Int32 = p_FileSystem.IndexOfKey(DateStart) + 1

        NewList.Add(p_BaseDirectory + "\" + DateStart.ToString())
        While p_FileSystem.Keys.Item(DateIndex) <= DateEnd
            NewList.Add( _
                p_BaseDirectory + "\" _
                + p_FileSystem.Keys.Item(DateIndex).ToString())
            DateIndex = DateIndex + 1
        End while

        Return NewList ' New List should contain at least one entry.
    End Function

Function extrapolations to get Files is pretty straightforward :

    Private Function GetFilesRange(DateStart, DateEnd) As List(Of String)
        Dim NewList As New List(Of String)
        Dim DateIndex As Int32 = p_FileSystem.IndexOfKey(DateStart)
        Dim CurrentFilesList As List(Of String)

        Do
            CurrentFilesList = p_FileSystem.Item(p_FileSystem.Keys.Item(DateIndex))
            For Each CurrentFile As String In CurrentFilesList
                NewList.Add( _
                    p_BaseDirectory + "\" _
                    + p_FileSystem.Keys.Item(DateIndex).ToString() + "\" _
                    + CurrentFile)
                ' As CurrentFile if FileName without root path.
            Next
            DateIndex = DateIndex + 1
        Until p_FileSystem.Keys.Item(DateIndex) > DateEnd

        Return NewList ' New List should contain at least one entry.
    End Function

The public declaration is almost the same as the one for Directories, but calling GetFilesRange(DateStart, DateEnd) instead.


Then you could just write :

Try
    Dim MyListOfDir As List(Of String) = _
        New CustomDateTypeDirectory(myPath)().GetDirectoryRange( _
            DateStart, DateEnd, True, False)
    ' ....
' ....

You could replace the Try with If blocks like

    Dim MyDir As New CustomDateTypeDirectory(myPath)
    Dim MyListOfDir As List(Of String) = _
        MyDir.GetDirectoryRange(DateStart, DateEnd, True, False)
    If MyListOfDir IsNot Nothing Then
    ' or If MyListOfDir.Count > 0 ... depending on how the Function returns value
        ' ...
    End If

Do the same with GetFilesRange(DateStart, DateEnd, True, False) ....


You can also overloads the public declarations with new parameter types :

    Public Function GetFilesRange( _
        ByVal DateStart As Date, ByVal DateEnd As Date, Bool1, Bool2) _
        As List(Of String)

        ' Convert Dates to Int32
        Dim IntDateStart As Int32 = _
            DateStart.Year * 1000 _
            + DateStart.Month * 100 _
            + DateStart.Day

        ' ...
        Return GetFilesRange(IntDateStart, IntDateEnd, Bool1, Bool2)
    End Function

    Public Function GetFilesRange( _
        ByVal DateStart As String, ByVal DateEnd As String, Bool1, Bool2) _
        As List(Of String)

        ' Convert Dates to Int32
        Dim IntDateStart As Int32 = IsDateFolder(DateStart)
        Dim IntDateEnd As Int32 = IsDateFolder(DateEnd)

        If (IntDateStart > -1) AndAlso (IntDateEnd > -1) Then
            Return GetFilesRange(IntDateStart, IntDateEnd, Bool1, Bool2)
        Else
            Return Nothing ' Or Return New List(Of String)()
        End If
    End Function

Of Course, everything above doesn't work with Date Folder that are named after any date that is below year 1000 for example. ie if you have leading zeroes "0" in your folders like "07821125" which stands for the 25th of November, 782...

1
  • But Karl, your scenario is much complicated. Note that my Dir's and file names are already such to be sortable as strings. Here should be much simpler solution. Thank anyway for examples worth to study.
    – Wine Too
    Nov 25, 2014 at 17:24
1

Hint: While this may not be the most elegant way, it is easy enough to implement. You could read in the files information into an ADO.NET DataTable. While filling the DataTable, you could fill the missing dates using the parent's date. Use ADO sort, filter capabilities to get what you want. Some of the code is already here: filling-a-grid-with-files-in-a-folder.

1
  • I don't know. Involving ADO here seem' a bit overcomplicating.
    – Wine Too
    Nov 25, 2014 at 21:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.