DirectoryList

• DirectoryListDemo - 49.8 Kb
• Source - 48.1 Kb
• 2Backup - 52.2 Kb

Introduction

Every time one works with a computer its data is manipulated for many unique and interesting tasks: playing MP3's, browsing pictures, or even reading email... In all cases this data could be moved, read, written, copied etc. to arbitrary destinations within the storage sub-system. Amazingly this all happens under the hood allowing the everyday user to enjoythe benefits of what today's computers have to offer. Their comes a point, however, when certain data becomes important not only to the computer, but also to its user. Whether ones data is important is ultimately subjective however the question of how to manipulate this data is what first initiated my quest into learning how the .NET framework could help. Its been a long and exciting road since first venturing into this project not only because of the many solutions already provided but also because of the broadness and complexity the problem encompasses. My goals for this article are to share my initial problems,ideas, and solutions as to help others better understand the dynamics of working with data within the .NET framework and moreover how I used it to help manipulate data for my needs.

Background

Although manipulating data can mean many things, my initial problem involved creating a program that easily allowed one to backup files and folders to a designated location they specify. The project, which I called 2Backup, includes a ListBox for all files/folders added, buttons to add and remove files/folders to and from the listbox, a destination textbox, and a copy button to start the backup. Here is a picture of the basic layout.

The idea was to either drag and drop, or use the add buttons to add the files or folders you want to backup into the ListBox. After choosing a destination, you select a copy mode, and start the backup. I decided on three copy modes:

• Directory Copy - copies all files/folders remembering the source folder structure
• Consolidate Files - copies all files/folders into one folder
• CD Backup - same as Directory Copy except every 700MB it creates a new folder

The first problem that arose was how to implement a function that added files and folders into a listbox using the .NET framework. I decided to use a OpenFileDialog that, with multiSelect set to true, copied all the selected files into the listbox.Their were a few problems with this approach:

• The .NET OpenFileDialog Class only allows so many files to be selected before it errors
• Loading folders would be easier than selecting tons of files

note: If anyone can offer a work around for the "to many files selected" error feel free to contact me.

The solution for both problems was to use a FolderBroswer Dialog AND a OpenFileDialog so that if a user wanted to load lots of files quickly, adding the folder would be most efficient way. On the other hand if they only want to add one or two files, they can then use the OpenFileDialog for that purpose. The idea managed to work except that adding a FolderBroswer Dialog added problems of its own: When users select a folder should 2Backup add just the files within that folder or does it traverse through all sub-folders and add any file it finds?

My solution was the void GetFiles(String* directoryData[], bool subdirsFlag) function with parameters:

• String* directoryData[] - a String Array of all files/folders/data
• bool subdirsFlag - a boolean flag indicating if sub-directories should be analyzed

The code is as follows:

GetFiles(String* directoryData[], bool subdirsFlag)
{
    //To guarantee thread safety, lock the ArrayLists
    Monitor::Enter(syncfileList);
    Monitor::Enter(syncfolderList);

    int droppedfoldersIndex = 0;
    String* initalFolder;
    copy = false;

    //To remember the starting point of the mainloop
    if(directoryData->Length == 0)
    {
        initalFolder = "DONE";
    }
    else
    {
        initalFolder = directoryData[droppedfoldersIndex];
    }

    //Initialize mainloop and UpdateUI variables
    complete = false;
    hasErrored = false;
    int folderarrayLength = directoryData->Length;
    int loopIndex = syncfolderList->Count;
    String* currentItem;

    //Begin mainloop
    while(droppedfoldersIndex < folderarrayLength)
    {
        //Starting point
        currentItem = directoryData[droppedfoldersIndex];

        //Add 1 to folderCount if the currentItem is a folder
        if(currentItem->LastIndexOf(".") == -1)
        {
            folderCount++;
        }

        //Initialize currentItemInfo loop variables
        bool loopComplete = false;
        int folderIndex = 0;
        
        //Begin currentItemInfo loop
        while(loopComplete == false)
        {
            try
            {
                //CASE 1: currentItem is a directory
                if (Directory::Exists(currentItem)) //dirInfo->Exists
                {
                    //Get files and folders inside currentItem
                    String* files[] = Directory::GetFiles(currentItem);
                    String* subDirs[] = 
                                     Directory::GetDirectories(currentItem);
                    int subDirsLength = subDirs->Length;
                    int filesLength = files->Length;

                    //Add files/subDirs into a particular ArrayList 
                    //depending on which case you have
                    if(folderIndex <= subDirsLength)
                    {
                                                            
                        //CASE 1: subdirsFlag is set false;
                        //Add only the files of initalDirectory
                        if(subdirsFlag == false)
                        {
                            syncfileList->AddRange(static_cast<ICollection*>
                                                    (files->SyncRoot));

                            //File Count
                            fileCount = fileCount + filesLength;
                            loopComplete = true;
                        }
                        //CASE 2: 0 files and folders
                        else if(filesLength == 0 && subDirsLength == 0)
                        {
                            //File and Folder Count
                            fileCount = fileCount + filesLength;
                            folderCount = folderCount + subDirsLength;
                            folderIndex++;
                        }

                        //CASE 3: Only files
                        else if(filesLength > 0 && subDirsLength == 0)
                        {
                            //Add files
                            syncfileList->AddRange(static_cast<ICollection*>
                                                     (files->SyncRoot));

                            //File Count
                            fileCount = fileCount + filesLength;
                            folderIndex++;
                        }

                        //CASE 4: Only folders
                        else if(filesLength == 0 && subDirsLength > 0)
                        {
                            //Add folders of current directory
                            syncfolderList->AddRange(static_cast<ICollection*>
                                                      (subDirs->SyncRoot));

                            //Folder Count
                            folderCount = folderCount + subDirsLength;
                            folderIndex++;
                        }

                        //CASE 5: Both Files and Folders
                        else if(filesLength > 0 && subDirsLength > 0)
                        {
                            //Add files    
                            syncfileList->AddRange(static_cast<ICollection*>
                                                     (files->SyncRoot));

                            //Add folders of current directory        
                            syncfolderList->AddRange(static_cast<ICollection*>
                                                      (subDirs->SyncRoot));

                            //File and Folder Count
                            fileCount = fileCount + filesLength;
                            folderCount = folderCount + subDirsLength;
                            folderIndex++;
                        }
                    }

                    delete [] files;
                    delete [] subDirs;
                    
                    //If there are more folders in syncfolderList that 
                    // haven't been checked, move to the next folder in the 
                    // list
                    if(syncfolderList->Count-1 >= loopIndex)
                    {
                        currentItem = static_cast<String*>
                                      (syncfolderList->get_Item(loopIndex));
                        loopIndex++;
                        folderIndex = 0;    
                    }
                    else
                    {
                        //When all folders in syncfolderList have been 
                        // checked
                        loopComplete = true;
                    }            
                }
                //CASE 2: currentItem is a file
                else if(File::Exists(currentItem))//filInfo->Exists
                {
                    //Add file to syncfileList
                    syncfileList->Add(currentItem);
                    fileCount++;
                    loopComplete = true;
                }
                //CASE 3: there was an error
                else
                {
                   MessageBox::Show("There was an error loading your FileList!",
                                     "Files Not Added!", MessageBoxButtons::OK,
                                                      MessageBoxIcon::Warning);
                   syncfileList->Clear();
                   syncfolderList->Clear();
                   fileCount = 0;
                   folderCount = 0;
                   droppedfoldersIndex = folderarrayLength;
                   loopComplete = true;
                   hasErrored = true;    
                }
            }
            catch(System::Exception* ex)
            {
                MessageBox::Show(ex->Message, "Warning!", 
                                 MessageBoxButtons::OK, 
                                 MessageBoxIcon::Warning);

                if(MessageBox::Show("Would you like to continue?","Continue?",
                           MessageBoxButtons::YesNo,
                           MessageBoxIcon::Question) == DialogResult::Yes)
                {
                    currentItem = static_cast<String*>(syncfolderList->get_Item
                                                           (loopIndex));
                    loopIndex++;
                    folderIndex = 0;
                }
                else
                {
                    loopComplete = true;
                    hasErrored = true;
                    droppedfoldersIndex = directoryData->Length;
                }
            }
            //UpdateUI
            ShowProgress();
        }
        droppedfoldersIndex++; 
    }

    syncfileList->TrimToSize();
    syncfolderList->TrimToSize();

    //Unlock the ArrayLists when done
    Monitor::Exit(syncfileList);
    Monitor::Exit(syncfolderList);
    
    complete = true;

    //Return results to listbox in GUI
    Display(directoryData);
}

Yet another issue that popped up was how to overcome the unusable GUI when working with lots of files/folders. This was a problem because the project needed to flexible enough to allow both light, and heavy users to make Backup's without slow downs.My solution, after quite a bit of research, was to call void GetFiles(String* directoryData[], bool subdirsFlag) asynchronously as to keep the GUI responsive.

I decided to use Microsoft's asynchronous programming approach as seen here:

• Define a delegate with the same signature as the method you want to call
  • the common language runtime automatically defines BeginInvoke and EndInvoke methods for this delegate, with the appropriate signatures.
• The BeginInvoke method is used to initiate the asynchronous call.
  • It has the same parameters as the method you want to execute asynchronously, plus an instance of the delegate you just created
• BeginInvoke returns immediately and does not wait for the asynchronous call to complete
• The EndInvoke method is used to retrieve the results of the asynchronous call.

Note 1: Always call EndInvoke after your asynchronous call completes.

Note 2: The details of how I implemented GetFiles asynchronously are below in the using the code section.

Using the code

The semi-completion of 2Backup was an accomplishment although their were a few problems with my overall approach:

• Windows XP Professional already provides a backup program; why recreate the wheel ?
• 2Backup has limited schedule backup support; is high user interaction necessary ?
• The code doesn't follow an object oriented approach; code is hard to read and hard to re-use in similar projects.

My solution was to rewrite the code using many of the same functions just in a more object oriented approach allowing integration into a user control as simple as possible. Although I will include my 2Backup code for reference, the rest of this article's focus is on my improved code for a user control that wraps the same functionality. With 2Backup behind me, the first thing I wanted to do was create a Managed C++ Class that inherited from Microsoft's Control Class. My goal this time around was to take all the good out of 2Backup and streamline it into an easy to use User Control for manipulating data. By doing so:

• The amount of code is reduced
• The code is easier to read
• The code is easier to use

Here is the blue print of the class:

public __gc class DirectoryList : public Control

DirectoryList's Public Properties and Functions are listed below:

• Public Properties
  • FileCount - Returns File count
  • FolderCount - Returns Folder count
  • Items - Returns a reference to the DirectoryList's internal ListBox ObjectCollection
  • ShowProgressBar Sets or Returns a boolean flag to show or hide the DirectoryList's internal Listbox progressPanel
• Public Functions
  • void Build(String* directoryData[], bool subdirsFlag) - Starting point; Builds any files/folders added into the DirectoryList
  • void Copy(String* destinationPath,bool overwrite,bool cdBackup, bool consolidate, bool directorCopy) - Starts the file copy
  • void Deserialize(String* filename) - Reads a binary file with all saved file and folder info
  • void Remove() - Removes selected items from the DirectoryList
  • void Serialize(String* filename) - Creates a binary file of all files and folders
  • void Sort() - Sorts all files and folders by last write time

he starting point for any data manipulation within a DirectoryList is the Build Function

void DirectoryList::Build(String* directoryData[], bool subdirsFlag)
{
    //Disable listbox
    listbox->Enabled = false;
    Cursor = Cursors::WaitCursor;
    progressPanel->ProgressBar->Value = 0;
    progressPanel->ProgressBar->Visible = true;
    buildTime = DateTime::Now;

    GetFilesDelegate* getFilesDelegate = new GetFilesDelegate(this,GetFiles);
    getFilesDelegate->BeginInvoke(directoryData,subdirsFlag,
                  new AsyncCallback(this,GetFilesCallback),getFilesDelegate);
}

Using the same asynchronous approach as mentioned above, the Build function creates a Delegate named getFilesDelegate with the following parameters:

• this - a pointer to the DirectoryList itself
• GetFiles - the address to the private void GetFiles(String* directoryData[], bool subdirsFlag) function.

By calling BeginInvoke through our newly created getFilesDelegate with parameters:

• String* directoryData[] - a String Array of all the files and folders
• bool subdirsFlag - a boolean flag that decides whether to include sub-directories with the build
• void GetFilesCallback(IAsyncResult* ar) - An AsyncCallBack Delegate with the parameter IAsyncResult; used to call EndInvoke on the getFilesDelegate's asynchronous call to GetFiles.

we accomplish two things:

• The GetFiles function can run asynchronous allowing the GUI to stay responsive
• The GetFilesCallback function can be called to call EndInvoke on our getFilesDelegate

Here is the callback code when GetFiles completes:

void DirectoryList::GetFilesCallback(IAsyncResult* ar)
{
    //Get Delegate
    GetFilesDelegate* getFilesDelegate = 
                         static_cast<GetFilesDelegate*>(ar->AsyncState);

    //Always call EndInvoke
    getFilesDelegate->EndInvoke(ar);
}

I know what your thinking... less code!? ...easier to read!? alas this is just the code within the class. Now lets see what YOU as the user will do:

• Either add the compiled DirectoryList.dll to your toolbox and drag and drop the control into a new Windows Project
• Or create a new instance manually

private: System::Void btnFolder_Click(System::Object *  sender, 
                                      System::EventArgs *  e)
{
    //Normally you would declare your DirectoryList Globally like other 
        //controls but so you can see what I'm doing I declare it locally

    DirectoryList *  myList;

    FolderBrowserDialog* myFolder = new FolderBrowserDialog();
    if(myFolder->ShowDialog() == DialogResult::Cancel)
    {
        myFolder->SelectedPath = "";
    }
    else
    { 
        String* data[] = { myFolder->SelectedPath };
        myList->Build(data,cboxSubdirs->Checked);
    }
    myFolder->Dispose();
}

See the beauty of an Object Oriented approach! All you need to do is call Build and give it the parameters it needs: the string data of all files and folders and a boolean indicating whether to build sub-directories.

This approach is reproduced similarly with the Copy and Sort functions.

private: System::Void btnCopy_Click_1(System::Object *  sender, 
                                      System::EventArgs *  e)
{
    //Normally you would declare your DirectoryList Globally like other 
        //controls but so you can see what I'm doing I declare it locally

    DirectoryList *  myList;

    myList->Copy(textBox1->Text,cboxOverwrite->Checked,
                     rbtnCDBackup->Checked, rbtnConsolidateFiles->Checked,
                     rbtnDirectoryCopy->Checked);
}

private: System::Void btnSort_Click(System::Object *  sender, 
                                    System::EventArgs *  e)
{
    //Normally you would declare your DirectoryList Globally like other 
        //controls but so you can see what I'm doing I declare it locally
             
    DirectoryList *  myList;

    myList->Sort();
}

Points of Interest

As written by Microsoft, when working with multiple threads, the only way to return the DirectoryList's Build results is through a cross-thread call - that is, by calling Invoke or BeginInvoke to marshal the GetFiles function to the creation thread of your DirectoryList. This is done as follows with the private void Display(String* allData[]) function:

void DirectoryList::Display(String* allData[])
{
    if(listbox->InvokeRequired == true)
    {
        Object* pList[] = { allData };
        DisplayDelegate* displayDelegate = new DisplayDelegate(this,Display);

        //Note: Because you are passing immutable objects into this invoke 
        //method, you do not have to wait for it to finish by calling EndInvoke
    
        this->BeginInvoke(displayDelegate, pList);
    }
    else
    {
        if(hasErrored == true)
        {
            Cursor = Cursors::Default;
            progressPanel->ProgressBar->Visible = false;
            hasErrored = false;
        }
        else if(copy == true || remove == true)
        {
            copy = false;
            remove = false;
            listbox->Items->Clear();
            listbox->Items->AddRange(allData);
            Cursor = Cursors::Default;
            progressPanel->ProgressBar->Visible = false;
        }
        else
        {
            listbox->Items->AddRange(allData);
            Cursor = Cursors::Default;
            progressPanel->ProgressBar->Visible = false;
        }
        listbox->Enabled = true;
        filesPanel->Text = String::Concat("Total Files: ",
                                          Convert::ToString(fileCount));
        foldersPanel->Text = String::Concat("Total Folders: ",
                                            Convert::ToString(folderCount));
    }
}

First I create a instance of a DisplayDelegate with the following parameters:

• this - a pointer to the DirectoryList itself
• GetFiles - the address to the private void Display(String* allData[]) function.

By calling BeginInvoke through our newly created displayDelegate with parameters:

• displayDelegate - a instance of a DisplayDelegate
• Object* pList[] - a parameter list of all data

we accomplish two things:

• The Display function returns the Build results to the internal ListBox
• Does so in a thread-safe manner

Another point of interest is in the void ShowProgress() function. ShowProgress visually updates the GUI with a StatusBarProgressPanel and has a similar pattern as above:

void DirectoryList::ShowProgress()
{
    if(InvokeRequired)
    {
        //Note: Because you are passing immutable objects into this invoke 
        //method, you do not have to wait for it to finish by calling 
        //EndInvoke

        IAsyncResult* ar = this->BeginInvoke(showProgressDelegate);
    }
    else
    {
        if(copy == false || remove == true)
        {
            if (progressPanel->ProgressBar->Value == 
                progressPanel->ProgressBar->Maximum)
            {
                progressPanel->ProgressBar->Value = 0;
                progressPanel->ProgressBar->Maximum = 500; //3000
            }
        }
        else
        {
            if (progressPanel->ProgressBar->Value == 
                progressPanel->ProgressBar->Maximum)
            {
                progressPanel->ProgressBar->Visible = false;
            }
        }
        progressPanel->ProgressBar->PerformStep();
    }
}

I add a StatusBarProgressPanel to an internal StatusBar so as to keep track of progress and keep the user informed on the amount of files and folders. Through the void InitializeControls(void) function I add both the StatusBarProgressPanel and other panels as follows:

void DirectoryList::InitializeControls(void)
{
    listbox = new System::Windows::Forms::ListBox();
    listbox->Dock = DockStyle::Fill;
    listbox->HorizontalScrollbar = true;
    listbox->SelectionMode = SelectionMode::MultiExtended;
    listbox->AllowDrop = false;
    statusbar = new StatusBar();
    statusbar->Dock = DockStyle::Bottom;
    statusbar->ShowPanels = true;
    statusbar->SizingGrip = false;

    //Namespace.ResourceFiles 
    resources = 
         new System::Resources::ResourceManager("DirectoryList.ResourceFiles",
                                                GetType()->Assembly); 

    filesPanel = new StatusBarPanel();
    filesPanel->AutoSize = 
                 System::Windows::Forms::StatusBarPanelAutoSize::Contents;
    filesPanel->Text = S"Files : 0";
    filesPanel->Icon = static_cast(resources->GetObject("documents.ico"));

    foldersPanel = new StatusBarPanel();
    foldersPanel->AutoSize = 
                   System::Windows::Forms::StatusBarPanelAutoSize::Contents;
    foldersPanel->Width = 110;
    foldersPanel->Text = S"Folders : 0";
    foldersPanel->Icon = static_cast(resources->GetObject("folder.ico"));

    progressPanel = new MarkHarmon::Controls::StatusBarProgressPanel();
    statusbar->DrawItem += new StatusBarDrawItemEventHandler(
        this->progressPanel,
        &StatusBarProgressPanel::ParentDrawItemHandler);
    progressPanel->AutoSize = 
                     System::Windows::Forms::StatusBarPanelAutoSize::Spring;
    progressPanel->ProgressBar->Maximum = 0;
    progressPanel->ProgressBar->Value = 0;
    progressPanel->ProgressBar->Step = 1;
    progressPanel->ProgressBar->Visible = false;

    StatusBarPanel* panels[] = { filesPanel, foldersPanel, progressPanel };    
    statusbar->Panels->AddRange(panels);
    
    Control* temp[] = {listbox,statusbar};
    Controls->AddRange(temp);

}

Notice the use of Microsoft's ResourceManager class. This is how I embed the two icons for files and folders on the StatusBar.The only way I could embed the icons was with Resourcer, a program I found online. All you do is add the icons you want to embed and save it as a ResX file. Then, using an instance of Microsoft's ResourceManager class you call the GetObject function and use the full icon filename as its parameters.

Another point of interest is within the private void SortFiles() function.

void DirectoryList::SortFiles()
{
    int count = fileCount;
    FileInfo* files[] = new FileInfo*[count];
    int index = 0;

    while(index < count)
    {
        files[index] = new FileInfo(static_cast<string*>
                                        (syncfileList->get_Item(index)));
        index++;
    }
    //Sort files by last write time
    Array::Sort(files,(new CompareFileInfo()));
    
    Monitor::Enter(syncfileList);
    syncfileList->RemoveRange(0,count);
    index = 0;
    while(index < count)
    {
        syncfileList->Add(files[index]->FullName);
        index++;
    }
    Monitor::Exit(syncfileList);
    delete [] files;
}

As seen here I used an internal class called CompareFileInfo thats inherits from Microsoft's IComparer interface:

__gc class CompareFileInfo : public IComparer
        {
        public:
        int Compare(Object* x, Object* y)
            {
                FileInfo* file = static_cast<fileinfo* />(x);
                FileInfo* file2 = static_cast<fileinfo* />(y);
                return DateTime::Compare(file->LastWriteTime, 
                                         file2->LastWriteTime);
            }
        };

which simply creates an array of FileInfo instances from all the files in the DirectoryList's internal ArrayList and sorts them with this syntax:

• Array::Sort(files,(new CompareFileInfo()));

note: this sorts by last access time, but other options are available.

My last point of interest is Drag N Drop support. Although not directly related to the DirectoryList class, here's how I added Dran N Drop support for the DirectoryList Demo. First I create a DragEnter event and assign the effect property equal to a FileDrop.

private: System::Void myList_DragEnter(System::Object *  sender, 
                               System::Windows::Forms::DragEventArgs *  e)
         {
             //Enable Drag and Drop support
             if (e->Data->GetDataPresent(DataFormats::FileDrop))
             {
                 e->Effect = DragDropEffects::Copy;
             }
         }

Then i simply create an array from the GetData function and pass it to the Build function. I also added support for a dropped fileList.

private: System::Void myList_DragDrop(System::Object *  sender, 
                               System::Windows::Forms::DragEventArgs *  e)
{
    String* fileDropArray[];
    fileDropArray = static_cast<string*[]>
                              (e->Data->GetData(DataFormats::FileDrop));

    if(fileDropArray[0]->IndexOf(".lst") > 0)
    {
        myList->Deserialize(fileDropArray[0]);
    }
    else
    {
        myList->Build(fileDropArray,cboxSubdirs->Checked);
    }
}

Conclusion

Sometimes when searching for answers all one finds is more questions. My journey through the .NET framework has been both fun and exciting allowing me to explore the many different angles that arise when working with data. As I hope you can see, using the .NET framework for your data manipulation needs is a real treat for anyone who can harness the tools that are available. Coding in a object oriented fashion allowed me to simplify my code ultimately allowing easier integration into a user control. The DirectoryList is a user control that allows one to manipulate data to their liking in a easy to use package. My only hope is that everyone can learn, and benefit from the work done here. Their is still much to learn, and much to add upon, including fixing bugs, and adding new features so I leave you with my plans for the future:

• finish an updated C++/CLI version for .NET 2.0 (I'm almost done!)
• Create a C# version
• fix any bugs that are discovered; I'm sure there's quite a few

Thank you all for reading my first article here at The Code Project!

Sources

• http://msdn.microsoft.com/msdnmag/issues/03/02/Multithreading/default.aspx
• http://msdn.microsoft.com/msdnmag/issues/01/08/Async/
• http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dv_vstechart/html/datastructures_guide.asp
• http://www.yoda.arachsys.com/csharp/threads/
• http://www.devx.com/dotnet/Article/16099/0
• http://www.informit.com/guides/content.asp?g=dotnet&seqNum=125&rl=1
• http://www.codeproject.com/dotnet/embeddedresources.asp
• http://www.codeproject.com/cs/algorithms/Beginners_Sort.asp
• http://www.codeproject.com/csharp/AsyncMethodInvocation.asp