0

This application seems to eat a lot of memory. Maybe I'm not managing the memory properly. Can some please figure out what code needs to be optimized. And secondly, file copying is not working properly. It sometimes throws an exception, cannot access file being used by another process, this exception is being thrown at both ends the watch folder and the destination folder where I'm copying. Let me give you a brief of what I'm trying to achieve here.

I have a system that would give me an xml file in ansi format. This file would be updated regularly maybe every 3-4mins sometimes even 10-20secs. Now I'm watching this folder, as soon as it is changed I convert it to UTF-8 and copy this to another server via sftp. This sftp folder is mapped on this same machine where the conversion is happening. So the problems I'm facing is the exception it throws, can't access the file being used by another process, after a while this get's cleared. And even the memory exception, that the system is run out of memory. It's leaking memory as well. Starts at 5k, after a few hours reaches 1.2gb of memory usage. Now I need to run 3 of these similar programs watching 3 different folders. Any clues to my problems ?

class Test
{
    class Class1
    {
        private static FileSystemWatcher watcher = new FileSystemWatcher();

        public static void Main()
        {
            WatchFile();
            Console.ReadLine();
        }

        private static void WatchFile()
        {
            watcher.Path = @"c:\test";
            watcher.NotifyFilter = NotifyFilters.LastWrite;
            watcher.Filter = "*.xml";
            watcher.Changed += new FileSystemEventHandler(convert);
            watcher.Error += new ErrorEventHandler(WatcherError);
            watcher.EnableRaisingEvents = true;

            Console.WriteLine("Press \'q\' to quit.");
            Console.WriteLine("Press \'q\' to quit.");
            while (Console.Read() != 'q') ;
        }

        public static string CrL = "\r\n";

        private static void convert(object source, FileSystemEventArgs f)
        {
           string FileName = f.FullPath;
           string destinationFile = @"z:\xml\test.xml";

           Thread.Sleep(2000);
           try
           {
                watcher.EnableRaisingEvents = false;
                Encoding utf8 = new UTF8Encoding(false);
                Encoding ansi = Encoding.GetEncoding(1256);
                Thread.Sleep(2000);

                string xml = File.ReadAllText(FileName, ansi);
                XDocument xmlDoc = XDocument.Parse(xml);
                File.WriteAllText(FileName, @"<?xml version=""1.0"" encoding=""utf-8""?>" + xmlDoc.ToString(), utf8);

                 if (File.Exists(destinationFile))
                     File.Delete(destinationFile);
                 File.Copy(FileName, destinationFile,true);
                 Console.WriteLine("File Copied"); // for troubleshoooting only
                 Console.Write(CrL);
            }
            catch (Exception e)
            {
                Console.WriteLine("The process failed: {0}", e.ToString());
            }
            finally
            {
                watcher.EnableRaisingEvents = true;
            }
        }

        private static void WatcherError(object source, ErrorEventArgs e)
        {
            Exception watchException = e.GetException();
            watcher = new FileSystemWatcher();
            while (!watcher.EnableRaisingEvents)
            {
                try
                {
                    WatchFile();
                    Console.WriteLine("I'm Back!!");
                }
                catch
                {
                    Thread.Sleep(2000);
                }
            }
        }
    }  
}
6
  • What for all this magic numbers in Thread.Sleep()? Jan 16, 2013 at 4:14
  • -1 for "Please debug my application for me" 'question'
    – RJ Lohan
    Jan 16, 2013 at 4:16
  • Thanks, I just asked for some clues as I'm not able to figure this out.
    – user726720
    Jan 16, 2013 at 4:20
  • @abatishchev: Just to wait for a while before the file get's released by other processes to avoid the exception, yet I'm getting the file copy exceptions.
    – user726720
    Jan 16, 2013 at 4:22
  • Try this stackoverflow.com/a/11060322/495455 if you want to see if the file is being used (locked) by another process. Jan 16, 2013 at 5:10

1 Answer 1

2

Every time you get an error, you call WatchFile() again, which adds the convert and WatcherError methods to the Changed and Error invocation lists again. That would explain a slow memory leak. The delegate invocation lists keep growing.

Because the events are raised on pool threads, it's possible for your code to be processing multiple changed events concurrently.

Your error handler should just re-enable the events (i.e. Watcher.EnableRaisingEvents = true;) It certainly shouldn't be adding event handlers again.

You also need to synchronize access in your convert method. You could do it with a lock, but probably a better idea would be with a Monitor.TryEnter, like this:

private static object lockObject = new Object();
private static void convert(object source, FileSystemEventArgs f)
{
    if (!Monitor.TryEnter(lockObject))
    {
        // unable to get lock, return.
        return;
    }
    try
    {
        // do stuff here
    }
    finally
    {
        Monitor.Exit(lockObject);
    }
6
  • Thank you, I understood the code above and am trying this at the moment. But I can't understand 'Your error handler should just re-enable the events (i.e. Watcher.EnableRaisingEvents = true;) It certainly shouldn't be adding event handlers again.' Can you please demonstrate this. Thank you
    – user726720
    Jan 16, 2013 at 4:56
  • @user726720: I misread your error handling code. I've modified my answer. Jan 16, 2013 at 5:08
  • Thank you, I just got confused for a while, and couldn't figure out whats wrong with my error handling, anyways. I have changed my code by adding the above technique you suggested. I'm monitoring this, and it looks good so far. I would report back if any trouble comes up.. Thank you.
    – user726720
    Jan 16, 2013 at 5:16
  • It's not giving me those exceptions any more after, but the memory leak still seems persistent. It keeps on growing in the task manager. Is there more clues to my memory leak problem
    – user726720
    Jan 16, 2013 at 5:34
  • @user726720: One thing in your error handler is that you're not calling Dispose on the watcher before you create a new one. That can cause a short-term resource problem, but I don't think it'd cause a memory leak. If you're not getting lots of errors anymore, then I don't see any way your code could be the cause of a memory leak. By the way, Task Manager is notoriously unreliable for diagnosing memory leaks. Search for [memory leak C#] here on SO for information about diagnosing memory leaks. Jan 16, 2013 at 14:18

Not the answer you're looking for? Browse other questions tagged or ask your own question.