3

I would like to poll a directory every 10 seconds to see if any files have been added or modified. If their are any changes within the 10 seconds I would like to have a set of all file paths that I can then pass to another method.

Problem

When a file is added it is instantly recognized and the addedFiles method is called. Instead I would be expecting it to wait 10 seconds and to call the addedFiles method with multiple files that have been found.

Example
I've created a complete example that watches a directory. A thread then waits 5 seconds and copies 2000 files into the watched directory.
The expected behavior is for the WatchService to check for changes every 10 seconds. Instead it seems to be picking up changes instantly.

Code

import java.io.IOException;
import java.io.PrintWriter;
import java.nio.file.FileSystems;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardWatchEventKinds;
import java.nio.file.WatchEvent;
import java.nio.file.WatchKey;
import java.nio.file.WatchService;
import java.util.Collection;
import java.util.HashSet;
import java.util.concurrent.TimeUnit;

public class DirectoryWatcherExample 
{
    private static final int POLLING_TIME = 10;

    public static void main(final String args[]) throws InterruptedException, IOException
    {
        final Path directory = Paths.get("directory/to/be/watched");

        /**
         * Start a thread that will create 2000 files to the selected directory
         * This will occur after waiting 5 seconds.
         */
        new Thread(new Runnable()
        {
            @Override
            public void run() 
            {
                try 
                {
                    Thread.sleep(5000);         
                    System.out.println("Copying 2000 files to directory: " + directory);
                    for(int i = 0; i < 2000; i++)
                    {
                        final PrintWriter writer = new PrintWriter(directory.resolve("test_file_" + i + ".txt").toFile(), "UTF-8");
                        writer.println("The first line");
                        writer.println("The second line");
                        writer.close();
                    }
                    System.out.println("Finished copying files to directory: " + directory);
                } 
                catch (final Exception e) 
                {
                    e.printStackTrace();
                } 
            }
        }).start();

        /**
         * Start the watch service polling every 10 seconds
         */
        new DirectoryWatcherExample().startWatchService(directory);
    }

    public void startWatchService(final Path directory) throws InterruptedException, IOException
    {
        final WatchService watchService = FileSystems.getDefault().newWatchService();
        directory.register(watchService, StandardWatchEventKinds.ENTRY_CREATE, StandardWatchEventKinds.ENTRY_MODIFY);

        while(true)
        {
            System.out.println("Start polling");
            final WatchKey key = watchService.poll(POLLING_TIME, TimeUnit.SECONDS);
            System.out.println("Finished polling and retrieved key");

            if(key != null)
            {
                final Collection<Path> paths = new HashSet<>();
                for (final WatchEvent<?> watchEvent : key.pollEvents())
                {
                    final Path path = ((Path) key.watchable()).resolve((Path) watchEvent.context());
                    paths.add(path);
                    System.out.println("Path added: " + path);
                }

                // Do something with the paths
                addedFiles(paths);

                if (!key.reset())
                {
                    break;
                }   
            }

        }
    }

    // Unimplemented
    public void addedFiles(final Collection<Path> paths)
    {

    }
}

What could be causing this?

3
  • Can you explain me why you have used Thread.sleep(5000)? Does it has to do with size of file, suppose we have a file thats good enough for 5000 ms sleep time, but what if a file that is of huge size comes up? Do we need more sleep time or is it only to stop the thread for few time before starting the process? Please explain I am facing similar problem Nov 18, 2020 at 8:25
  • @sh4r4d I created this question 2 years ago so I might be wrong. It looks like I make it sleep so that it has started the watch service before it starts adding the files. I likely could have avoided the sleep and just put the start watch service method above the thread code.
    – Michael
    Nov 18, 2020 at 9:02
  • So it Sleep is not for the size of file, instead it is for starts watch service before adding files? I am facing same issue, when I am creating a new file, it should call CREATE but even before CREATE is getting finished, MODIFY is being triggered. Nov 18, 2020 at 9:31

2 Answers 2

2

There are two options:

  1. You need to invoke poll on the watchService after a certain interval by sleeping in between. As others pointed out, the timeout in the poll method is for scenarios where no event is available in the buffer. Also, since you are not handling the events immediately, some of the events may overflow the operating system buffer and eventually lost. Therefore, you need to handle the overflow scenario as well.

  2. Alternatively, you may want to use Apache Commons IO File Monitoring library. It polls the file system as you want. You can even set the polling interval.

Refer to the following three class/interface here:

  • FileAlterationMonitor - It's basically a thread (a Runnable implementation) which sleeps for the polling interval and after every interval invokes FileAlterationObserver
  • FileAlterationObserver - It lists the files in the directory, compares the current list with the previous list, identifies the file changes and invokes the appropriate method in the FileAlterationListener implementation
  • FileAlterationListener - The interface that you need to implement and write your logic

What you may consider doing for your use case is, keep adding all the file details in a list as and when they are added or modified. Finally, when the onStop() method is invoked, you call your addedFiles method with the complete list, clear the list and start again.

1
  • This makes sense. Both great answers but you also explain alternatives. Thank you for the help!
    – Michael
    Nov 23, 2018 at 11:14
2

The timeout parameter in WatchService.poll(timeout, unit) is not for defining how long it has to delay. It only defines a maximum wait time (after that it returns whether an event has been detected or not.)

It still returns as soon is it has detected a change. Read the JavaDoc for WatchService.poll

Retrieves and removes the next watch key, waiting if necessary up to the specified wait time if none are yet present.

Nowhere is written that it will always wait that long.

2
  • I see. What would be the correct approach for polling every n seconds? Adding a Thread.sleep in the while loop? The main thing I wanted to avoid doing was listening to messages from the OS specifying if changes to the file system have occurred. I was thinking this would just keep a list of current files and compare every time it polls.
    – Michael
    Nov 22, 2018 at 17:22
  • @Michael the WatchService was specifically created so that java code can react to file system notifications from the OS. If you want to have a list of current files and compare them at specific intervals you would need to implement it yourself. Nov 22, 2018 at 17:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.