0

I have a wordCount(CharacterReader charReader) function which takes a stream of characters, converts them to words.

I also have a Collection<CharacterReader> characerReaders, containing multiple character streams. The number of readers in a collection can vary, I want to read from all streams and have a count of all words.

I'm a little confused about threads and couldn't find any examples which were similar to this. I essentially want multiple threads outputting their words into a SortedMap so I can have a real time total word count.

How would I go about doing this?

Thanks

1
  • Do you need the words themselves or just the counter? A synchronized counter should be easy to get examples of.
    – Thomas
    Jul 18, 2012 at 20:41

3 Answers 3

2
  1. If you are going to have multiple threads writing to the map, you need to use a ConcurrentSkipListMap which is both a SortedMap and a ConcurrentMap.
  2. You can create for each CharacterReader in the collection a Runnable which calls the wordCount function (which accesses the map described previously).
  3. After creating the Runnables you can create an ExecutorService (for example using Executors.newCacheThreadPool()), pass it all the Runnables and wait for them to finish (see the example in the javadoc of class ExecutorService).

You can also create the Runnables just before sending them to the ExecutorService.

0
1

Create a WordMap class which encapsulates your sorted map, and makes sure all the accesses to the map are properly synchronized. Or use a concurrent map that is already thread safe.

Create an instance of this class. Use the Executors class to create an ExecutorService with the characteristics that you desire.

Then iterate through the collection, and for each reader, create a Callable or a Runnable filling the WordMap instance with the words found in this reader, and submit this Callable or Runnable to the ExecutorService.

1

vainolo and JB's answers are both good.

I will add one thing, which is a description of how to make a highly concurrent data structure to store your word counts.

As vainolo said, a ConcurrentSkipListMap is the basic data structure you want, because it is both sorted and concurrent. To make good use if it, you want to avoid doing any locking. That means you must avoid patterns which involve a lock-read-write-unlock cycle. That has two consequences: firstly, putting a new word in the map should not involve a lock, and incrementing the count of an existing word should not involve a lock.

You can safely add new things to the map using ConcurrentMap's putIfAbsent method. However, that alone is not quite enough, because you have to supply a potential value every time you use it, which is potentially expensive. The easiest thing to do is to use a sort of double-checked locking pattern, where you first simply try to get an existing value, then if you find there isn't one, add a new one with putIfAbsent (you can't simply call put, because there could be a race between two threads putting at the same time).

Incrementing without locking can easily be done by not storing integers in the map, but rather objects which themselves contain integers. That way, you never have to put an incremented value in the map, you just increment the object already there. AtomicInteger seems like a good candidate for this.

Putting that together, you get:

public class WordCounts {
    private final ConcurrentMap<String, AtomicInteger> counts
         = new ConcurrentSkipListMap<String, AtomicInteger>();

    public void count(String word) {
        AtomicInteger count = getCount(word);
        count.incrementAndGet();
    }

    private AtomicInteger getCount(String word) {
        AtomicInteger count = counts.get(word);
        if (count == null) {
            AtomicInteger newCount = new AtomicInteger();
            count = counts.putIfAbsent(word, newCount);
            if (count == null) count = newCount;
        }
        return count;
    }
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.