11

I was wondering if it is possible to split a HashMap into smaller sub-maps.

In my case I have a HashMap of 100 elements and I would like to create 2 (or more) smaller HashMaps from the original one, the first containing the Entries from 0 to 49, the second containing the Entries from 50 to 99.

Map <Integer, Integer> bigMap = new HashMap <Integer, Integer>();

//should contains entries from 0 to 49 of 'bigMap'
Map <Integer, Integer> smallMap1 = new HashMap <Integer, Integer>(); 


//should contains entries from 50 to 99 of 'bigMap'
Map <Integer, Integer> smallMap2 = new HashMap <Integer, Integer>();

Any suggestions? Many thanks!

0

12 Answers 12

17

Do you have to use HashMap?

TreeMap is really good for this kind of things. Here's an example (note that 0, 50, and 99 are map keys, not indices):

TreeMap<Integer, Integer> sorted = new TreeMap<Integer, Integer>(bigMap);

SortedMap<Integer, Integer> zeroToFortyNine = sorted.subMap(0, 50); // toKey inclusive, fromKey exclusive
SortedMap<Integer, Integer> fiftyToNinetyNine = sorted.subMap(50, true, 99, true);
7
  • I don't know why you received some downvote, I have chosen your answer because it does not waste much memory. The TreeMap enabled me to efficiently achieve what I wanted. P.S. I should have specified that the result should have been performing, and your suggestion it was. Thanks again
    – RNO
    Commented Jan 31, 2013 at 23:02
  • In this answer's current state items 49 and 99 will not be included in either map, because the toKey argument of SortedMap's subMap method is exclusive. Why they didn't include an inclusive version I have no idea.
    – Shane
    Commented Feb 11, 2014 at 2:30
  • 1
    What you probably want is Java 6's NavigableMap, which includes subMap(K fromKey, boolean fromInclusive, K toKey, boolean toInclusive);
    – Shane
    Commented Feb 11, 2014 at 3:09
  • 3
    This answer will only work with a map having as keys the numbers from 0 to 99. The parameters of method subMap(a,b) does not refers to the number of elements but to the values of the keys.If your maps contains the 100 numbers from 500 to 599, subMap(0,49) will be empty, as well als subMap(50,99). The only full map would be subMap(100, 99999999). Commented Oct 24, 2014 at 14:46
  • 10
    This answer is confusing: it does not take the indices of entries into account but directly the keys. It can't be called a solution to split a map. Commented Jun 3, 2016 at 13:00
3

You'll basically need to iterate over the entries in bigMap, and make a decision as to whether they should be added to smallMap1 or smallMap2.

4
  • or set a threshold and iterate to it, moving an entry from big to small each iteration
    – amphibient
    Commented Jan 31, 2013 at 15:44
  • @foampile: A threshold? What do you mean? (Remember that a hash-map isn't sorted) Commented Jan 31, 2013 at 15:44
  • iterate up to a certain number of iterations, e.g. 50% rounded up to the next int of the original size. but even if it is not sorted, he is moving entries, i.e. putting them in new, deleting from old, so it is safe. so when he iterates again, the moved entries won't be there in the old
    – amphibient
    Commented Jan 31, 2013 at 15:45
  • @foampile: Oh, I see. I had misread the OP's question. I assumed that (s)he wanted to filter the entries based on their values. But it sounds like they actually just want 50% to go one way, and 50% the other. Commented Jan 31, 2013 at 15:46
3

As the HashMap is unordered (entries may come in any order), it makes no sense to exactly split it. We can simply use the alternating boolean flag.

boolean b = false;
for (Map.Entry e: bigMap.entrySet()) {
  if (b)
    smallMap1.put(e.getKey(), e.getValue());
  else
    smallMap2.put(e.getKey(), e.getValue());
  b = !b;
}
3
  • 2
    huh? on a HashMap wouldn't this just arbitrarily pick every other entry?
    – sharakan
    Commented Jan 31, 2013 at 15:46
  • There is no such thing as the order of entries in HashMap. If the order is important for your task, use LinkedHashMap and then of course it must be a different algorithm. Commented Jan 31, 2013 at 15:50
  • 1
    What if I don't consider ordering, its just that i want to split Map into smaller maps. How can I do it ?
    – vkrishna17
    Commented Jan 27, 2017 at 5:46
1

Iterate over the bigMap with for (Entry<Integer, Integer> entry : bigMap.entrySet()), and increment an i to check whether you have to add the entry in the first small map or in the second one.

1

Here is a solution with a SortedMap:

public static <K, V> List<SortedMap<K, V>> splitMap(final SortedMap<K, V> map, final int size) {
    List<K> keys = new ArrayList<>(map.keySet());
    List<SortedMap<K, V>> parts = new ArrayList<>();
    final int listSize = map.size();
    for (int i = 0; i < listSize; i += size) {
        if (i + size < listSize) {
            parts.add(map.subMap(keys.get(i), keys.get(i + size)));
        } else {
            parts.add(map.tailMap(keys.get(i)));
        }
    }
    return parts;
}
0
1

You can use Guava Iterables partition method and Java stream interface to solve it.

import com.google.common.collect.Iterables;
import com.google.common.collect.Lists;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;

public static <K, V> List<Map<K, V>> split(Map<K, V> map, int size) {
    List<List<Map.Entry<K, V>>> list = Lists.newArrayList(Iterables.partition(map.entrySet(), size));

    return list.stream()
            .map(entries ->
                    entries.stream().collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue))
            )
            .collect(Collectors.toList());
}    
1

Here are two simple methods to split the map by,

  1. size of the partition or

  2. number of partitions

     /**
      *
      * @param bulkyMap - your source map to be partitioned
      * @param batchSize - partition size
      * @return
      */
     public List<Map<String, Object>> getMiniMapsInFixedSizeBatches(Map<String, Object> bulkyMap, int batchSize) {
         if (batchSize >= bulkyMap.size() || batchSize <= 0) {
             return Arrays.asList(bulkyMap);
         }
         List<Map<String, Object>> batches = new ArrayList<>();
         int innerBatchcount = 1;
         int count = 1;
         Map<String, Object> tempMap = new HashMap<>();
         for (Map.Entry<String, Object> entry : bulkyMap.entrySet()) {
             tempMap.put(entry.getKey(), entry.getValue());
             innerBatchcount++;
             count++;
             if (innerBatchcount > batchSize || count > bulkyMap.size()) {
                 innerBatchcount = 1;
                 Map<String, Object> batchedMap = new HashMap<>();
                 batchedMap.putAll(tempMap);
                 batches.add(batchedMap);
                 tempMap.clear();
             }
         }
         return batches;
     }
    
     /**
      * the number of partitions is not always guaranteed as the algorithm tries to optimize the number of partitions
      * @param bulkyMap - your source map to be partitioned
      * @param numPartitions  - number of partitions (not guaranteed)
      * @return
      */
     public List<Map<String, Object>> getMiniPartitionedMaps(Map<String, Object> bulkyMap, int numPartitions) {
         int size = bulkyMap.size();
         int batchSize = Double.valueOf(Math.ceil(size * 1.0 / numPartitions)).intValue();
         return getMiniMapsInFixedSizeBatches(bulkyMap, batchSize);
     }
    
0
for (Map.Entry<Integer,Integer> entry : bigMap.entrySet()) {
   // ...
}

is the fastest way to iterate through your original map. You'd then use the Map.Entry key to decide which new map to populate.

0

This was one of the functions which did the work me, I hope its helpful for others. This one works irrespective of the Object/primitive stored as key.

The TreeMap approach suggested above would work only if the keys are primitives, ordered and in exact sequence of index..

    public List<Map<Integer, EnrichmentRecord>> splitMap(Map<Integer, EnrichmentRecord> enrichmentFieldsMap,
            int splitSize) {

        float mapSize = enrichmentFieldsMap.size();
        float splitFactorF = splitSize; 
        float actualNoOfBatches = (mapSize / splitFactorF);
        double noOfBatches = Math.ceil(actualNoOfBatches);



        List<Map<Integer, EnrichmentRecord>> listOfMaps = new ArrayList<>();

        List<List<Integer>> listOfListOfKeys = new ArrayList<>();


        int startIndex = 0;
        int endIndex = splitSize;

        Set<Integer> keys = enrichmentFieldsMap.keySet();
        List<Integer> keysAsList = new ArrayList<>();
        keysAsList.addAll(keys);

        /*
         * Split the keys as a list of keys,  
         * For each key sub list add to a Primary List - listOfListOfKeys
         */
        for (int i = 0; i < noOfBatches; i++) {
            listOfListOfKeys.add(keysAsList.subList(startIndex, endIndex));         
            startIndex = endIndex;
            endIndex = (int) (((endIndex + splitSize) > mapSize) ? mapSize : (endIndex + splitSize));
        }

         /**
         * For Each list of keys, prepare a map
         *
         **/
        for(List<Integer> keyList: listOfListOfKeys){
            Map<Integer,EnrichmentRecord> subMap = new HashMap<>();
            for(Integer key: keyList){
                subMap.put(key,enrichmentFieldsMap.get(key));
            }
            listOfMaps.add(subMap);
        }

        return listOfMaps;
    }
0

This Could be another solution by using headMap and tailMap

SortedMap<Integer, String> map1 = new TreeMap<>();
    map1.put(2, "Abc");
    map1.put(3, "def");
    map1.put(1, "xyz");
    map1.put(5, "mddf");
    System.out.println(map1);

    SortedMap<Integer, String> sm1 = map1.headMap(4); // from 0 to x (key) from front
    SortedMap<Integer, String> sm2 = map1.tailMap(1); //tail starting from key
    System.out.println("Head Map"+ sm1);
    System.out.println("Tail Map"+sm2);

Output was

{1=xyz, 2=Abc, 3=def, 5=mddf}
Head Map{1=xyz, 2=Abc, 3=def}
Tail Map{1=xyz, 2=Abc, 3=def, 5=mddf}
0

Building from @Nizamudeen Karimudeen's answer, which I could not get to work without a significant re-write ... this method works with any HashMap with any classes in it.

So, let's say that the Map you want to split is defined like this:

Map<String, MyClass> myMap = new HashMap<>();

And you wanted it split into 20 separate maps, you would simply split it like this:

List<Map<String, MyClass>> splitMapList = splitMap(myMap, 20);

Then to use each separate map, you can iterate through them like this:

for (Map<String, MyClass> mySplitMap : splitMapList) {
     for(String key : mySplitMap.keySet()) {
         MyClass myClass = mySplitMap.get(key);
     }
}

Or you could reference them directly by the index of the list etc.

Map<String, MyClass> subMap = splitMapList.get(3);

Here is the method:

public static List<Map<KeyClass, ValueClass>> splitMap(Map<KeyClass, ValueClass> originalMap, int splitSize) {
    int mapSize = originalMap.size();
    int elementsPerNewMap = mapSize / splitSize;
    List<Map<KeyClass, ValueClass>> newListOfMaps = new ArrayList<>(); //Will be returned at the end after it's built in the final loop.
    List<List<KeyClass>> listOfMapKeysForIndexing = new ArrayList<>(); //Used as a reference in the final loop.
    List<KeyClass> listOfAllKeys = new ArrayList<>(originalMap.keySet());
    int maxIndex = listOfAllKeys.size() - 1; //We use this in the first loop to make sure that we never exceed this index number or we will get an index out of range.
    int startIndex = 0;
    int endIndex = elementsPerNewMap;
    for (int i = 0; i < splitSize; i++) { //Each loop creates a new list of keys which will be the entire set for a new subset of maps (total number set by splitSize.
        listOfMapKeysForIndexing.add(listOfAllKeys.subList(startIndex, endIndex));
        startIndex = Math.min((endIndex + 1), maxIndex);//Start at the next index, but don't ever go past the maxIndex or we get an IndexOutOfRange Exception
        endIndex = Math.min((endIndex + elementsPerNewMap), maxIndex);//Same thing for the end index.
    }
    /*
     * This is where we use the listOfMapKeysForIndexing to create each new Map that we add to the final list.
     */
    for(List<KeyClass> keyList: listOfMapKeysForIndexing){
        Map<KeyClass,ValueClass> subMap = new HashMap<>(); //This should create a quantity of these equal to the splitSize.
        for(KeyClass key: keyList){
            subMap.put(key,originalMap.get(key));
        }
        newListOfMaps.add(subMap);
    }
    return newListOfMaps;
}
0

If you want to process in batch, try this :

SortedMap<String, Account> subTreeMap = new TreeMap<>();
List<SortedMap<String, Account>> sortedMapList = new ArrayList<>();
int i = 1;
String first`enter code here`Key = null;
String lastKey = null;
for (Map.Entry<String, Account> entry : treeMap.entrySet()) {
    if (i % MAX_NUM == 1) {
        firstKey = entry.getKey();
    }
    if (i % MAX_NUM == 0 || i % treeMap.size() == 0) {
        lastKey = entry.getKey();
        subTreeMap = treeMap.subMap(firstKey, true, lastKey, true);
        sortedMapList.add(subTreeMap);
    }
    i++;
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.