11

I am trying to prevent duplicate entries from being added to an ArrayList as the list is being populated whilst reading through each line of a file. Each line of the file is in the format "node1 node2" (tab delimiter). A duplicate here could either be "node1 node2" or "node2 node1". Here is my code to try and perform this operation:

while((line = bufferedReader.readLine()) != null) {

     String delimiter = "\t";
     String[] tempnodelist;  
     tempnodelist = line.split(delimiter);

     for (int i=0; i <= edgesnumber; i++) {   //edgesnumber = edges.size()

         if (tempnodelist[0] && tempnodelist[1] != edges.get(i)) {

             edges.add(line);

            }
        }

     nodes.add(tempnodelist[0]);  
     nodes.add(tempnodelist[1]); //intial intended use of tempnodelist.

}

Since I'm already splitting each line to make a HashSet of each node, I'm trying to use this to check for duplicates. At the moment I just can't seem to get the syntax right. How can I check through previous entries of the ArrayList for duplicates, and prevent them from being added, whist continuing to populate the ArrayList? what is wrong with this code currently?

Please ask any questions if anything is unclear,

Thanks in advance!

3
  • 5
    If you don't want duplicate entries, then you want a Set, not a List. Dec 16, 2013 at 13:10
  • But entries could be reversed, with would still count as a duplicate...Set wouldn't see this would it? Dec 16, 2013 at 13:15
  • Set uses equals() to find duplicates. You should create a specific class for the object you want to put in your set and redefine equals() on this class. Dec 16, 2013 at 13:19

6 Answers 6

17

Use a LinkedHashSet and then convert it to an ArrayList, because a LinkedHashSet has a predictable iteration order (the insertion-order) and it is a Set.

For example

LinkedHashSet<String> uniqueStrings = new LinkedHashSet<String>();

uniqueStrings.add("A");
uniqueStrings.add("B");
uniqueStrings.add("B");
uniqueStrings.add("C");
uniqueStrings.add("A");

List<String> asList = new ArrayList<String>(uniqueStrings);
System.out.println(asList);

will output

 [A, B, C]
3
  • I want to use this with ArrayList with class object, It did not worked.
    – Anant Shah
    Mar 10, 2017 at 6:09
  • @AnantShah You want unique Class objects in an ArrayList? Is this what you mean? And what did not work? When I try it, it works. I guess it is hard to help you using comments. Please ask a question and provide example code.
    – René Link
    Mar 10, 2017 at 6:32
  • I tried and it returned same size of arraylist which has duplicate class object data. But for that i override two methods equals() and hashCode() inside the class object file. and then it worked. Thank you for your solution and prompt reply. :)
    – Anant Shah
    Mar 10, 2017 at 7:48
2

If you want to keep the order of the lines read keep using the list but for the duplicates you can use a set for determining if a line (in its two forms as you described) was already added :

Set<String> duplicates = new HashSet<String>();
while((line = bufferedReader.readLine()) != null) {

     String delimiter = "\t";
     String[] tempnodelist;  
     tempnodelist = line.split(delimiter);

     String lineReversed = tempnodelist[1] + delimiter + tempnodelist[0];

     if (!duplicates.contains(line) && !duplicates.contains(lineReversed )) {
         edges.add(line);
     }
}
3
  • reverse() will turn "node2 node1" into "1edon 2edon" Shouldn't duplicates be outside the loop? Dec 16, 2013 at 13:18
  • For some reason this isn't preventing duplicates from being added. Inputting a txt of "123 234" and "234 123" still populated the ArrayList with both entries :/ Dec 16, 2013 at 14:05
  • 1
    @user2941526 my mistake, I initialized the set on each iteration. I moved it to the outside of the while loop
    – giorashc
    Dec 16, 2013 at 14:20
1

For each addition to the ArrayList you will have to iterate over all previous entries and check if duplicates entry exists(You can use .contains()) which is O(N).

Better I would suggest use a set.

0

Firstly, use equals to compare strings.

Secondly, you can use Set rather than a List

And lastly, you can use contains method to check if the item already exists.

0
ArrayList<String> ar=new ArrayList<String>();
String a[]={"cat","bat","cat","knife"};
for(int i=0;i<a.length;i++){
    if(!ar.contains(a[i])){
        ar.add(a[i]);
    }
}

Create an array list, and check whether it contains the string to be inserted. If it does not contain the string, then you can add it into the array list. This way you can avoid the duplicate entries in the array list.

The elements in the array list for the above code would be

cat bat knife

-1

It sounds like what you really want is a Set<Set<String>>

Set<Set<String>> pairs = ...
try(BufferedReader br = ... ) {
    for(String line; (line = br.readLine()) != null;) 
        pairs.add(new HashSet<String>(Arrays.asList(line.split(" ")));
}

This creates a collection of pairs without duplicates regardless of the order of the words.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.