How would you write a java function boolean sameContent(Path file1,Path file2)
which determines if the two given paths point to files which store the same content? Of course, first, I would check if the file sizes are the same. This is a necessary condition for storing the same content. But then I'd like to listen to your approaches. If the two files are stored on the same hard drive (like in most of my cases) it's probably not the best way to jump too many times between the two streams.
-
3The size can differ, also for same content. Depending on several factors. If you realy want to compare the content, then an easy check is to make a checksum of both files and compare them. You can use md5 on the bytearray of the files. Also an compare of the bytearrays can be used.– Rene M.Dec 9, 2014 at 12:36
-
@ReneM. I'm curious under what circumstances two files with the same byte content would have different sizes?– IntegrateThisNov 14, 2021 at 2:32
-
For example the block size and used partition format of the blockstorage can result in different byte sizes for the exact same content.– Rene M.Apr 14 at 10:28
9 Answers
Exactly what FileUtils.contentEquals
method of Apache commons IO does and api is here.
Try something like:
File file1 = new File("file1.txt");
File file2 = new File("file2.txt");
boolean isTwoEqual = FileUtils.contentEquals(file1, file2);
It does the following checks before actually doing the comparison:
- existence of both the files
- Both file's that are passed are to be of file type and not directory.
- length in bytes should not be the same.
- Both are different files and not one and the same.
- Then compare the contents.
-
2To add value, I found the
FileUtils.contentEqualsIgnoreEOL
may offer convenience for less stringent assertions. May 19, 2016 at 20:04 -
This is a great way of doing it, but if I want to match the contents of the file line by line, and then print the matching name and id each from the 2 files May 7, 2018 at 17:32
-
-
1Beware though that this is performance-wise probably not the optimal solution. commons-io (at least up to v2.5) uses IOUtils which in turn does a (byte-wise read)[github.com/apache/commons-io/blob/commons-io-2.5/src/main/java/… (at least on a buffered stream, though). If the difference is far down a large file, your performance might suffer.– SkyrAug 8, 2019 at 13:04
If you don't want to use any external libraries, then simply read the files into byte arrays and compare them (won't work pre Java-7):
byte[] f1 = Files.readAllBytes(file1);
byte[] f2 = Files.readAllBytes(file2);
by using Arrays.equals.
If the files are large, then instead of reading the entire files into arrays, you should use BufferedInputStream
and read the files chunk-by-chunk as explained here.
-
2I want my program also to work with big files. This could result in a OutOfMemoryError - if an array of the required size cannot be allocated, for example the file is larger that 2GB. Edit: Sorry, I just saw your remark about dealing with large files. Dec 9, 2014 at 13:01
-
2True. That is why I included a link to an SO page which mentions using BufferedInputStream and reading chunk-by-chunk, not the whole file. There's no point duplicating answers that are already present in SO. Dec 9, 2014 at 13:03
Since Java 12 there is method Files.mismatch which returns -1
if there is no mismatch in the content of the files. Thus the function would look like following:
private static boolean sameContent(Path file1, Path file2) throws IOException {
return Files.mismatch(file1, file2) == -1;
}
-
does it load the whole files in memory? I don't find any reference on google for it– AkinnSep 11, 2019 at 10:05
-
2
If the files are small, you can read both into the memory and compare the byte arrays.
If the files are not small, you can either compute the hashes of their content (e.g. MD5 or SHA-1) one after the other and compare the hashes (but this still leaves a very small chance of error), or you can compare their content but for this you still have to read the streams alternating.
Here is an example:
boolean sameContent(Path file1, Path file2) throws IOException {
final long size = Files.size(file1);
if (size != Files.size(file2))
return false;
if (size < 4096)
return Arrays.equals(Files.readAllBytes(file1), Files.readAllBytes(file2));
try (InputStream is1 = Files.newInputStream(file1);
InputStream is2 = Files.newInputStream(file2)) {
// Compare byte-by-byte.
// Note that this can be sped up drastically by reading large chunks
// (e.g. 16 KBs) but care must be taken as InputStream.read(byte[])
// does not neccessarily read a whole array!
int data;
while ((data = is1.read()) != -1)
if (data != is2.read())
return false;
}
return true;
}
-
Can't you just wrap the input streams in
BufferedInputStream
? Then the method would be as efficient as if you usedread(byte[])
but without the complexity, right?– aioobeJul 21, 2015 at 23:23 -
@aioobe Yes, we could. The reason why I used byte-by-byte comparison is because the
read(byte[])
method is not guaranteed to fully read the passed byte array (javadoc says "it reads up tobytes.length
"). If the source of the underlying stream is a file, current implementations will read the full array, but there is no guarantee for this. And the code which properly handles non-full array reads would be more complex and would draw attention from the principle my code snipplet is trying to show.– iczaJul 27, 2015 at 6:57 -
1I understand this, but what I'm trying to say is that you would get away without that complexity by using
BufferedInputStream
(while still achieve the efficiency).– aioobeJul 27, 2015 at 8:11 -
@aioobe Yes, you're right. A little background for other readers: the javadoc of
BufferedInputStream.read(byte[] b, int off, int len)
does state that it tries to read the full array. And althoughBufferedInputStream
does not overrideFilterInputStream.read(byte[] b)
, the javadoc ofFilterInputStream.read(byte[] b)
states that the implementation callsread(b, 0, b.length)
which in case of aBufferedInputStream
will be a call toBufferedInputStream.read(byte[] b, int off, int len)
method.– iczaJul 27, 2015 at 8:28 -
3I'm still not sure we're on the same page here. I'm thinking by simply changing
Files.newInputStream(file1)
tonew BufferedInputStream(Files.newInputStream(file1))
youris1.read()
call will correspond to a simply array access (in most cases) and the wholeread(byte[] ...)
stuff will be handled behind the curtains. So, I'm suggesting you could improve your answer by wrapping the input streams inBufferedInputStreams
and drop the comment on how it can be sped up at the cost of extra complexity.– aioobeJul 27, 2015 at 12:25
This should help you with your problem:
package test;
import java.io.File;
import java.io.IOException;
import org.apache.commons.io.FileUtils;
public class CompareFileContents {
public static void main(String[] args) throws IOException {
File file1 = new File("test1.txt");
File file2 = new File("test2.txt");
File file3 = new File("test3.txt");
boolean compare1and2 = FileUtils.contentEquals(file1, file2);
boolean compare2and3 = FileUtils.contentEquals(file2, file3);
boolean compare1and3 = FileUtils.contentEquals(file1, file3);
System.out.println("Are test1.txt and test2.txt the same? " + compare1and2);
System.out.println("Are test2.txt and test3.txt the same? " + compare2and3);
System.out.println("Are test1.txt and test3.txt the same? " + compare1and3);
}
}
If it for unit test, then AssertJ provides a method named hasSameContentAs. An example:
Assertions.assertThat(file1).hasSameContentAs(file2)
-
1
-
2
hasSameContentAs
method has been deprecated, please use instead hasSameBinaryContentAs:assertThat(file1).hasSameBinaryContentAs(file2);
Mar 7, 2021 at 9:40
I know I'm pretty late to the party on this one, but memory mapped IO is a pretty simple way to do this if you want to use straight Java APIs and no third party dependencies. It's only a few calls to open the files, map them, and then compare use ByteBuffer.equals(Object)
to compare the files.
This is probably going to give you the best performance if you expect the particular file to be large because you're offloading a majority of the IO legwork onto the OS and the otherwise highly optimized bits of the JVM (assuming you're using a decent JVM).
Straight from the FileChannel JavaDoc:
For most operating systems, mapping a file into memory is more expensive than reading or writing a few tens of kilobytes of data via the usual read and write methods. From the standpoint of performance it is generally only worth mapping relatively large files into memory.
import java.io.IOException;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;
public class MemoryMappedCompare {
public static boolean areFilesIdenticalMemoryMapped(final Path a, final Path b) throws IOException {
try (final FileChannel fca = FileChannel.open(a, StandardOpenOption.READ);
final FileChannel fcb = FileChannel.open(b, StandardOpenOption.READ)) {
final MappedByteBuffer mbba = fca.map(FileChannel.MapMode.READ_ONLY, 0, fca.size());
final MappedByteBuffer mbbb = fcb.map(FileChannel.MapMode.READ_ONLY, 0, fcb.size());
return mbba.equals(mbbb);
}
}
}
-
I've just discovered that map can only do files up to Integer.MAX_VALUE bytes, about 2Gb. Sep 28, 2021 at 12:19
-
I ended up comparing files in chunks using the position and length fields of
map()
. Sep 28, 2021 at 13:53
It's >=JR6 compatible, library-free and don't read all content at time.
public static boolean sameFile(File a, File b) {
if (a == null || b == null) {
return false;
}
if (a.getAbsolutePath().equals(b.getAbsolutePath())) {
return true;
}
if (!a.exists() || !b.exists()) {
return false;
}
if (a.length() != b.length()) {
return false;
}
boolean eq = true;
FileChannel channelA;
FileChannel channelB;
try {
channelA = new RandomAccessFile(a, "r").getChannel();
channelB = new RandomAccessFile(b, "r").getChannel();
long channelsSize = channelA.size();
ByteBuffer buff1 = channelA.map(FileChannel.MapMode.READ_ONLY, 0, channelsSize);
ByteBuffer buff2 = channelB.map(FileChannel.MapMode.READ_ONLY, 0, channelsSize);
for (int i = 0; i < channelsSize; i++) {
if (buff1.get(i) != buff2.get(i)) {
eq = false;
break;
}
}
} catch (FileNotFoundException ex) {
Logger.getLogger(HotUtils.class.getName()).log(Level.SEVERE, null, ex);
} catch (IOException ex) {
Logger.getLogger(HotUtils.class.getName()).log(Level.SEVERE, null, ex);
}
return eq;
}
package test;
import org.junit.jupiter.api.Test;
import java.io.IOException;
import java.nio.file.FileSystems;
import java.nio.file.Files;
import java.nio.file.Path;
import static org.junit.Assert.assertEquals;
public class CSVResultDIfference {
@Test
public void csvDifference() throws IOException {
Path file_F = FileSystems.getDefault().getPath("C:\\Projekts\\csvTestX", "yolo2.csv");
long size_F = Files.size(file_F);
Path file_I = FileSystems.getDefault().getPath("C:\\Projekts\\csvTestZ", "yolo2.csv");
long size_I = Files.size(file_I);
assertEquals(size_F, size_I);
}
}
it worked for me :)
-
Welcome to Stack Overflow! While this code snippet may solve the problem, it doesn't explain why or how it answers the question. Please include an explanation for your code, as that really helps to improve the quality of your post. Remember that you are answering the question for readers in the future, and those people might not know the reasons for your code suggestion. Aug 8, 2019 at 11:46
-
2