4

Java 7 defines this option, yet I fail to understand its usefulness.Consider this simple program, run on a recent enough Linux machine, with a Java 6 JVM:

public static void main(final String... args)
    throws IOException
{
    final long offset = 1L << 31;
    final RandomAccessFile f = new RandomAccessFile("/tmp/foo", "rw");
    f.seek(offset);
    f.writeInt(2);
    f.close();
}

When I query the file "shell wise", I get, as expected:

$ cd /tmp
$ stat --format %s foo
2147483652
$ du --block-size=1 foo
4096    foo

That is, the inode truthfully declares that the file has a size close to 2 GB, but its disk usage is in fact a single block since the underlying fs has a 4k block size. Good.

But I didn't need Java 7's StandardOpenOption.SPARSE for that. In fact, if I run this exact same code with a Java 7 JVM, the results do not vary.

Now, on to some Java 7-only code:

public static void main(final String... args)
    throws IOException
{
    final ByteBuffer buf = ByteBuffer.allocate(4).putInt(2);
    buf.rewind();

    final OpenOption[] options = {
        StandardOpenOption.WRITE,
        StandardOpenOption.CREATE_NEW
    };
    final Path path = Paths.get("/tmp/foo");
    Files.deleteIfExists(path);

    try (
        final SeekableByteChannel channel
            = Files.newByteChannel(path, options);
    ) {
        channel.position(1L << 31);
        channel.write(buf);
    }
}

This also creates a sparse file, and I did not have to specify StandardOpenOption.SPARSE at all.

So, what is it used for? Is there any OS/filesystem combination where this option actually influences the behaviour?

1
  • since you pointed out that you are working on a linux. so what file system are you using?ext4 or ext3? Because I want to know since create a file with java on linux will be sparse by default, but how can I create the same file with non-sparse in java?
    – Ryan Zhu
    Commented Nov 13, 2013 at 5:11

2 Answers 2

7
+200

Oracle's notes in the I/O tutorial list NTFS as one filesystem in which the option matters. Microsoft's docs on sparse file support in NTFS says that sparse files must be explicitly marked as sparse, and it lists actions specific to sparse files (zeroing out regions, searching for ranges with non-zero data, etc).

I don't have a Windows box handy on which to try this out, but seeing as the tutorials specifically call out NTFS, that might be a place to focus the search.

1
  • 1
    It's like said, the option makes a difference on NTFS: Without it one has lazy allocation, but writing at the end of a large file physically allocates 0-bytes for the whole file. With that option it doesn't, e.g. a 4 GiB file with 1 byte written at the end consumes only ~64 KiB of data on the disk. github.com/frohoff/jdk8u-jdk/blob/master/src/windows/classes/… Commented Dec 21, 2018 at 17:38
1

When you use a RandomAccessFile like this on Linux it is Sparse by default on ext4, however if you try ext3 or Windows you may find it is not sparse. i.e. the behaviour of the plain RandomAccessFile constructor is OS dependent.

5
  • Uh, ext3? No sparse support? ;) Sorry but no, it has supported sparse since its inception, and so has ext2 for that matter (open(2) and seek(2) haven't changed). What is more, my fs is btrfs. As to Windows, well, are there options when you open/create a file to create it sparse or not?
    – fge
    Commented Jul 13, 2013 at 21:35
  • @fge If it didn't support sparse files, the option would be pointless, the option is to switch the default. Commented Jul 13, 2013 at 21:38
  • Wait wait wait. Ext2+ have had support for sparse files since the beginning. You don't need to open a file with this option in order for the created file to be sparse (and as I said, open(2) has no such option -- neither does fopen(2) for that matter). My question really is, when does this option make sense? In what context? As far as I know, it has no effect at all on any well behaved Unix system
    – fge
    Commented Jul 13, 2013 at 21:40
  • OK, let me rephrase: since POSIX has existed, neither open(2) nor fopen(2) have had a flag to open a file with an option saying, "this will be a sparse file". And each and every Unix filesystem having seen the day in the last 15+ years (ext*, btrfs, ufs, you name it) supports sparse files from the get go. Hence my question: why does Java have that flag?
    – fge
    Commented Jul 13, 2013 at 21:47
  • Some options in Java are added for "forward compatibility" for lack of a better word. It is possible it is one of these, options which end up doing nothing because they never did anything and then this get burned in for backward compatibility reasons. The only way to know for sure is to trace through the code, a simpler option would be strace and see if the system calls are any different on LUNIX at least. Commented Jul 14, 2013 at 19:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.