Timeline for Quickly create a large file on a Linux system
Current License: CC BY-SA 4.0
23 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 7 at 6:01 | history | edited | rogerdpack | CC BY-SA 4.0 |
clarify
|
| Jun 1, 2022 at 14:00 | comment | added | Alexander Stohr | zfs and fallocate do cope with each other. other limitations mentioned by others should have been resolved as well. rationale: a state of 2013 is no longer valid in 2022. (just in case, please re-evaluate for your specific environment.) github.com/openzfs/zfs/pull/10408 | |
| Sep 30, 2020 at 15:37 | comment | added | Guilherme |
@lxgr This happens because you're not writing directly to the laptop's hard drive, you're filling up your operating system's cache before hitting the disk. Try writing more data than you have RAM, also, try using status=progress to check how your speed varies as you run out of cache and oflag=direct or oflag=nocache
|
|
| Feb 17, 2019 at 4:52 | history | edited | rogerdpack | CC BY-SA 4.0 |
apparently it's POSIX now...
|
| Apr 18, 2018 at 17:17 | comment | added | AKV |
noticed that fallocate doesn't work with exFAT as well.
|
|
| Feb 21, 2018 at 9:40 | comment | added | ronnix |
looks like support for fallocate was added to tmpfs in Linux kernel 3.5: kernelnewbies.org/Linux_3.5
|
|
| Jun 16, 2016 at 1:00 | comment | added | Jim |
Doesn't work on memfs either. # fallocate -l 1G testfile.dat fallocate: testfile.dat: fallocate failed: Operation not supported However, dd worked like a champ (and very fast): # time dd if=/dev/zero of=filename.dat bs=1G count=1 1+0 records in 1+0 records out 1073741824 bytes (1.1 GB) copied, 0.701103 s, 1.5 GB/s real 0m0.704s user 0m0.000s sys 0m0.701s
|
|
| Jan 28, 2016 at 11:46 | comment | added | CMCDragonkai | Doesn't work on FAT32 either, but dd does using /dev/zero. | |
| Sep 19, 2014 at 10:25 | comment | added | nucc1 | It happens fast on filesystems that support sparse-files, since you're reading from /dev/zero and writing to a filesystem that supports sparse files. | |
| Sep 2, 2014 at 13:33 | comment | added | Franta |
In Debian GNU/Linux fallocate is part of the util-linux package. This tool was written by Karel Zak from RedHat and source code can be found here: kernel.org/pub/linux/utils/util-linux
|
|
| Jul 16, 2014 at 7:15 | comment | added | jQueen | May I get more information on fallocate? I'm unable to call it in adb shell in android. how to check support for commands like falloctae/truncate/mkfile? does it require any other shell like bash? | |
| Jul 7, 2014 at 13:43 | comment | added | Eddie | fallocate is not supported by ext3 either. bugzilla.redhat.com/show_bug.cgi?id=563492 | |
| Jun 18, 2014 at 21:37 | comment | added | froggythefrog | ^^ Note: The above, of course, will not work if your file system is a 32-bit file system or has file limitations smaller than the file you're trying to create on it. | |
| Jun 18, 2014 at 21:22 | comment | added | froggythefrog | For those having a problem creating a large file on your filesystem of choice (because fallocate is not allowing it for some reason), you can create a relatively large file (say 2GB) with fallocate, use mv to move the file over to your other filesystem. Then you can use cat to put together the size of file you need. For example, if you have a 2GB file named "a" and you wanted to create a 10 GB file called "b", you can execute "cat a a a a a > b". That will take a bit, but you will have a large file where you want it when fallocate won't create it directly at the location for you. | |
| Feb 1, 2014 at 18:30 | comment | added | Orlo | very fast. recommended. | |
| Nov 19, 2013 at 1:51 | comment | added | Joe |
This (fallocate) will also not work on a Linux ZFS filesystem - github.com/zfsonlinux/zfs/issues/326
|
|
| S Oct 31, 2013 at 15:05 | history | suggested | Totor | CC BY-SA 3.0 |
beautifying
|
| Oct 31, 2013 at 14:59 | review | Suggested edits | |||
| S Oct 31, 2013 at 15:05 | |||||
| Jun 28, 2012 at 4:07 | comment | added | A B |
fallocate is exactly what I was looking for.
|
|
| Jan 3, 2012 at 14:18 | history | edited | Franta | CC BY-SA 3.0 |
edited body
|
| Jan 3, 2012 at 14:14 | comment | added | Franta |
I have just tried on 3.0.0-14 kernel dd if=/dev/zero of=zerofile bs=1G count=1 and it was very slow: 37,3 MB/s. Maybe it depends on filesystem…
|
|
| Dec 5, 2011 at 12:32 | comment | added | lxgr | Is it possible that dd is internally using that already? If I do 'dd if=/dev/zero of=zerofile bs=1G count=1' on a 3.0.0 kernel, the write finishes in 2 seconds, with a write data rate of over 500 megabytes per second. That's clearly impossible on a 2.5" laptop harddrive. | |
| Apr 16, 2011 at 18:28 | history | answered | Franta | CC BY-SA 3.0 |