-2

I have read that Linux ext3 and ext4 can hold around 213 and 232 files in one directory. But storing 4 billion files in one directory doesn’t seem fair. In real life experience I have not seen such large directories. The only large directories I have seen are OS installions directories like MS windows which perform slower than other directories on CDs and antivirus enabled machines.

I have developed a website that could receive files from users. I am storing files in a folder by renaming files by ID say 1.dat, 2.dat, 3.dat and so on, and a reference in database for each file as “id”, “owner”, “realname”, and “folder”.

1. I want to ask Is it good practice to have all files in one folder on a server?

2. If not, how many files I should I start grouping the files in folder, let’s say at 100,000 files in folder 1, and next 100,000 in folder 2? What number I should I group and switch the folder to another? What is the best practice for the performance?

3
  • There should be no problem in grouping the files in one folder..
    – brunsgaard
    Oct 8, 2014 at 18:53
  • And you've just taught me that you can use <sup> in a question! I suppose that was obvious but it hadn't occurred to me. Oct 8, 2014 at 19:05
  • A lot will depend on your specific filesystem and performance envelope requirements. As written, this question is too broad. Oct 8, 2014 at 21:27

1 Answer 1

1

This isn't a problem at all. Store them all in one directory.

The time when it's a bad idea is if it's a directory you're going to be managing by hand. Then, very large numbers start to cause you a headache. It would be like having one massive drawer in your filing cabinet rather than lots of separate ones.

But if you've made sure that the files all have unique names, and if you have a programmatic method of managing everything, then having them all in one directory is a good plan. In fact, it's easier, because if you split them up then you need to store something to help you find the directory containing the file you want.

Not the answer you're looking for? Browse other questions tagged or ask your own question.