Which file system is best

Whether you are using Linux on the desktop or as a server, it's still good that you're using Linux. Linux related questions go here.

Moderator: General Moderators

Post Reply
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

Which file system is best

Post by alex.barylski »

Here are my requirements:

1) Fastest in terms of read/write
2) Store the largest number of files in single directory (100+ million)

Error checking, rollbacks, etc are not nessecary as the data is backed up daily to another external drive with all those bells and whistles. This is purely required for performance and ease of use (hence the single directory requirements).

I recall reading an article once that said Linux cops out at about 10000 files per folder. This kind sucks, is there a work around short of re-writing the FS driver?

Cheers :)
User avatar
Kieran Huggins
DevNet Master
Posts: 3635
Joined: Wed Dec 06, 2006 4:14 pm
Location: Toronto, Canada
Contact:

Post by Kieran Huggins »

afaik there aren't any filesystems without a soft folder limit

I would use multiple folders: i.e. "fahadjfhkahfa" would be stored at "/f/a/h/a/fahadjfhkahfa"
User avatar
volka
DevNet Evangelist
Posts: 8391
Joined: Tue May 07, 2002 9:48 am
Location: Berlin, ger

Post by volka »

Hockey wrote:I recall reading an article once that said Linux cops out at about 10000 files per folder.
Check the date of that article. Once upon a time there was a 2/4 color limit and computers were called c64 ;)


Hockey wrote:files in single directory (100+ million)
8O
http://namesys.com/faq.html#reiserfsspecs wrote:max number files a dir can have

3.5
518701895,
but in practice this value is limited by hash function. r5 hash allows about 1 200 000 file names without collisions

3.6
232 - 4 => 4 Gi - 4
but in practice this value is limited by hash function. r5 hash allows about 1 200 000 file names without collisions
http://www-128.ibm.com/developerworks/library/l-jfs.html wrote:JFS dynamically allocates space for disk inodes as required, freeing the space when it is no longer required. This support avoids the traditional approach of reserving a fixed amount of space for disk inodes at the file system creation time, thus eliminating the need for users to estimate the maximum number of files and directories that a file system will contain.
But still
Hockey wrote:files in single directory (100+ million)
8O smells like bad design.
User avatar
Jenk
DevNet Master
Posts: 3587
Joined: Mon Sep 19, 2005 6:24 am
Location: London

Post by Jenk »

User avatar
jayshields
DevNet Resident
Posts: 1912
Joined: Mon Aug 22, 2005 12:11 pm
Location: Leeds/Manchester, England

Post by jayshields »

Suprised no one's mentioned Sun's ZFS! http://www.sun.com/2004-0914/feature/

Is XFS a modified ZFS?
User avatar
volka
DevNet Evangelist
Posts: 8391
Joined: Tue May 07, 2002 9:48 am
Location: Berlin, ger

Post by volka »

jayshields wrote:Is XFS a modified ZFS?
not really ;)
http://en.wikipedia.org/wiki/Xfs wrote:XFS is the oldest journaling file system available for UNIX systems, and has a mature, stable and well-debugged codebase.
[...]
Introduced 1994 (IRIX v5.3)
http://en.wikipedia.org/wiki/Zfs wrote:Introduced November 2005 (OpenSolaris)
Post Reply