New to Linux and the open source world? We have compiled a huge list of resources to help you go through Linux and its distributions. Visit the full Linux guide page right now.
What are Linux Filesystems?
Linux filesystems are the responsible pieces of software to handle the manipulation and storage process for data stored in your hard drive, without using a filesystem, you would end up with a big one block of data that you can’t manipulate or modify, which is why filesystems came to solve this problem.
What filesystems do is that they separate the data into different partitions, files, folders and different other forms of divided data, it gives each one of them a special name and a metadata that identifies that chunk of data, by doing so, the data will be easily manipulated and categorized into whatever the user needs.
Filesystems control how your data are being stored, when and where to be stored and what extra features do they provide for you (like snapshots, encryption, live resize.. etc). There are many filesystems out there for different operating systems like EXT, Btrfs, NTFS, ZFS.. And each one of them has its own way to work, features, targeted platforms and performance.
Therefore, we’ll be able to browse, create, edit, delete and do whatever we need with some specific chunk of data, not the whole hard disk drive, which is very essential in today’s technologies.
The hierarchy of any root Linux filesystem looks similar to the following diagram:
Extended Filesystem (EXT)
The default and original filesystem for Linux that was developed in 1992 to be an alternative to Unix Filesystem (UFS), today it’s almost the main filesystem that most users choose when installing their Linux distributions, it has been there almost since the release of Linux itself.
Ext was released over many versions, first ext was released in 1992, ext2 was introduced in 1993, ext3 came to the hall in 2001 and finally ext4 (the latest version of Ext) was released in 2008. Each one of those versions came with its own special features, better I/O manipulation and hundreds of bug fixes.
Currently, Ext filesystem offers many features like:
- Holding large files, Ext can create files up to 16 terabytes in size and volumes up to 1 Exabytes, which is considered huge even in terms of big data, and far way more than enough for normal users.
- Journaling filesystem, Ext creates a special file called “journal” that monitors that data consistency, in case of crash or improper shutdowns, the file system doesn’t need to preform a hard drive check (Like in old Windows 98 if you remember), more ever, data can be restored to most recent version available.
- Can create files with names up to 255 bytes.
- Multi-Block allocation, which simply allows writing data to multiple blocks at the same time instead of waiting to fill each block by its own, which gives better performance and speed while writing the data.
- Pre-Allocation, Ext4 – thanks to the fallocate() function in the Linux kernel – Ext can create a persistent space for a file before its creation, holding that space until it’s used by that file, this is good for databases and multimedia files.
- No sub-directory limit, you can create unlimited sub-directories on the file system (This was introduced in Ext4).
- Backward compatibility, Ext4 is compatible with Ex3 and Ext2 filesystems, you can mount those filesystems easily to your current filesystem without a haste.
- Timpstamps for critical real time applications, Ext4 supports timestamps in nanoseconds which allows critical applications (such as Medicine or Economy applications) to get the time in nanoseconds.
However Ext4 is still considered part of an old technology; according to Theodore Ts’o who’s one of the main developers for Ext4, Btrfs would be a better approach to choose after its development completes. It’s good for a lot of purposes, but not the newest or best in the industry yet. Unfortunately, no new Ext5 filesystem is about to come anytime soon.
One of the newest filesystems in the family, it was released in 2007 by Oracle in order to be used with Big data and for scale-ability purposes, in 2014, the filesystem disk format was marked as stable and today, it’s available to choose in almost all Linux distributions during the installation process.
The main code for Btrfs is still under development, many bugs are fixed, many possible failure scenarios are being solved, new features are also added at every new release, which is why developers always recommend using the latest Linux kernel if you are going to use Btrfs filesystem with your hard drives.
There’s a long list of Btrfs’s features, like:
- Subvolumes: in Btrfs, a subvolume is like a namespace (a category) that includes other subvolumes or files, it can be mounted or unmounted on need, there’s always a main subvolume (usually for /) that is loaded on boot. Subvolumes are treated as filesystems theirselves.
- Copy-on-write filesystem: when a file is modified, the original data is never changed, instead, a new “link” or inode is created for that change, both the original file + new data share the same storage block and they are not duplicated, what happens is that Btrfs saves each change in a place for its own closely to the original file, instead of just overwriting it.
- Snapshots: One of the most important features in Btrfs is snapshots; if you enabled this feature, you can go back in time for any state of your Linux distribution, or get any older version of a file you want, I remember getting back to a fresh install on openSUSE 42.1 from the GRUB boot loader menu (using snapper), Btrfs does this by its copy-on-write feature, the snapshot and the current file share the same data block and the filesystem saves the changes that occurs on the files only, it doesn’t create “backup” files, which decreases the size of snapshots, it saves the metadata of each change and the state of the files before any change, making it possible to revert it later.
- Quota groups: You can specify the maximum size you want the snapshot or the subvolume to use, this is good if you want to save some space by limiting how much users can use of it.
- Ext Conversion: Using the btrfs-convert tool, you can convert any Ext 2/3/4 filesystem into a Btrfs filesystem, usually without any data loss, but Ext 3/4 conversion to a Btrfs is not tested yet and not recommended.
- Live Rescale: You can change the size of a mounted Btrfs filesystem without the need to unmount it, unlike the situation with Ext partitions, however; you can’t rescale the root filesystem, you have to unmount it first.
- Large files manipulation: You can create files with up to 16 Exabyte of size.
- Compression: You can use both the Zlib and LZO algorithms to compress the Btrfs filesystem you want. You can also choose to have compressed files on the same filesystem using different algorithms along with uncompressed files.
Btrfs is one of the rising filesystems in the *NIX family, it’s very fast comparing to Ext I/O and offers a lot of features like snapshots and sub-volumes which make your life easier; however, it’s still under development and not yet 100% ready for all the possible scenarios. Read more about it in the Btrfs wiki.
Another filesystem that was initially developed in 1993, XFS is a 64-bit journaling filesystem that focuses on I/O operations, it’s very fast in writing and reading data, which is why it’s recommended to use it by default for your home partition (if you separate it from the root partition), it was introduced in the Linux kernel in 2001. It works with FreeBSD system as well.
XFS has a special design that gives it a great performance in doing parallel I/O processes, it uses allocation groups (special divisions for disk sectors, like sub-volumes in Btrfs) which allows it to create huge number of I/O threads and do operations on large files and folders in the same time, it also supports metadata journaling to avoid inconsistency of data.
Among those who use it is NASA; They are using XFS on two +300TB hard drives in their archival servers because of its high I/O performance. XFS doesn’t support snapshots, it supports live resizing (but not shrinking), it also provides a defragmentation utility called xfs_fsr that can be use to do live defragmentation on the filesystem.
ReiserFS is a journaled filesystem that was firstly released in 2001, it was the first journaled filesystem actually to be supported on Linux, the benchmarks have shown that ReiserFS is better than Ext 3/4 in serving file smaller than 4K in size; this makes it very good for caching services and other small-files-manipulating software.
However; it has some bugs with mail servers and other problems, and its latest version; Reiser4 is not included in the Linux kernel, you would need to use a patched kernel or apply it yourself.
Another copy-on-write filesystem, OpenZFS is actually a fork of ZFS; a filesystem that was originally developed by Sun for Solaris operating system, later after Sun was sold to Oracle, this last one didn’t show any intentions to continue the development on ZFS specially that they had Btrfs, therefore ZFS was marked as abandoned.
Later on, the community and the older developers behind ZFS came together and forked the filesystem to OpenZFS, it’s now sponsored by many companies, one of them is Intel, there will be a developer summit for OpenZFS developers in September.
Our final filesystem in this list is JFS, it’s a 64-bit journaling filesystem that was developed by IBM in 1999 and marked stable for Linux in 2001, it’s good as a general filesystem and works well with the limited resources computers, for example database servers that require high-CPU usage, JFS has proven that it’s not using very much CPU comparing to other filesystems when put in this field, which can mark it as a lightweight filesystem.
JFS supports journal, compression, allocation groups, dynamic inode allocation (leaving a space for disk inodes when needed) and also offers superblock; where the information of the super filesystem is stored (like the size, data blocks, block size.. etc). JFS is not available to choose by default in most Linux distributions because it’s support package jfsutils is not installed.
There are many filesystems out there, and an enormous number of them work well on Linux, you can choose whatever filesystem that fits your needs, and usually as an end user, it won’t be that very much difference, unless you would like to have some extra features like snapshots or fast I/O operations, you may consider trying those filesystems yourself to find the best one for you.
If you would like also, you can make a benchmark between them using GNOME Disk utility, it’s a part of the GNOME environment itself and comes under the name “Disks”, it offers a benchmark option: