Linux Kernel Filesystem Development

This module contains introductory material. It starts by covering some basic information about filesystems in Linux. How many are there? How complex are they and how many LOCs code do they have?

There will be a short session on how to download the kernel source and browse through the filesystem code. This covers the system call and VFS layers as well as individual filesystems.

We’ll then cover who uses which filesystems and where filesystems reside in the kernel.

Finally we’ll highlight how to get started if you want to make a filesystem from scratch and describe some basic information about SPFS, the filesystem that we’re using for this course.

We’re going to dive in and get our hands dirty in this module by seeing how SPFS stores information on disk.

First though, we’ll introduce some of the features of SPFS and describe some of its limitations.

We’ll cover the major structures that SPFS stores on disk. Think superblock, inodes and directory entries.

Then we’ll go and explore these structures by running the SPFS mkfs command and use the SPFS fsdb command to explore how to access these structures.

We’ve played around with the “mkfs” and “fsdb” commands. The next step is to download the SPFS source code and look at what’s there. We will also walk through the source code for “mkfs” and “fsdb”.

We now know how to get the basic filesystem on disk using “mkfs” and we’ve covered how to analyze it using “fsdb”.

The next phase is to understand the order of events to follow in order to start building the filesystem. What do we do first? What do we do second and so on. As mentioned, it's a lot easier to read information off disk than is to write information.

Creating new files involves changes to disk-based structures. We’ll be discussing how to add additional files to the filesystem during “mkfs” time so can focus on operations that read structures off disk before the need to update and write them back.

We'll talk about different debugging options from simple use of “printk” calls, having a better solution that allows debugging at the source code level, and also what to do when the kernel panics, which it inevitably will.

Finally we’ll talk a little bit about error checking. SPFS is a teaching filesystem so it’s very sparse on error checking. This reduces the amount of code making it easier to read and understand.

Now it’s time to start having fun … we’ll start by covering what’s needed in the Linux VM that we’ll be playing with.

There aren’t many things to set up but we’ll need SSH access to the VM, the “make” command and of course, the compiler. We’ll need a disk device for testing so we’ll cover partitioning.

We need to get the source code onto the VM and of course, we need to compile it. And once we’ve done all of that, we can finally start playing with the filesystem.

It's now time to start digging into the source code. We’ll start with the module load and unload code, which is the most logical place to begin.

It's fairly straightforward and includes two functions provided by SPFS which are called during module load and module unload.

We need to introduce a structure provided by SPFS which is used during registration of the file system with the kernel. We'll also briefly cover initialization of the SPFS inode cache which is shared between all mounted SPFS systems.

We have a kernel module, it’s compiled, and we’re able to load and unload the module into and out of the kernel. The next step is to understand how to mount an SPFS filesystem.

We’ll first review how the kernel knows which filesystem to call when the filesystem type is passed as an option to the mount command using –t.

Next we’ll cover what steps the filesystem needs to go through to complete the mount and we’ll take a little digression to highlight the Linux directory cache, more commonly known as the dcache.

There will also be a couple of "gdb" sessions, to show the paths through the kernel into the filesystem when performing a mount operation.

This is quite a long module so you might want to watch it a couple of times before moving on to the next module.

Now that we can mount a filesystem, it's time to look at the process for accessing a file inside the filesystem, commonly known as pathname resolution.

This is actually one of the most complex parts of the kernel and several things need to be explained in order for this to make sense.

As a consequence, pathname resolution will be split across the next two modules and followed by module 10 which is a big "gdb" session summarizing how all the structures get linked together.

In this module, we'll talk more about the Linux dcache, spend  time looking at how inodes are read off disk and introduce the concepts behind pathname resolution.

In the last module, we started looking at pathname solution, and explored interaction between SPFS inodes, Linux inodes and the Linux dcache. We also revisited the SPFS disk layout to show how inodes are stored on disk.

Then we looked again at sp_read_inode() to explore in more detail how inodes and read from disk and information is stored in-core.

In this module, we will be looking at how SPFS finds files on disk, and how the SPFS “lookup” function interacts with a Linux dcache.