See this page as a slide show
The Filesystem
Original slides from Dr. James Walden at Northern Kentucky University.
Topics
- Overview
- Pathnames
- Mounting
- Structure
- Organization
- File types
- Kernel Data Structures
Overview
- Basic purpose of the Linux filesystem:
To represent and organize the system’s storage resources, i.e.,
directories and files.
- Because the filesystem was well organized and easy to access,
system programmers uses it to store other resources:
- Processes
- Devices
- Mount Points
- Kernel Data
- Communication Channels
Filesystem Components
- Namespace:
A way of naming things and organizing them in a hierarchy
- Application Programmer Interface:
A set of system calls for navigating and manipulating objects
- Security model:
A scheme for protecting, hiding, and sharing things
- Implementation:
Software that ties the logical model to the actual hardware
Filesystem Types
- ext2, ext3, ext4: Linux
- FAT16, FAT32, NTFS: Windows
- HFS, HFS+: Apple
- XFS: Silicon Graphics
- NFSv2, NFSv3, NFSv4: Sun (distributed)
- ISO 9660, UDF: Optical Discs
https://en.wikipedia.org/wiki/Comparison_of_file_systems
Hierarchy
Files are located by traversing a directory tree:
- Root directory:
Represented by single forward slash (
/)
- Home directory:
Represented by the tilde symbol (
~)
- Not really a filesystem thing—a shell thing.
- Working directory:
Processes have a current working directory (
.)
- Parent directory:
All directories except root have parent (
..)
- Absolute path:
Pathname all the way from root (
/usr/local/bin)
- Relative path:
Pathname from current location (
../local/bin)
Linux filesystem is a single unified hierarchy, unlike Windows.
Filenames
- File, filename, path, pathname.
- Files are represented by strings.
- Pathname includes directory structure.
- File and directory names use the same namespace.
- Each component can be 255 characters.
- The entire path can be 4095 characters.
- Linux
find command provides recursive listing.
Characters in Filenames
- Spaces and other special characters can be used.
- Anything but slash or null is ok.
- If you can sneak your bizarre character past the shell.
- Periods are followed by file extension, periods aren’t special.
- Except that
ls, by default, ignores files starting with one.
Really, anything?
% touch Jack Applin
% date >J*
% ls -l
total 4
-rw------- 1 neutron neutron 0 Jul 3 17:24 Applin
-rw------- 1 neutron neutron 29 Jul 3 17:24 Jack
% date >don't
> What is going on?
> I can't keep this up forever.
date: extra operand ‘this’
Try 'date --help' for more information.
% touch '~`!#$^&*( )[]\{}|;'\'':",.<>?½→⻥☂'
% ls -lhog *:*
-rw------- 1 0 Jul 3 17:28 ~`!#$^&*( )[]\{}|;':",.<>?½→⻥☂
%
Anything‽
% rm -z
rm: invalid option -- 'z'
Try 'rm --help' for more information.
% rm '-z'
rm: invalid option -- 'z'
Try 'rm --help' for more information.
% rm ./-z
rm: cannot remove ‘./-z’: No such file or directory
% rm -- -z
rm: cannot remove ‘-z’: No such file or directory
Mounted Filesystems
Global filesystem contains mounted filesystems:
- Textbook uses “file tree” for global filesystem
- Mounted filesystems are attached to file tree
- Mounted filesystems are usually disk partitions
- Also can be network file servers
- Each must obey the proper API
- Attached to the tree with the
mount command
Mounting
mount /dev/hd4 /MST3K
- Installs filesystem stored on disk under the path
/MST3K
- Files that were originally at
/MST3K no longer accessible!
- Filesystem navigation occurs seamlessly
/etc/fstab
- List of filesystems customarily mounted on system
- Filesystems can be checked using
fsck command
- Shows layout of system and allows
mount -a
- Show mounted filesystems and their types with
df -T
Unmounting
umount /MST3K
- Detaches a file system
- Cannot unmount a file system that is busy
- No open files
- No processes whose current working directory is in the file system
- No executable programs from the filesystem
umount -l /MST3K
- Lazy unmount, makes it unavailable
- Waits for none of the above to be true
- Then physically unmounts
Identify Processes
fuser -mv /MST3K
(Shows all open files, processes, and executables)
| Code | Meaning |
|---|
| f | Process has a file open for reading |
| F | Process has a file open for writing |
| c | Process has a current working directory |
| e | Process is currently executing a file |
| r | Process has set a root directory (chroot) |
| m | Process has mapped a file or a shared library |
Filesystem Structure
- Blocks that stores metadata with structural info
- Superblock: filesystem type, size, status
- Blocks that store user data contained in files (really, in inodes)
- Objects in filesystem represented by inodes
- inode has file type, permissions, owner, group,
file size, number of links, access control,
file creation, access, modification times
- Directory is a list of [filename,inode number] pairs
Examples
$ ls -l /etc/passwd
-rw-r--r--. 1 root root 2733 May 17 12:46 /etc/passwd
$ ls -i /etc/passwd
1468033 /etc/passwd
$ stat /etc/passwd
File: `/etc/passwd’
Size: 2733 Blocks: 8 IO Block: 4096 regular file
Device: 802h/2050d Inode: 1468033 Links: 1
Access: (0644/-rw-r--r--) Uid: (0/ root) Gid: (0/ root)
Access: 2012-08-29 18:50:01.213638229 -0600
Modify: 2012-05-17 12:46:25.000000000 -0600
Change: 2012-05-18 18:41:06.000000000 -0600
inode
An inode contains, at least:
- file type (regular file, symlink, directory, character, block, …)
- file size in bytes
- UID / GID
- protection bits (e.g., rwxr-x---)
- ctime: inode last changed (changes to any field, incl. times)
- mtime: modification time
- atime: access time
- link count (reference count)
- pointers to the disk blocks
that store the file's contents
inode block pointers
A classic inode has thirteen pointers:
- ten direct blocks
- one indirect block
- one double-indirect block
- one triple-indirect block (not shown)
Some filesystems optimize the storage of tiny files by storing the data
itself in the inode.
Filesystem Organization
- Organization has always been haphazard
- Several naming schemes used, some incompatible
- That’s what you get when things evolve.
Standard directories
| Directory | Meaning | Directory | Meaning |
|---|
/boot | Boot directory | /usr | Most standard programs |
/dev | Device files | /var | Spool directories |
/etc | Critical system files | /home | Mount point for users |
/sbin | System utilities | /lib | Libs and parts of the C compiler |
/bin | Important utilities | /media | Removable media |
/tmp | Temp files | /opt | Optional applications |
Standard directories
/usr/bin | Most commands and executables |
/usr/include | Header files |
/usr/lib | Libraries, support files for standard programs |
/usr/local | Local software |
/usr/local/bin | Local executables |
/usr/local/… | Other local (etc, lib, sbin, src) |
Standard directories
/usr/man | Man pages |
/usr/sbin | Less essential sysadmin commands |
/usr/share | Common to multiple systems |
/usr/share/man | Shared man pages |
/usr/src | Source code for nonlocal packages |
Standard directories
/var/adm | Logs, system setup records |
/var/log | System log files |
/var/spool | Spooling directories (mail, printers) |
/var/tmp | More temp space (preserved between boots) |
File types
$ ls -ld
crw------- 1 root root 5, 1 Aug 24 15:24 /dev/console
brw-rw---- 1 root disk 8, 0 Aug 24 15:23 /dev/sda
srw-rw-rw- 1 root root 0 Aug 24 15:23 /dev/log
File type encoding
| File type | Symbol | Created by | Removed by |
|---|
| Regular file | - | cp, mv, vi | rm |
| Directory | d | mkdir, cp -r | rmdir, rm -r |
| Character device file | c | mknod | rm |
| Block device file | b | mknod | rm |
| Local domain socket | s | socket(2) | rm |
| Named pipe | p | mknod | rm |
| Symbolic link | l | ln -s | rm |
Regular file
- File of bytes
- Linux imposes no structure
- Sequential and random access allowed
Directories
- Contains named references to other files
. Refers to current directory
.. Refers to parent directory
- File’s name stored within the parent’s directory
- Not in file itself.
- More than one directory can refer to a file (links)
ln creates hard links, removed with rm
- Indistinguishable from first entry. They’re all just links.
Character and Block Device files
- Device files allow programs to communicate with
hardware and peripherals
- Modules that know how to talk to the devices are
linked into the kernel when built
- Device file not the same as a device driver
- File is just a rendezvous point
- Device files characterized by major, minor numbers
- Major number tells kernel which driver
- Minor number tells driver which device
Some Device Files
| Name | Description |
|---|
/dev/sda2 | disk |
/dev/ttyS0 | RS-232 serial line |
/dev/pts/1 | Pseudo-tty (pty) |
/dev/tty | An alias for your terminal |
/dev/null | Read fails, write succeeds |
/dev/full | Read succeeds, write fails |
/dev/zero | Read yields zeroes |
/dev/random | Read yields random data |
Local domain sockets
- Sockets are communication mechanisms
- Allow communication between processes
- Or more importantly, sockets control network communication
- Local domain sockets
- Only accessible from the local host
- Filesystem object rather than a network port
- Visible to other processes
- Accessible only by connected processes
- Examples include X, NFS
Named Pipes
Communication mechanisms
- Between processes running on the same host
- Pipes are also known as FIFOs.
- Similar to pipes on the command line, but the data is persistent.
Symbolic links
- Also commonly referred to as a “soft link” or “symlink”.
- A special kind of file that contains another name.
- For example, if
/alpha/beta were a symlink to gamma/delta,
then it’s the same as /alpha/gamma/delta
- However, if
/alpha/beta were a symlink to /epsilon/iota,
then it’s the same as /epsilon/iota
- Deleting the target can leave a dangling link.
- Permissions of the symlink just don’t matter.
- Where is the target string stored?
Hard Links
- A hard link is just another directory entry that refers
to the same inode.
- An inode has a reference count.
- When the reference count goes to zero, the inode (and associated data)
is deallocated.
- Do you now see why you need directory write permission,
and not file write permission, to delete a file?
Hard/Soft Link Differences
- Soft links can cross mount points, hard links cannot.
- Hard link with
ln, soft link with ln -s
- Soft links work on directories, hard links do not.
- Soft links require additional filesystem reads.
Kernel Data Structures
- Filesystem acts as an interface to kernel data
- Separate filesystem mounted as
/proc
- Can be used to obtain info about system
% grep MHz /proc/cpuinfo
cpu MHz : 1199.000
cpu MHz : 1199.000
cpu MHz : 1199.000
cpu MHz : 1199.000
cpu MHz : 1199.000
cpu MHz : 1199.000
cpu MHz : 1199.000
cpu MHz : 1199.000
Modifying kernel parameters
#! /bin/bash
# Set display to half-bright:
for max in /sys/class/*/*/max_brightness
do
echo $(($(<$max)/2)) >${max%/*}/brightness
done
