LPI Linux Certification/LPIC1 Exam 101/Linux Filesystems

Section Overview

 * 1.104.1 Create partitions and filesystems
 * 1.104.2 Maintain the integrity of filesystems
 * 1.104.3 Control mounting and unmounting filesystems
 * 1.104.4 Managing disk quota
 * 1.104.5 Use file permissions to control access to files
 * 1.104.6 Manage file ownership
 * 1.104.7 Create and change hard and symbolic links
 * 1.104.8 Find system files and place files in the correct location


 * Linux filesystems
 * Create partitions and filesystems
 * Maintain the integrity of filesystems
 * Control mounting and unmounting filesystems
 * Managing disk quota
 * Use file permissions to control access to files
 * Manage file ownership
 * Create and change hard and symbolic links

Overview
Description: ''Candidates should be able to configure disk partitions and then create filesystems on media such as hard disks. This objective includes using various mkfs commands to set up partitions to various filesystems, including ext2, ext3, reiserfs, vfat, and xfs.''

Key files terms and utilities include: fdisk mkfs

Partitions
Media can be divided into partitions. Partitions are usually created at installation time but can also be created with the fdisk program or other utilities. This will divide the media in partitions and different filesystems can be built and different operating systems can be installed.

IDE is recognized as follows:


 * Primary Master:
 * /dev/hda : Full disk
 * /dev/hda1: First partition
 * /dev/hda2: Second partition


 * Primary Slave: /dev/hdb
 * Secondary Master: /dev/hdc
 * Secondary Slave: /dev/hdd

SCSI is recognized as follows:


 * ID1:
 * /dev/sda: Full disk
 * /dev/sda1: First partition


 * ID2: /dev/sdb

USB and FireWire disks are recognized as SCSI disks.

Once it is partitioned it is possible to build a filesystem on every partition.

Filesystems
Filesystems exist to allow you to store, retrieve and manipulate data on a media. Filesystems maintain an internal data structure (meta-data) that keeps all you data organized and accessible. The structure of this meta-data gives the characteristic of the filesystem. A filesystem is accessed by a driver through the organized meta-data structure. When Linux boots it reads in /etc/fstab all the filesystems that needs to be mounted and checks if they are in an usable state.

When a power failure occurs Linux won't be able to unmout the filesystem properly and some data in the cache won't be synchronized on the media. The meta-data may be corrupted.

Once you reboot the system, it will detect this and do a fsck to the full meta-data structure for a consistency check. This can take a very long time. Few minutes to few hours in proportion of the media size. Journaling a filesystem is adding a new data structure called a journal. This journal is on-disk and before each modification to the meta-data is made by the driver it is first written into the journal. Before each meta-data modification the journal maintains a log of the next operation.

Now, when a power failure occurs there is only the need to check the journal. A journaling filesystem recovery is very fast. It just needs to go through the logs and fix the latest operation. Journaling filesystem recovery can take only few seconds.

On a cluster systems, journaling allows to quickly recover a shared partition of a node that went down.

Linux filesystems
...
 * ext2: Old very stable Linux filesystem. Efficient for files larger than ~2-3K.
 * ext3: Journaling extension for ext2. Possible to move a filesystem back and forth between ext2 and ext3.
 * Reiserfs: Journaling filesystem. 8-15 time faster than ext2 when manipulating small files.
 * XFS: A powerful journaling filesystem with possible quota and ACL
 * Msdos: MS-Windows FAT filesystem type. (mainly used for floppy)
 * Vfat: MS-Windows FAT filesystem type. (mainly used for large hdd partitions)
 * NTFS (ReadOnly but loop files): MS-Windows journaling filesystem
 * SMBFS: A filesystem to mount windows or samba sharing from Linux
 * NFS: Network File System

Filesystem tree
A Linux file system has one top directory called root (/) where all sub directories of the entire system is stored. The sub directories can be another partition, a remote directory, or a remote partition accessible through the network with NFS protocol.

Create filesystems
To create a file system on a partition, use mkfs.

mkfs [options] -t [fstype] device [blocksize]

Common options:
 * -t: fstype: File system type.
 * -c : Check the device for bad blocks before building the filesystem.

The full partition will be erased and organized to the type of filesystem requested. There is no undo command. The fstype possible are: msdos, ext2, ext3, reiserfs,minix,xfs

The blocksize allows to customize the block size for your filesystem.

Examples: mkfs -t msdos /dev/fd0 mkfs -t reiserfs /dev/hdd1 4096

Create extended filesystems
To create an extended (ext2, ext3) filesystem on an partition, use mke2fs.

mke2fs [options] device [blocksize]

Common options:
 * -b: Specify the block sizefstype: File system type.
 * -c : Check the device for bad blocks before building the filesystem.
 * -j: Create the filesystem with an ext3 journal.
 * -L: Set the volume label for the filesystem.

With mke2fs it is possible to store the super-block the journal information on another device. Examples:

mkefs -b 2048 /dev/fd0 -L floppy mkfs -V mke2fs 1.26 (3-Feb-2002) Using EXT2FS Library version 1.263

Monitoring disk usage
To print the disk usage, use du.

du [options] [files...]

Common options:
 * -a: All files not just directories
 * -b: Print size in bytes
 * -c: Total
 * -h: Human readable format. (1K, 20M,...)

Examples:

$ du -ch Documents 112k   Documents/Cours/LPI101 4.0k   Documents/Cours/LPI102 4.0k   Documents/Cours/LPI201 4.0k   Documents/Cours/LPI202 124k   total

du -sk ~ # Sums up your total disk usage in kilobytes

du -ak ~ | sort -n more # Display every file and its disk space in numerical order.

Filesystem disk space
A filesystem is composed of a meta-data structure plus a list of blocks. To print the filesystem disk space usage, use df.

df [options] [files...]

Common options:
 * -a: All included filesystems with 0 blocks.
 * -t: Limit listing to a filesystem type.
 * -h: Human readable format. (1K, 20M,...)

Examples:

$ df -t reiserfs -h F          1k-blocks      Used Available Use% Mounted on /dev/hda3             28771528   3121536  25649992  11% /

$ df -t ext2 -h Filesystem           Size  Used Avail Use% Mounted on /dev/hda1              15M  3.8M   10M  27% /boot

Exercises

 * Exercises results.

Overview
Description: ''Candidates should be able to verify the integrity of filesystems, monitor free space and inodes, and repair simple filesystem problems. This objective includes the commands required to maintain a standard filesystem, as well as the extra data associated with a journaling filesystem.''

Key files terms and utilities include: du df fsck e2fsck mke2fs debugfs dumpe2fs tune2fs

Checking filesystems
To check filesystems consistency, use fsck.

fsck [options] -t [fstype] device [fsck-options]

Common options:
 * -A: Go through the /etc/fstab file and try to check all file systems. Typically  used  at boot time from a script.
 * -t fslist: Specify the type of file system to be checked. With -A,  only  filesystems  that match fslist are checked
 * -C: Display completion/progression bar.

Common fsck-options:
 * -a: Automatically repair.
 * -r: Interactively repair.

Examples:

fsck -t msdos /dev/fd0 -a fsck -t reiserfs /dev/hda2 -r

Checking extended filesystems
To check extended filesystems consistency, use e2fsck.

e2fsck [options] device

Common options:
 * -b: Use an alternate super block filename.
 * -c: This option makes badblocks program to run and marks all the bad blocks.
 * -f: Force checking even if the filesystem seems clean.
 * -a or -p: Automatically repair.
 * -y: non-interactive mode

Examples:

e2fsck -ay /dev/fd0 e2fsck -f /dev/hda2

Debugging extended filesystems
The debugfs program is an interactive file system debugger. It can be used to examine and change the state of an ext2 file system.

debugfs device

Common commands:
 * Help: Give an help on all the command available.
 * Stats: Give a statistics of the media.
 * Stat file: Give a statistic of the file on the media.
 * logdump: Dump the journal's media into a file

Example: stat haut.gif Inode: 14  Type: regular    Mode:  0644   Flags: 0x0   Generation: 67558 User:    0   Group:     0   Size: 3786 File ACL: 0   Directory ACL: 0 Links: 1  Blockcount: 8 Fragment: Address: 0    Number: 0    Size: 0 ctime: 0x3ddf3840 -- Sat Nov 23 09:11:44 2002 atime: 0x3ddf3840 -- Sat Nov 23 09:11:44 2002 mtime: 0x3ddf3840 -- Sat Nov 23 09:11:44 2002 BLOCKS: (0-3):55-58 TOTAL: 4

Dumping extended filesystems info
To print the super block and blocks group information of an extended filesystem, use dumpe2fs.

dumpe2fs [options] device

Common options:
 * -b: print the bad blocks of the filesystem.
 * -h: Display only the superblock information.

Example: dumpe2fs -h /dev/fd0 dumpe2fs 1.26 (3-Feb-2002) Filesystem volume name:  floppy Last mounted on: Filesystem state:        clean Errors behavior:         Continue Filesystem OS type:      Linux Inode count:             184 Block count:             1440 Reserved block count:    72 Free blocks:             1258 Free inodes:             168 First block:             1 Block size:              1024 First inode:             11 Inode size:              128 ...

Tuning extended filesystems
To tune an extended filesystem, use tune2fs.

tune2fs [options] device

Common options:
 * -i#: Interval between filesystem checks [d|m|w].
 * -l: List the contents of the filesystem superblock.
 * -L: Set the  volume  label  of  the  filesystem.

Examples: tune2fs -L floppy /dev/fd0 tune2fs -l /dev/fd0 (Same output as dumpe2fs -h /dev/fd0)

tune2fs 1.26 (3-Feb-2002) Filesystem volume name:  floppy Block count:             1440 Reserved block count:    72 Free blocks:             1258 Free inodes:             168 First block:             1 Block size:              1024 First inode:             11 Inode size:              128 ...

Exercises

 * Exercises results

.
 * 1) Build an ext2 file system, with a block size of 2048 bytes, on a floppy.
 * 2) Change the label of the floppy to BACKUP.
 * 3) Try to add a journal on the floppy media.
 * 4) Use debugfs to validate your floppy file system information, and check when it was last accessed.
 * 5) Use watch to monitor the size when you copy a big file.
 * 6) Create a shell script to list all files on the floppy bigger than 100 Kb.
 * 7) Display file system usage for all MSDOS file systems.
 * 8) Which directory MUST exist in / to qualify this OS as Linux?
 * 9) What is the file system usage of /proc?

Attach and detach filesystems

 * Control mounting and unmounting filesystems

Overview
Description: ''Candidates should be able to configure the mounting of a filesystem. This objective includes the ability to manually mount and unmount filesystems, configure filesystem mounting on bootup, and configure user mountable removable filesystems such as tape drives, floppies, and CDs.''

Key files terms and utilities include: /etc/fstab mount umount

Attach a filesystem
The mount command serves to attach the file system found on some device to the big file tree.

mount [options] mount [options] [-t vfstype] [-o options] device dir

If the device or directory is listed in /etc/fstab you can use the following:

mount [options] [-o options [,...]] device | dir

Normally only root has the privilege to mount devices unless it is specified in the /etc/fstab file. Examples: mount
 * 1) Print all the mounted filesystems (/etc/mtab).

mount -a
 * 1) Mount devices or dirs listed in /etc/fstab.

mount -n -o ro /dev/hdc /mnt
 * 1) Mount /dev/hdc partition in read only mode without updating /etc/mtab.

mount /media/cdrom mount /dev/cdrom
 * 1) Allow a user to mount the CDROM if the following line is in /etc/fstab:
 * 2) /dev/cdrom /media/cdrom iso9660 ro,user,noauto,unhide

mount -o sync /dev/sdb1 /mnt/usb
 * 1) Sync in realtime

Detach a filesystem
To detach a filesystem from the file tree, use umount.

umount [options] umount [options] [-o options [,...]] device | dir

A busy filesystem cannot be unmounted.
 * Open files
 * Working directory of a process.

Examples: umount -a # Unmount devices or dirs listed in /etc/fstab. umount /mnt # Unmount the filesystem attached to /mnt. umount /media/cdrom # Allow a user to unmount the CDROM if the following line is in /etc/fstab: /dev/cdrom /media/cdrom  iso9660  ro,user,noauto,unhide

File system information
The file /etc/fstab contains all the file systems and related information that will be used when doing a mount -a. (Boot time)

The file /etc/mtab is maintained by the kernel and keeps track on what is or isn't mounted. The /etc/fstab format is:

/dev/hda3  /              reiserfs defaults            1 2 /dev/hda1  /boot          ext2     defaults            1 2 /dev/cdrom /media/cdrom   auto     ro,noauto,user,exec 0 0 usbdevfs   /proc/bus/usb  usbdevfs noauto              0 0 /dev/hda2  swap           swap     pri=42              0 0
 * 1) Device    Mount point    Fs type  Options             1 2

Common options:
 * ro:     read only
 * noauto: Don't mount automatically
 * exec:   Can execute binary on the filesystem
 * suid:   Allow to setuser bit
 * user:   Allow a user to mount/unmount it
 * unhide: hidden file visible
 * async:  All operations will be done asynchronously
 * default: rw, suid, dev, exec, auto, nouser, and async

Exercises

 * Exercises results


 * 1) Create a line in /etc/fstab that allows any user to access the floppy disk. Check that you can mount the floppy and can create a file with touch.
 * 2) Do the following manipulation:
 * 3) * Create an ext2 file system on the floppy.
 * 4) * Mount the floppy.
 * 5) *Copy all the files /etc/*.conf into the floppy.
 * 6) * Unmount it. What's happening?
 * 7) * Mount it back and check that all the files are there.
 * 8) * Issue the following command:
 * 9) * Tar cvf /dev/fd0 /etc/*.conf
 * 10) * Try to mount it back. What's happening?
 * 11) * Use tar to view the contents of the floppy..

Overview
Description: ''Candidates should be able to manage disk quotas for users. This objective includes setting up a disk quota for a filesystem, editing, checking, and generating user quota reports.''

Key files terms and utilities include: quota edquota repquota quotaon


 * Managing quotas for filesystems
 * Printing quota reports
 * Managing disk quota

Quotas
On a system, root can manage the usage of disk space per user and per filsystems. The two limits that can be setup are: The soft limit (soft =) specifies the maximum amount of disk usage a quota user is allowed to have. The hard limit (hard =) specifies the absolute limit on the disk usage a quota user can't go beyond it. There is also the possibility to setup a grace period that will enforce the soft limit only after an amount of time specified.

Setting up quotas for users
1) The keyword usrquota or/and grpquota must be added in file /etc/fstab for the partition interested.

/dev/fd0 /home/yann/mnt auto    rw,noauto,user,usrquota 0 0 /dev/hda5 /home    ext2    defaults,usrquota,grpquota 1 2

2) Add in each root filesystems the file user.quota or/and group.quota.

touch /mnt/aquota.user touch /home/aquota.user touch /home/aquota.group chmod 600 /mnt/aquota.user chmod 600 /home/aquota.user chmod 600 /home/aquota.group

Only root can do the quota administration and once the empty files have been created some disk quota can be set such as:
 * Soft limitation on number of files and inodes.
 * Hard limitation on number of files and inodes if the grace time is set.

Setting up quotas for users
3) Check the setting

quotacheck -v mnt quotacheck: Scanning /dev/fd0 [/home/yann/mnt] done quotacheck: Checked 6 directories and 1 files

4) Enable quota on the disk

quotaon -av /dev/fd0 [/home/yann/mnt]: user quotas turned on

5) Customize the disk quota limits:

$ edquota -u yann Disk quotas for user yann (uid 500): Filesystem   locks       soft       hard     inodes     soft     hard /dev/fd0      15          0          0          4        0        0

$ edquota -g yann

$ edquota -t Grace period before enforcing soft limits for users: Time units may be: days, hours, minutes, or seconds Filesystem            Block grace period     Inode grace period /dev/fd0                     7days                  7days

List quotas
To list quotas for a user or group, use quota.

quota [options] [user|group]

Common options:
 * -u:Default, print user quotas.
 * -g:Print group quotas for the group of which the user is a member.
 * -q: Print a more terse message, containing only information on filesystems where usage is over quota.

Example: quota -u yann

Display a quota report
To display a quota report, use repquota.

repquota [options] [user|group]

Common options:
 * -a: Report on all filesystems indicated in /etc/mtab to be read-write with quotas.
 * -g: Report for group.

Example:

$ repquota /dev/fd0 *** Report for user quotas on device /dev/fd0 Block grace time: 7days; Inode grace time: 7days Block limits                  File limits User           used    soft    hard  grace    used  soft  hard  grace -- root  --       8       0       0              2     0     0 yann  --      15       0       0              4     0     0

Exercises

 * Exercises results


 * 1) Setup a soft limitation for any users that have their home directory in /home to 500M.
 * 2) Change the grace time to 0.
 * 3) Log as the user and check if the limitation works..

Overview
Unix File Security File and Directory Permissions Default Permissions Changing File Permissions Changing File Owner and Group More Privileges Unix file security The biggest security problem may be you. You own any directory or files you created. You are responsible for accessibility of your files. You decide who can access which files and directories.

In your home directory you will be able to grant different levels of permission to yourself, users in your group and the all other users.

File and Directory Permissions
The permission of a file or of a directory can be viewed with ls -l.

File permissions
Examples of file permissions:

ls -l readme -rwxrw 1 toto users 14 Jul 5 10:00 readme

This means read,write, and execution permissions for user toto, read and write permissions for members of group users. No permissions for others. (0760)

ls -l /etc/hosts -rw-r--r-- 1 root root 14 Jul 5 10:00 /etc/hosts

This means read and write permissions for user root, read permissions for members of group root and all others. (0644)

Examples of directory permissions:

ls -ld /bin drwxr-xr-x 2 root root 4096 Jul 5 10:00 /bin

This means read,write, and execution permissions for user root, read and execution permissions for members of group root and others. (0755)

ls -l /home/toto drwxr-xr-x 10 toto users 4096 Jul 5 1:00 /home/toto

This means read, write, and execution permissions for user toto, read and execution permissions for members of group users and others. (0755)

Default permissions
The default permissions when creating a file are 0666 and when creating a directory are 0777. Most of the systems overwrite this at boot time with the program umask. Generally the mask value is 022. It means the write for group and other will be blocked. To check or change the mask value, do:

umask umask 066

Examples for file: default: rw- rw- rw- (0666) umask: 0 2 2 (0022) Block result: rw- r-- r-- (0644)

Examples for directory: default: rwx rwx rwx (0777) umask: 0 2 2 (0022) Block result: rwx r-x r-x (0755)

Changing file permissions
To change permissions on a file or directory, use chmod. To overwrite the existing permissions, do:

chmod 0755 /tmp #rwx for user, rx for group and others

To change add or cancel some permissions without overwriting all the existing permissions, do:

chmod u+w readme # Add write permission for user chmod +r readme # Add read permission for everybody chmod -r readme # Remove read permission for everybody chmod u+x,g=r readme # Add execution for user and set read for group chmod u=rwx,go=rx readme # Set read write and execution for user, read and execution for group and others

To change in recursive mode, use the -R option.

chmod -R +x /sbin/*

Exercises

 * Exercises results.

Overview
Description: ''Candidates should be able to control user and group ownership of files. This objective includes the ability to change the user and group owner of a file as well as the default group owner for new files.''

Key files terms and<tt> utilities</tt> include: <tt>chmod chown chgrp</tt>

Changing file owner and group
To change the owner of a file or directory, use chown.

chown yann mon_fichier.txt

To change the group of a file or directory, use chgrp.

chgrp dialout caller

The programs gpasswd and yast2 allow you to administrate groups.

gpasswd [-A user,...] [-M user,...] group
 * -A: Add users with group administrator privileges.
 * -M: Add members in group.

Group administrators can add or delete members of the group

gpasswd -d toto users gpasswd -a toto users

Group administrators can set or remove the password for the group.

gpasswd users gpasswd -r users

More privileges
It is possible to give more privileges to a user when it executes a particular script or program by setting the uid or gid bit of the file.

If the bit is set, the process will inherit the permissions of the owner of the file not the permissions of the user. To set the effective uid or gid, use chmod.

chmod 2640 [file] # (2) gid is inheritable for group. chmod 4640 [file] # (4) uid is inheritable for user.

Example of such program is /bin/passwd.

The sticky bit can also be set and can make the program text segment resident in RAM. chmod 1640 [file] (1) The file program stays in RAM.

Exercises

 * Exercises results

1) Write the command line by using letters with chmod to set the following permissions:

rwxrwxr-x : rwxr--r-- : r--r- : rwxr-xr-x : rwxr-xr-x : r-x--x--x : -w-rx : -xrwx :

2) Write the command line by using octal numbers with chmod to set the following permissions:

rwxrwxrwx : --x--x--x : r---w---x : -w--- : rw-r- : rwx--x--x :

3) With the following umask values what would be the files and directories creation permissions?

umask = 0027 File permissions: Directory permissions:

umask = 0011 File permissions: Directory permissions:

umask = 0541 File permissions: Directory permissions:

umask = 0777 File permissions: Directory permissions:

4) Create two user accounts

Logging in id: tst1, group users, with bash shell, home directory /home/tst1 Logging in id: tst2, group public, with bash shell, home directory /home/tst2 For the two accounts set a password.

Logging in as tst1 and copy /bin/ls into tst1 home directory as myls. Change the owner of myls to tst1 and the permissions to 0710. What does this permission value mean?

Logging in as tst2 and try to use /home/tst1/myls to list your current directory. Does it work ?

Create in /etc/group and /etc/gshadow a new group labo with tst1 and tst2. Change the owner group of myls to labo.

Try again from tst2 account to execute /home/tst1/myls to list your current directory. Does it work?.

Overview
Description: ''Candidates should be able to create and manage hard and symbolic links to a file. This objective includes the ability to create and identify links, copy files through links, and use linked files to support system administration tasks.''

Key files terms and<tt> utilities</tt> include: <tt>ln</tt>

Links
Use link when: You want to create a pathname to a file. Set a shorter or fixed pathname to a file.

To link one file to another, use ln:

ln [options] filename linkname ln [options] filename linkdirectory

Common options:
 * -f force: clobber existing link
 * -s symbolic link

The default links are hard links (ln without an option). On Windows they're called shortcuts. A hard link can only be created to an existing file on the same physical device, after creation no visible association can be displayed between a link name and a file name. A symbolic link can be created on a file that doesn’t exist yet, the association between the link name and the file name can be viewed with the ls command.

Linking to a file
The symbolic and hard link can be displayed with ls -l. Symbolic link are indicated with an arrow: link_name->real_filename.

$ ls -l /dev/midi lrwxrwxrwx  1   root   root        6    Jul 4 21:50   /dev/midi -> midi00

Hard links are indicated with the number of links counter (3-1=2 in this case).

$ ls -l readme -rwxrwxrwx  3   yann   users       677  Jul 4 21:50   readme

When removing a link name, use rm. Only the link will be removed not the linked file.

Exercises

 * Exercises results


 * 1) Create a directory etc and bin in your home directory.
 * 2) Copy all the files in recursive mode from /etc to your etc directory  and do the same for /bin to bin.
 * 3) In your local etc directory rename all files *.conf by *.conf.bak
 * 4) Create in your home directory a symbolic link call dir that points to your local bin/ls. Check if dir do execute ls.
 * 5) Remove the dir link. Is bin/ls still there?.

Exercices

 * Exercises results

Périphériques & Systèmes de fichiers Linux (LPIC101)