1

File Management

Instructors: Fu-Chiung Cheng

(鄭福炯 )Associate Professor

Computer Science & EngineeringTatung Institute of Technology

email:cheng@cse.ttit.edu.twhttp://www.cse.ttit.edu/~fccheng

2

Outline

• Unix File System• Basic File Manipulation Utilities• Advanced File Manipulation Utilities

3

Unix File Systems

• A file system is a collection of files and directories on a disk or tape in standard UNIX file system format. • Low capacity disks (such as floppy disks) usually contain a single file system.• Large disks usually are divided into several regions (partitions), each containing a file system.• Each UNIX file system contains four major parts: A. boot block: loader programs for booting B. superblock: key information of the file system C. i-node table: i-nodes store information about files. D. data block: file storage

4

File System layout

Block 0: bootstrap

Block 1: superblock

Block 2

Block n

...

Block n+1

The last Block

...

Block 2 - n:i-nodes

Block n+1 - last:Files

5

Boot Block

• A boot block may contains several physical blocks.• Note that a physical block contains 512 bytes

(or 1K or 2KB)• A boot block contains a short loader program for

booting• It is blank on other file systems.

6

Super Block

• Superblock contains key information about a file system• Superblock information: A. Size of a file system and status:

label: name of this file systemsize: the number of logic blocksdate: the last modification date of super block.

B. information of i-nodesthe number of i-nodesthe number of free i-nodes

C. information of data block: free data blocks.• The information of a superblock is loaded into memory.

7

I-nodes

• i-node: index node (information node)• i-list: the list of i-nodes • i-number: the index of i-list.• The size of an i-node: 64 bytes. • i-node 0 is reserved.• i-node 1 is the root directory.• i-node structure: next page

8

I-node structuremode

owner

timestamp

Size

Block count

Direct blocks0-9

Double indirect

Triple indirect

Single indirect

Data block

Data block

Data block

Indirect block

...

Data block

Data block

Data block

...

Indirect blockIndirect block

Indirect block

...

Reference count

9

I-node structure

• mode: A. type: file, directory, pipe, symbolic link B. Access: read/write/execute (owner, group,)

• owner: who own this I-node (file, directory, ...)• timestamp: creation, modification, access time• size: the number of bytes• block count: the number of data blocks• direct blocks: pointers to the data• single indirect: pointer to a data block which

pointers to the data blocks (128 data blocks).• Double indirect: (128*128=16384 data blocks)• Triple indirect: (128*128*128 data blocks)

10

Data Block

• A data block has 512 bytes. A. Some FS has 1K or 2k bytes per blocks.B. See blocks size effect (next page)

• A data block may contains data of files or data of a directory.• File: a stream of bytes. • Directory format:

i-# Next size File name pad

11

Report.txt

home

john

bin

find

alex jenny

notes

grep

i-# Next 10 Report.txt pad i-# Next 3

bin pad i-# Next 5 notes pad 0 Next

12

Block size

Max Accessible Capacity (bytes)Blocksize

Directblock

IndirectBlock

DoubleIndirectblock

Tripleindirectblock

512 5120 64K(128)

8M 1G

1K 10240 256K(256 )

64M 16G

2K 20480 1M(512)

512M 256G

13

Current Working Directory

login: kcpassword:******

Welcome to UNIX !$ pwd/home/kc$ cd /usr/kc/source$ pwd/usr/kc/source

• pwd: print current working directory

Report.txt

home

kc

source

find

alex

notes

grep

14

Changing Working Directory

• The cd utility makes another directory the working directory.

$ cd /home/alex/literature$ pwd/home/alex/literature$ cd$ pwd/home/kc

15

List Files

• list the contents of directories• Example: list the current working direcory

$ ls Report.txt sourcenotes

16

Currentdirectory

Per-processdata region

Boot Block

...

SuperBlock

i-node

i-node

i-node

i-node

...

...

...

Current Dir

Report.txt

source

notes...

...

...

...

I-nodes

Data Blocks

17

List Files

• list with options• -l: long format$ ls -l-rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txtdrwxrwxr-x 2 kc pubs 512 May 5 14:03 sourcedrwxr-xr-x 1 jo staff 512 Jun 3 1997 notes

permissions

number oflinks

owner groupuser

Size Last access timecode

File ordirectoryname

18

File Code (types)

code meaning

- Ordinary file

d Directory file

c Character special file

b Block special file

l Symbolic link

p FIFO (pipe) file

19

Remove Files

• rm fileName: remove fileName in current directory• rm * : remove all files in current directory• rm -rf dirName: remove all the files in dirName• rm -i fileName: interactive option

$ ls -l-rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txtdrwxrwxr-x 2 kc pubs 512 May 5 14:03 sourcedrwxr-xr-x 1 jo staff 512 Jun 3 1997 notes$rm *$ls -ltotal 0$

20

Move Files

• mv file or files to someOtherDir

$ ls -l-rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txtdrwxrwxr-x 2 kc pubs 512 May 5 14:03 sourcedrwxr-xr-x 1 jo staff 512 Jun 3 1997 notes$mv * ../notes$ls -ltotal 0$ls -l ../notes-rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txtdrwxrwxr-x 2 kc pubs 512 May 5 14:03 sourcedrwxr-xr-x 1 jo staff 512 Jun 3 1997 notes

21

Copy Files

• Copy file will duplicate files

$ ls -l-rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txtdrwxrwxr-x 2 kc pubs 512 May 5 14:03 sourcedrwxr-xr-x 1 jo staff 512 Jun 3 1997 notes$ls -l ../notestotal 0$cp * ../notes$ls -l ../notes-rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txtdrwxrwxr-x 2 kc pubs 512 May 5 14:03 sourcedrwxr-xr-x 1 jo staff 512 Jun 3 1997 notes

22

Links

• link: a reference to a file or directory• two kinds of link: hard links and soft link• hard link: ln file1 file2

(create a hard link file2 to file1)• soft link: ln -s file1 file2

(create a soft link file2 to file1)

23

Hard Links(I)$ ls -l-rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txtdrwxrwxr-x 2 kc pubs 512 May 5 14:03 sourcedrwxr-xr-x 1 jo staff 512 Jun 3 1997 notes$ln Report.txt hardLink.txt$ls -l -rw-r--r-- 2 kc pubs 3355 May 2 10:52 Report.txt-rw-r--r-- 2 kc pubs 3355 May 9 7:22 hardLink.txtdrwxrwxr-x 2 kc pubs 512 May 5 14:03 sourcedrwxr-xr-x 1 jo staff 512 Jun 3 1997 notes$rm Report.txt$ls -l -rw-r--r-- 1 kc pubs 3355 May 9 7:22 hardLink.txtdrwxrwxr-x 2 kc pubs 512 May 5 14:03 sourcedrwxr-xr-x 1 jo staff 512 Jun 3 1997 notes$more hardLink.txtthe content is shown here

24

Hard Links (II)

i-# Next 10 Report.txt pad

i-# Next 12 hardLink.txt pad

Reference count=2

Description of filei-node File

25

Remove Hard Links (III)

i-# Next 12 hardLink.txt pad

Reference count=1

Description of filei-node File

26

Soft Links (I)$ ls -l-rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txtdrwxrwxr-x 2 kc pubs 512 May 5 14:03 sourcedrwxr-xr-x 1 jo staff 512 Jun 3 1997 notes$ln -s Report.txt softLink.txt$ls -l -rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txtlrwxrwxrwx 1 kc pubs 3355 May 9 7:22 softLink.txtdrwxrwxr-x 2 kc pubs 512 May 5 14:03 sourcedrwxr-xr-x 1 jo staff 512 Jun 3 1997 notes$rm Report.txt$ls -l lrwxrwxrwx 1 kc pubs 3355 May 9 7:22 softLink.txtdrwxrwxr-x 2 kc pubs 512 May 5 14:03 sourcedrwxr-xr-x 1 jo staff 512 Jun 3 1997 notes$more softLink.txtsoftLink.txt: No such file or directory$

27

Soft Links (Symbolic) II

i-# Next 10 Report.txt pad

i-# Next 12 softLink.txt pad

Reference count=1

i-nodeFile

Reference count=1

Description of file

i-node

i-node/home/kc

/Report.txt

28

Soft Links (Symbolic) III

0 Next 10 Report.txt pad

i-# Next 12 softLink.txt pad

Reference count=1i-node

/home/kc/Report.txt

29

Change File modes

• Permission type: r: read w:write x:execute• Permission object: user(u), Group(g), others(o), all(ugo)• chmod: change permission

+ add permission = set permission- remove permission

• Example:

• chmod -R 755 SomeDir

$ chmod a+rw Report.txt$ ls -l Report.txt-rw-rw-rw- 1 kc pubs 3355 May 2 10:52 Report.txt

30

Change Owner and Group

• change owner: you need to have the right (such as superuer) to do so

$ chown john Report.txt$ ls -l Report.txt-rw-rw-rw- 1 john pubs 3355 May 2 10:52 Report.txt$ chgrp staff Report.txt$ ls -l Report.txt-rw-rw-rw- 1 john staff 3355 May 3 11:45 Report.txt

31

Create & Remove Directories

• mkdir: create a new directory• rmdir: remove a directory

Report.txt

home

kc

source

find

alex

notes

grep

$ mkdir /home/kc/notes/UNIX$ ls /home/kc/notesUNIX$rmdir /home/kc/notes/UNIX…$rm -r /home/kc/source

UNIX

32

Search Files/directories

/

home

john

temp literature

promo

demo

$find /home -name demo -print/home/alex/demo/home/john/demo

alex

demo

• find: locate the misplaced files (or directories)

33

Search Files/directories

$ find /usr -name ‘v*[0-9]’ -print/usr/share/lib/terminfo/v/vt100/usr/share/lib/terminfo/v/vt52….

• locate files/directories that start with a v and end with a digit

34

Search Files/directories

$ find /usr -size +1000 -print/usr/lib/libc.so/usr/lib/libnsl.so….

• locate files whose size have more than 1000 blocks

35

Search Files/directories

$ find . -mtime -1 -print./notes/softLink.txt./notes/hardLink.txt….

• locate files that have been modified within the last day (24 hrs)

36

Search Files/directories

$ find / -name core -exec rm {} \;$ find / -name core -ok rm {} \;

Lab: $ mkdir d1 d2 d3$ cp anyFile d1/xxx$ cp anyFile d2/xxx$ cp anyFile d3/xxx$ find . -name xxx -ok rm {} \;

• locate files and then remove them

37

Compression

• Two lossless compression: compress and pack• compressed files: *.z

$ ls -l-r-xr-xr-x 1 kc other 18292 Feb 4 10:52 ls-r-xr-xr-x 1 kc other 161260 Feb 4 10:52 vi-r-xr-xr-x 1 kc other 64148 Feb 4 10:53 who$ pack *UX:pack: INFO: ls: 24.7% CompressionUX:pack: INFO: vi: 17.9.7% CompressionUX:pack: INFO: who: 19.6% Compression$ ls -l-r-xr-xr-x 1 kc other 13775 Feb 4 10:52 ls.z-r-xr-xr-x 1 kc other 132403 Feb 4 10:52 vi.z-r-xr-xr-x 1 kc other 51581 Feb 4 10:53 who.z

38

Compression

• Two lossless compression: compress and pack• compressed files: *.Z

$ ls -l-r-xr-xr-x 1 kc other 18292 Feb 4 10:52 ls-r-xr-xr-x 1 kc other 161260 Feb 4 10:52 vi-r-xr-xr-x 1 kc other 64148 Feb 4 10:53 who$ compress -v *ls: Compression 34.22% -- replaced with ls.Zvi: Compression 34.43% -- replaced with vi.Zwho: Compression 30.90% -- replaced with who.Z$ ls -l-r-xr-xr-x 1 kc other 12032 Feb 4 10:54 ls.Z-r-xr-xr-x 1 kc other 105729 Feb 4 10:54 vi.Z-r-xr-xr-x 1 kc other 44323 Feb 4 10:55 who.Z

39

Compression

• Restore compressed files(*.z): unpack• Restore compressed files(*.Z): uncompress

$ unpack *$ ls -l-r-xr-xr-x 1 kc other 18292 Feb 4 10:52 ls-r-xr-xr-x 1 kc other 161260 Feb 4 10:52 vi-r-xr-xr-x 1 kc other 64148 Feb 4 10:53 who$ uncompress *$ ls -l-r-xr-xr-x 1 kc other 18292 Feb 4 10:54 ls-r-xr-xr-x 1 kc other 161260 Feb 4 10:54 vi-r-xr-xr-x 1 kc other 64148 Feb 4 10:55 who

40

Collect Files

• tar(tape archive): collect files into a single file• Space are allocated in clusters of two, four or even eight blocks at a time. (block size = 512 bytes) • Large disks usually have large cluster size.• If you create a file containing just a single character, Unix system will typically allocate 2 or 4 512-byte blocks• Thus compressing a 500-byte file into 256 bytes won’t save any space.• Collecting (tar) small files into a single file will save space.

41

Collect Files

• create a tar file: -c

$ ls -l *-r-xr-xr-x 1 kc other 18292 Feb 4 10:52 don.ltr-r-xr-xr-x 1 kc other 161260 Feb 4 10:52 don2.ltr-r-xr-xr-x 1 kc other 64148 Feb 4 10:53 jane.ltr$ tar -cvf aTarFile.tar *ltra don.ltr 37 tape blocksa don2.ltr 292 tape blocksa jane.ltr 129 tape blocks$ ls -l *-r-xr-xr-x 1 kc other 18292 Feb 4 10:52 don.ltr-r-xr-xr-x 1 kc other 161260 Feb 4 10:52 don2.ltr-r-xr-xr-x 1 kc other 64148 Feb 4 10:53 jane.ltr-r-xr-xr-x 1 kc other 243700 Feb 4 10:55 aTarFile.tar

42

Collect Files

• list the content of a tar file: -t

$ ls -l *-r-xr-xr-x 1 kc other 18292 Feb 4 10:52 don.ltr-r-xr-xr-x 1 kc other 161260 Feb 4 10:52 don2.ltr-r-xr-xr-x 1 kc other 64148 Feb 4 10:53 jane.ltr-r-xr-xr-x 1 kc other 243700 Feb 4 10:55 aTarFile.tar$ tar -tvf aTarFile.tar-r-xr-xr-x 1 kc other 18292 Feb 4 10:52 don.ltr-r-xr-xr-x 1 kc other 161260 Feb 4 10:52 don2.ltr-r-xr-xr-x 1 kc other 64148 Feb 4 10:53 jane.ltr$ rm *.ltr

43

Collect Files

• extract a tar file: -x

$ tar -xvf aTarFile.tar $ ls -l *-r-xr-xr-x 1 kc other 18292 Feb 4 10:52 don.ltr-r-xr-xr-x 1 kc other 161260 Feb 4 10:52 don2.ltr-r-xr-xr-x 1 kc other 64148 Feb 4 10:53 jane.ltr-r-xr-xr-x 1 kc other 243700 Feb 4 10:55 aTarFile.tar

44

Examine File Type

• file command determines the type information of files.• Type information: directory, binary file, ascii file

$ lsReadme makefile meggaa.c zsrc$ file *Readme: ascii textmakefile: ascii textmeggaa.c c program textzsrc: directory

45

Examine Disk Usage

• du command show how much disk storage (blocks) of your files.

$ ls -l *-r-xr-xr-x 1 kc other 18292 Feb 4 10:52 don.ltr-r-xr-xr-x 1 kc other 161260 Feb 4 10:52 don2.ltr-r-xr-xr-x 1 kc other 64148 Feb 4 10:53 jane.ltr$ du37 ./don.ltr 292 ./don2.ltr129 ./ jane.ltr$ du /ect/fc…$ du -s /ect/fc2750 /ect/fs

46

Examine Free Disk Space

• df command show how much free disk storage (blocks) • some system you may use -k option to show K bytes.

$ df -k Filesystem kbytes used avail cap mount/dev/dsk/c0t3d03s0 118679 83517 23302 79% //dev/dsk/c0t3d03s2 1970068 1556936 216132 88% /homesswap 26728 164 26564 1% /tmp$ df -k .Filesystem kbytes used avail cap/dev/dsk/c0t3d03s2 1970068 1556936 216132 88% /home2$ df . /home2 (/dev/dsk/c0t3d03s2 ): 826248 blocks 954962 files

47

Dump Files

• od (octal dump) command show a file with octal, decimal, ASCII, hexadecimal format.

$ cat spicesthyme nutmegsage cuminsalt pepper$ od -c spices000000 t h y m e \t n u t m e g \n s a g000020 e \t c u m i n \n s a l t \t p e p000040 p e r \n000044

48

On-line Manual

• Always use on-line manual (man) to find detail syntax and options

$ man cat ...$ man od ...

49

End of File Management Lecture