File System Implementation - Gadjah Mada Universityte.ugm.ac.id/~wibirama/tif206/01/ch12/ch12-presentation.pdf · File-System Implementation ... NTFS: master ﬁle table) Per-ﬁle

File System ImplementationSunu Wibirama

Thursday, December 16, 2010

11.

Outline File-System Structure

File-System Implementation

Directory Implementation

Allocation Methods

Free-Space Management

Discussion


11.

File System Structure File system is provided by OS to allow the data to be

stored, located, and retrieved easily Two problems on file system design:

How the file system should look to the user Algorithms and data structures to map logical file

system onto the physical secondary-storage devices

File system organized into layers, uses features from lower levels to create new features for use by higher levels.

I/O Control controls the physical device using device driver

Basic file system needs only to issue generic commands to the device driver to read and write physical block on the disk (ex. drive 1, cylinder 73, track 2, sector 11)


11.

File System Implementation

File organization module knows physical and logical blocks, translating logical block address to physical block address. It also manages free-space on the disk

Logical file system manages metadata information (all file system structure except the actual data or contents of the file).

Logical file system maintains file structure via file-control blocks (FCB)

File control block – storage structure consisting of information about a file


11.

A Typical File Control Block


11.

On-Disk File System Structures

Boot control block contains info needed by system to boot OS from that volume (UNIX: boot block, NTFS: partition boot sector)

Volume control block contains volume details (UNIX: superblock, NTFS: master file table)

Directory structure organizes the files (UNIX: inode numbers, NTFS: master file table)

Per-file File Control Block (FCB) contains many details about the file


11.

In-Memory File System Structures

It is used for file-system management and performance improvement (via caching).

The data are loaded at mount time and discarded at dismount.

The structures including: In-memory mount table: information of each mounted

volume In-memory directory structureSystem-wide open-file table: a copy of FCB of each open

filePer-process open-file table: a pointer to the appropriate

entry in the system-wide open-file table, as well as other information based on process that uses the file.


11.

New File Creation Process

Application program calls the logical file system

Logical file system knows the directory structures. It allocates a new FCB.

The system then reads the appropriate directory into memory, updates it with the new file name and FCB, and writes it back to the disk.

Now, the new created file can be used for I/O operation, which will be explained in the next slide


11.

In-Memory File System Structures


11.

Directory Implementation

Directory-allocation and directory-management algorithms significantly affects the efficiency, performance, and reliability of the file system.

Linear list of file names with pointer to the data blocks. simple to program time-consuming to execute, because it requires a

linear search to create or delete file.

Hash Table – linear list with hash data structure. decreases directory search time problem: fixed size of hash table


11.

Allocation Methods

Many files are stored in the disk

How to allocate space to these files so that disk space is utilized effectively and files can be accessed quickly

An allocation method refers to how disk blocks are allocated for files:

Contiguous allocation

Linked allocation

Indexed allocation


11.

Contiguous Allocation

Each file occupies a set of contiguous blocks on the disk Simple – only starting location (block #) and length (number of blocks)

are required Both sequential and direct access are supported Disadvantages:

Wasteful of space (dynamic storage-allocation problem) File cannot grow External fragmentation: free space is broken into chunks

One of several solutions: use a modified contiguous allocation scheme Extent-based file systems allocate disk blocks in extents An extent is a contiguous block of disks

Extents are allocated for file allocation A file consists of one or more extents


11.

Contiguous Allocation of Disk Space


11.

Linked Allocation

Linked allocation solves all problems of contiguous allocation

Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk.

Ex: File “Jeep” Start at block 9 Then: block 16, 1, 10 Finally end at block 25

pointer (4 bytes)

1 block (512 bytes) 508 bytes

visible part to user


11.

Linked Allocation

Advantages: Free-space management system – no waste of space File can grow, depends on available free blocks

Disadvantages: No random access (only sequential access) Space required for pointers (0.78 percent of the disk is being used for pointers, rather

than for information) ~> solution, uses clusters (unit of blocks) Reliability, pointer damage will cause unlinked blocks in a file.

FAT (File Allocation Table): variation on linked allocation Located at the beginning of each volume


11.

File-Allocation Table

To do random access: 1. The disk head move to the start of volumeto read the FAT2. Find the location of the desired block3. Move to the location of the block itself


11.

Indexed Allocation Brings all pointers together into the

index block Each file has its own index block, which

is an array of disk-block addresses Directory contains the address of index

block Support direct access without external

fragmentation Each file has its allocation for all

pointers, so that it has wasted space greater than linked allocation (which contains one pointer per block).

We want the index block as small as possible, then we have several mechanisms: 1. Linked scheme2. Multilevel index3. Combined scheme


11.

Indexed Allocation

Linked Scheme An index block is normally one disk block Large files -> we can link together several index blocks Ex.: an index block contains:

- a small header of file name- a set of 100 disk-block addresses- nil (for small file) or a pointer to another index block (for a large file)

Multilevel index First-level index block points to second-level index blocks which in turn point to the

file blocks (see next slide) Combined scheme

In unix, for example: 15 pointers in fileʼs inode 12 first pointer: direct blocks, for small file (no more than 12 blocks). If the block

size is 4KB, then up to 48KB (12 x 4KB) can be accessed directly The next pointers point to indirect blocks, which implement multilevel index based

on their sequence (see next two slide)


11.

Multilevel Index

1st-level index block

2nd-level index block file

Back to Indexed Allocation


11.

Combined Scheme: UNIX UFS (4K bytes per block)

Back to Indexed Allocation


11.

Free-Space Management Free-space list concept Bit vector (n blocks)

…

0 1 2 n-1

bit[i] = 0 ⇒ block[i] free

1 ⇒ block[i] occupied

Block number calculation

{(number of bits per word) *(number of 0-value words)} +offset of first 1 bit

Bit vector requires extra space. Example:" " block size = 212 bytes" " disk size = 230 bytes (1 gigabyte)" " n (amount of bits) = 230/212 = 218 bits (or 32K bytes)


11.

Free-Space Management Linked list

Link together all the free disk blocks Keeping a pointer to the first free block in a

special location on the disk and caching it in memory.

The first block contains a pointer to the next free disk block...

Must read each block to traverse list, increase I/O operation time.

Grouping Storing the address of n blocks in the first free

block. n-1 blocks are actually free blocks but the last

block contains the addresses of another n free blocks.

Counting Keep the address of the first free block and n of

free contiguous blocks that follow the first block. Each entry in free-space list consists of disk

address and a count


::DiscussionThursday, December 16, 2010

FAT 32

•FAT - 32 (File Allocation Table - 32 bits)

•Maximum size of file: 232 - 1 byte

•The last byte cannot be allocated to the file so that no file has file size bigger than 0 x FFFFFFFF (4, 294, 967, 296)

•You can convert to NTFS, or split your file. Each method has its advantages and disadvantages.


Fragmentation

http://en.wikipedia.org/wiki/Defragmentation




Why Linux rarely needs defragmentation tools?

• Does fragmentation occur in Linux? Yes, but in very small quantity

• Block Groups: group file-data together in ‘clumps’ to manage small and large file (remember combined scheme in indexed allocation)

• Only write files to unused portion of the disk that are not predictably being fragmented in shorter time.


Combined Scheme

Possible to allocate bigger blocks for a file


Unix System• Keep fragmentation level below 20%

• More than 20%? You certainly need to fix your hard disk using shake-fs

• Run : e2fsck -nv /dev/sda1 as root, resulting:

Fragmented Part


Implementation (*you should have known this before...)

http://geekblog.oneandoneis2.org/index.php/2006/08/17/why_doesn_t_linux_need_defragmenting

Empty hard disk (*simplified assumption)Thursday, December 16, 2010





Start with FAT file system....



I have hello.txt



I have hello.txt

OK, now add bye.txt



I have hello.txt

OK, now add bye.txt

I want to change hello.txt, dude...?



Just copy, delete the original content, and wrap it up in the

larger space.....

1st approach


Just copy, delete the original content, and wrap it up in the

larger space.....

Or, Put your extended file content to

the next space.....

If the first approach requires huge read and write operation, then the most possible approach is the second one. That’s why FAT

suffers from large fragmentation

1st approach

2nd approach


Initial condition

What About Linux?


What About Linux?

Add bye.txt


Change hello.txt

What About Linux?


Thank You


Documents

File System Implementation - Gadjah Mada Universityte.ugm.ac.id/~wibirama/tif206/01/ch12/ch12-presentation.pdf · File-System Implementation ... NTFS: master ﬁle table) Per-ﬁle