Joshua Xu / 许建华

中国上海 :电 话： 21-62470421,62470432地 址：上海市南京西路 1468 号东海电脑广场 3 号楼八层Email ： joshua@itlab.com.cn

新加坡：Tel : (65) 67760365Address: Blk432, #07-274 Clementi Avenue 3 120432

www.itlab.com.cn

CONFIDENTIAL

Unix/Linux Basic Course

Joshua Xu

高端软件培训Joshua@itlab.com.cn

Objectives(5 days)

Unix/Linux Operating System BackgroundUnix/Linux Basic commandsFile and Directory ManipulationEditor(vi)User and Process managementShell programmingMakefileLinux Installation and Setup

Agenda

上午 下午Unix/Linux 基本命令 文件系统VI(编辑器) 用户,进程管理,LINUX 安装SHELL 编程 SHELL 编程SHELL 编程 SHELL 编程SHELL 编程 Makefile

Reference Book

Unix

HistoryOther Compatible Unix Systems

Linux

MINIX (PC Unix) 1991 (Andrew S. Tanenbaum)

Linux 0.021991

(Linus benedict Torvalds)Linux 2.x.x POSIX Compliant

to current version

Linux VS Unix(1)

Linux is a real Unix, Functionality(Multi User, MultiTask,…)

POSIX compliant(from Ver2.x.x)System V & BSD compatible

Linux VS Unix(2)

Open Operating System

Code

System CustomizingMinimum Hardware RequirementsSupport Multi File System(minix,ext,fat,…..)Software Resources and Powerful FunctionalityUnix is certified to be the robust system, Linux no

t yet…

Unix VS Windows(1)

Hardware platform and Location

Reliability or Robustness

Unix VS Windows(2)

User interface:ls option parameters

ls ls -al

Redirection and pipe

grep

find

Multi task

Unix VS Windows(3)

Multi users

Unix VS Windows(4)

Security

System know who you are

File and directory priority

More About Unix/Linux

What can it do?

Production Line: Web Server, Mail server, Database Server

Program Development

Who should know Unix/Linux?

Unix administrator, System Manager, System Consultant

Oracle Administrator, Network Engineer, Program Developer

Linux/Unix two FunctionsWhat are the two functions of OS? How to?

Resource Management(1)

RAM, DISK difference

rams are paged

Resource Management(2)

CPU

eg: ticket system

Resource Management(3)

intangible resources’ management

opened files

sockets

Locks

Be Aware of Case Sensitive!!!

shutdownshutdown -[rhk] <time> warning

<time>:

hhmm[am|pm] MMDDYYYY + offset

MMDDYY JAN 20 2004 NOON MIDNIGHT

TEATIME now

Offset:n[minute,hour,day,week]

telnet

telnet 192.168.100.1 [port]

Add item into file: c:/windows/system32/desktop/driver/hosts

login

login: ( username)

password: (user’s password)

Logout

Logout or exit or ^D

Passwd

passwd

uname

uname -a

Exit

Exit or ^D

Telnet

What’s the telnet?

a program a main function with a window

how does it work?

a client connected to a telnet server

a protocol

ftp

ftp program, ftp command, ftp server

ftp client, ftp protocol

? --help

ls cd bin & asc get & put mget & mput bye

Terminal

What’s the terminal?

What’s console, terminal, pseudo-terminal

tty0 tty1 pts/0

Echo $TERM

stty -a /stty all

stty erase ^? intr ^C kill ^U eof ^D swtch ^Z quit ^\

^J stty sane ^J

logname

logname

whoami

whoami

who

Who

each user has an account, has a

password and a home directory

datedate

date –u

date +%Y/%m/%d

to set date:

date MMDDHHMMCCYY ( this need root priority)

calcal -y –j month year

clear

clear

echo

Echo

Echo –e abc“\a” -e:enable control char

\a alarm \c remove newline \n newline

\t tab \\ “\” \b backspace \f new page

\r linefeed \v vertical tab \nnn octal digit

write

write uid@host

Write tty(n)

Write ttyp(n)

mesg

mesg -n –y

:you cann’t mask root’s message

mesg joshua

mail

mail

manman -n command

1 command 2 syscall 3 lib 4 special file

5 file format info …

su

su

password: (root’s password)

--------

What’s file?

File is the storage of data, program

File is the most important data structure of operating system

The File Systems

Path Name

Path name uniquely specify particular file or directory

Path name separated by “/”A “/” in the first position represents the

root directory

ls(1)

ls

option: -a –A –l –v --help Note: file name rule no “/”

1 !@#$%&*()[]{}’ ” \ | / ; < > SPACE

2 case sensitive

3 .txt .doc .sh .html .pl .c .o .tar .gz .rmp

File Attribute(1)

File Attribute(2)

Type: - d l b c s p to see /dev/hda1 /dev/cdrom …

File Permission

chmod(1)

Mode value

chmod(2)

chmod a-w,o-rx *.cat

Meta-Characters

* ? [adeg] [0-9a-zA-Z] [!a-zA-Z]

ls(3)ls app*

ls [a-zA-Z]*

ls –al [!0-9]*

ls *.doc

ls fs??*s

ls /app*/usr? Conf.??.log

pwd

pwd print working directory

cdcd

cd ab*

cd ~

cd -

cd ~alice

cd ~/bin

CDPATH shortcut way to a directory

Absolute Path name

A pathname uniquely identifies a particular file or directory by

Specifying its location

The slashes within pathnames separate object names

A slash in the first position represents the / (root) directory

Relative Path Name

Specifies a file in relation to the current working directory are useful because they are shorter than absolute pathnames

Do not begin with a slash

cat

cat file1 file2 file3 >file4

cat > file1

:generate one text file with cat

Redirection & PIPE

What’s standard input output?

They are fd 0 1 2 (error)

/dev/null

< > >> |

Cat >&1 inout share the same fd

Cat > /dev/tty3

cp

cp

mv

mv

mkdir

mkdir

rm

Rm a*out*tmp

Rm file[0-9].tmp

Rm -i need confirm

-f ignore error report if file not exist

-R remove directory and its sub-directory

-d remove the directory even if not empty (root pri)

ln(1)

ln(2)

ln spacewar funky two items point to same file

ln /usr/harley/data1, /usr/harley/data2 /usr/addie/work

Or ln /usr/harley/data[12] /usr/addie/work

you let addie share your data

ln –s only a symbol, the file contain some characters

touch

touch filename

How to process a file name with special letter?

chown

chgrp

umask

umask

grep(1)

grep(2)Grep “sort” *.doc grep “sort it” *

Grep –c “48” data.f

Grep “48” data.f

Grep –n “48” data.f

Grep –v “48” d-ata.f

Grep “48 ” data.f grep ‘48\>’ data.f

Grep –i “sept” data.f

Grep ’48[34]’ data.f

Grep ‘^[^48]’ data.f

Grep ‘[Ss]ept’ data.f | grep 483

grep(3)

Grep ‘^[^48]’ data.f

Grep ‘[Ss]ept’ data.f | grep 483

Grep ‘K…D’ data.f

Grep ‘[A-Z][A-Z]..C’ data.f

Grep ‘5..199[6,8]’ data.f

Grep ‘[0-9][0-5][0-6]’ data.f

Grep ‘^[0-9][0-5][0-6]’ data.f

Grep ‘4\{2,\}’ data.f

Grep ‘9\{3,\}’ data.f

grep(4)

fgrep not support regular expression

egrep enhanced regular expression

egrep “([Ss]ome|[Aa]ny) one”

zgrep zipped files

Regular Expression

^ $ [] [^] *

Enhanced regular expression:

“([Ss]ome|[Aa]ny) one”

Find(1)find path expression action

options: -[i]name -type -link –perm

–size –user –atime –mtime –newer

operator -a –o ! ( )

.and. .or. .not. ( )

file type: f d c b l s p(pipe)

Find(2)

find ~/. –name “*[aA]ug*” –print

find ~/. –name *[aA]ug* –print

find ~/. –name \*\[aA\]ug\* –print

Filter commands(1)cat

grep

sort file1 >file2 output to same file, use –o option

cat file1 file2 | grep xxxx | sort | lp

-b ignore space and tab –r reverse order

-c(if already) –d(dictionary order) –f(case insensitive) –n(digit order)

–r(reverse) –u(same line occurs one time)

cut –c 1,8,10-20 file1

Filter commands(2)

Sed to processing string

Tr -[cds] [string1] [string2] <file1 > file2 replace string1 with string2

Header -n

display head n lines, default 10 lines

Tail -n

display n tail n lines, default 10 lines

wc

diff

Find difference between files

diff –[bef] old-file new-file

Device Files(1)

Block: Non_IDE interface CDROM: aztcd,bpcd,cdu31

a,cm260cd… IDE interface: hard disk,CDROM,etc.

Hd[num][partition num] SCSI interface: hard disk, CDROM,MO etc.

Sd[num][partition num] Floppy disk: fd[num][density]

fdaH1440,fbH1440

Two types of Device files : Block and Char

Device Files(2)

Char: Ports

serial : cua0-- cua3 parallel : lp0-lp2

Terminal: tty pty console Taper : nrft0-nrft3 Null device :/dev/null /dev/Zero(for read) Symbol link : /dev/cdrom /dev/hdb

fdformat

fdformat

mkfs

mkfs

mkdosfs

umask

mount

mount /dev/cdrom /mnt/cdrom

mount /dev/floppy /mnt/floppy

unmount /dev/cdrom or /mnt/cdrom

unmount

umask

dd

Dd duplicate floppy disk

df

df show file system informations

du

du display file space usage

Backup

tar –cvf d.tar /data

tar –czvf d.tar /data

tar –x d.tar

gzip

gzip filename

Exercises

vi

Vi file1 Open file1, if file1 not exist, create it

Vi Open a new file and name it later

View file1 Open file1 as read-only

Vi(states)

Vi(move)

Vi(insert)

Vi(delete ）

Vi(undo ， change ）

Vi(copy and paste ）

Y line copy (or yy)

p paste under the current line

P paste above the current line

:1,2 co 3 append line 1 to line 2 toline 3

:4,5 m 6 move line 4 to line5 to line6

Vi(save and quit ）Save

:w save if the file already exist

:w file1 save as a new file file1

:wq save and exit (ZZ)

:q! ignore change and exit

Vi(options ）:set nu set line number (:set nonu)

:(number) go to line (number) :21 or 21G

G go to the end of file

/string forward search string

?string backward search string

N repeat

:1,$s/oldstring/newstring/g global string substitution

:r file1 insert another file

Vi(exercises ）

--

Network Basic

Client-Server

Networking Model

Common Protocols and Applications

TCP/IP Networking Addressing

Networking Services

TCP/IP Networking Addressing

Important files and utilities

Ifconfig

ping

finger

Netstat - rn

traceroute www.itlab.com.cn

--

useradd userdelProhibit temp usersfinger usernamegroupadd –g groupdel

useradd

Useradd userid

Passwd userid

userdel

userdel

Prohibit temp user

passwd –l username

passwd –u username

finger

finger username

Groupadd

groupadd –g

Group delete

groupdel

telnet

ftp

rlogin

rusers

--

进程管理 ps –acdef kill -9 定时执行 at Batch crontab

Ps(1) ps (same group)

-a (other), -e(all)

querying:

-C httpd G gid U user

g (session title, ) p (ID) t (tty) u (uid)

format:

-f(full) j (working style) l(long) o(user defined) y(no tag) j(working control)

l (long) o(set user defined format)

s(signal format) u(user guidance) v(virtual memory)

Ps(2) process status: D uninterruptible sleep, usually IO R runnable process S sleeping T stopped or traced Z zombie process W no resident page < high priority process N low priority process L memory locked process

Ps(3) Process attribute: USER

%CPU percentage occupation

%MEM

SIZE

RSS (resident size may be smaller than SIZE

TTY

STAT status

START start time

TIME time executed

FLAGS only administrator need know

UID PID PPID PRI (time slice) NI (priority)

WCHAN kernel function name when process sleeping

kill

Kill -9

nice

• nice

• renice

nohup

Nohup results backend process running without interrupted

Nohup command & (>nohup.out)

jobs

fg

fg %N job number N

%string command begin with the string

%?string command contain the string

%+ last backend process

%% same as above

%- second last backend process

bg

Bg command brings jobs to backend

at

1.

At 21:00

At> command

^D

2. Echo command | at now + 2 minutes

batch

Batch is at similar command, the command will be run at system leisure time

CrontabVi Cronjobcommandscript

<minute> <hour> <day> <month> <weekday> <command>

30 21 * * * action

45 4 1,10,20 * * action

0,30 18-23 * * * action

Vi usercron

Crontab usercron

Crontab –l – e –r

--

Shell programming

KernelManage devices, memory and processes

Controls the functions between the system programs, utilities and the system hardware

Manage swap space, daemons, file systems and other functions

What’s the shell

Shells: chsh -l

Which kind of Shell we are using?

Who needs know it?

Unix/Linux administrator

DBA

Programmer

BASH

Echo $SHELL

Whereis bash

Which bash

Change shell

Usermod –s shellname username

Chsh -l

Chsh –s shellname username

Shell command

Command1;command2;command3

Command1;(command2;command3);command4

Shell Script

A file contains shell commands

#!/bin/bashCal 1 $1Cal 2 $1Cal 3 $1

Sub shell(1)What’s sub shell?

Inherited: Current directory Environment variables File descriptor 0,1,2 Ignored signal

Uniherited: User defined variables without exporting ignored signal

Sub shell(2)How to understand sub shell:

1 environment variables

2 Delcared variable with –x option

3 Print SHLVL in script

Define a shell variable

DISPLAY=192.168.0.10:0.0

X=12

echo $DISPLAY $X

echo ${DISPLAY} ${X}

Using declare statement

declare –a –f –i –r –x -F

Variable type

Val1=12 val2=5

Result1=val1*val2

Echo $result1 default is string

Declare –i val3=12 val4=5

Declare –i result2

Result2=val3*val4

Echo $result2

Un-define shell variable

Unset Display

Unset X

Array

Names[0]=alice Names[1]=tom Names[2]=jimmy

Names=(alice tom jimmy) Names=(alice [2]= tom jimmy)

${Names[0]} ${Names[@]}

${#Names[0]} ${#Names[@]}

All kinds of Variable

System defined Environment variable

env

Built-in variable

IFS RANDOM (0-32767) $@ $* $# $? $$ $+ $- $<

User defined variables

Command line parameter $0 $1 $2 $3 $4…

Math operator and expressions+ - * / % << >> & | ~ ! ^

< > <= >= == != && ||

$((3>2))

$(($RANDOM%10+1))

$((3>2)||(4<=1)))

If true return 1 otherwise return 0

Echo “only $(((365-$(date +%j))/7)) weeks until the New Year”

B#N

Let intvar=expression

references

For string: “ ” ‘ ’ { } b{ed,olt,ar}s ` `

String operation(1)• secure variable exists• Set default value• Capture undefined variable error• Delete sub-string of pattern string ${varname:-word} ${varname:=word} ${varname:?word}${varname:+word}${varname:offset}${varname:offset:length}

String operation(2)• Str1=str2

• Str1!=str2

• Str1<str2

• Str1>str2

• -n str str not null

• -z str str is null

String operation(3)

${variable#pattern} ${variable##pattern}${variable%pattern} ${variable%%pattern}${variable/pattern/string}${variable//pattern/string}

function function functionname{

shell command;}Or

function functionname(){

shell command;}

Return and exit return

return N

return only be used in function and source executed script

exit

exit N

other statements:

source break continue, eval,exec,export,

readonly, set, shift, trap, unset

Running Shell Script

chmod a+x abc abc Source ./abc Source ./abc &

eg #!/bin/bash${x:?’x undefined’}echo “exit with notice”

interactive mode

Script location Params

A shell script accepts parameters entered in command line.

#/bin/bash

Cal 1 $1

Cal 2 $1

Cal 3 $1 shift , $0 no change

Expression testtest <logic expression>

[ 3 -gt 2 ]

If true return 0 otherwise return 1

Test operators[! Expression]

[ expression1 –a expression2 ]

[ expression1 –o expression2 ]

[ file existence ]

math operators: -lt -le –eq –ge –gt –ne

[ \(3 –gt 2\) -o \( 4 - le 1 \) ]

note: [ expression ] is equivalent to

test expression

File Attributes -d file file exists and is a directory-e file file exists-f file file is a regular file–s file exists and not empty -r–w –x access priority–O owner -G Group ID match my GIDfile1 –nt file2 file1 is newer than file2file1 –ot file2 file1 is older than file2

Command substitution$(command)

The old style command substitution:

echo “Today is `date`”

Bash new style is

echo $(date)

Flow control(for) #!/bin/bash

for I in 1 2 3 cal $I 2003done

#!/bin/bashfor I cal $I 2003doneabc 1 2 3

Running Shell Script

chmod a+x abc abc Source ./abc Source ./abc &

Script location Params

A shell script accepts parameters entered in command line.

#/bin/bash

Cal 1 $1

Cal 2 $1

Cal 3 $1 shift , $0 no change

Read statement

Read varname

read into varname from input

Read varname < file

read into varname from file1

Flow Control(case)#!/bin/bash

case $1 in

China ) echo Beijing

USA ) echo Washington

British ) echo London

France ) echo Paris

* ) echo Out of my knowledge

esac

capital China

Flow Control(if)if condition

then statement;e

[elif conditionthen statement…]

[else statement]

fi#/bin/bashif test –w /dev/lpthen echo Printer is readyelse echo Printer is not readyfi

Flow Control(while until)while

do

done

until

do

done

Flow Control(<<here)command << label

…

…

label

#/bin/bash

grep $1 <<!

Huang w (21)81828888

Zhang QF (21)67678787

!

Customizing user env.bash_profile, .bashrc

.bash_login, .profile

.bash_logout

If .bash_profile not exist, find .bash_login,.profile respectively .bash_profile only read by login shell

.bashrc read by .bash_profiel or sub bash shell command

alias

Alias lf=‘ ls –F’

Alias printall=‘pr * lpr’

Alias cdvoy=‘cd ipp/demo;animation/voyager ’

Alias del=rm -i

options

Set –o vi

Ignoreeof mask ^D single ^D operation

Noclobber mask “>>”

Noglob mask meta-characters

Nounset no unset

PS1,PS2,PS3,PS4PS1 command line prompt

eg: PS1=‘\u@\h \w\$’

PS2 wait string for further input

PS3 select statement prompt PS4 debug prompt

I/O(1)

> 1> cat >Phones grep “John” Phones >Phone.bak

< 0< cat <Phones grep “John” <Phones tr –s “ ” <Phones

> 2>

I/O(2)

Sort 0< students 1>students.sorted 2>sort.err

Cat lab1 lab2 lab3 2>cat.errors 1>&2

Cat lab1 lab2 lab3 1>cat.output 2>&1

>> append data to file

Cat lab1 lab2 lab3 1>>filename 2>>error.log

I/O(3)

Command1 | command2 |command3

I/O(4)

>|file n>|file <<label <>file

n>>file

&>file 1 2 to file

<&- close 0

>&- close 1

Command processingStep1 –step8 :

step 1: if open key word encountered, continue

if a close key word encountered, error

step 2: is alias, replace it, go to step 1

step 3: { } expands

step 4: ~ substitution

step 5: parameter substitution

step 6: command substitution

step 7: math expression calculation

step 8: keyword recognition

step 9: meta-character expands

step 10: run

Process processing(1)

ps

Jobs

fg %N job number N, suspended process resumes

%string command begin with the string

%?string command contain the string

%+ last backend process

%% same as above

%- second last backend process

Process processing(2)

Signal kill –l to see all the signal

Control key signal ^C intr ^\ terminate ^S ^Q ^Z

Trap

Eval

waitting

In a shell script:

alice &

hatter

wait

: script running will wait until alice finished

Shell debugging

Bashdb

Set –o noexec verbose xtrace

Example#!/bin/bash

trap "nice_try" 2 3 15

TTY=$(tty)

nice_try()

{

echo "nice try, the terminal stays locked"

}

SAVEDSTTY= `stty -g` 1> /dev/null 2>&1

stty -echo

echo -n "Enter your password to lock $TTY"

read PASSWORD

clear

while :

do

read RESPONSE < $TTY

if [ "$RESPONSE" = "$PASSWORD" ];

then

echo "unlocking.."

break

fi

echo "wrong password and terminal is locked"

done

stty echo

stty $SAVEDSTTY > /dev/null

Awk(1)awk –F”x” ‘command’ input-file

mode and action

To see grade.txt

awk ‘{print$0}’ grade.txt

awk ‘{print$1}’ grade.txt > wow

awk ‘{print$1”,”$2}’ grade.txt |tee wow

awk ‘BEGIN {print”Name\tBelt\n--------------------”}\

{print$1”\t”$4} END{print”-------------------\n\

end of printing”} grade.txt

Awk(2)awk ‘{if($3~/Brown/) print$0}’ grade.txt

awk ‘$3==“48” {print $0}’ grade.txt

awk ‘$3!=“48” {print $0}’ grade.txt

awk ‘$4!~/Brown/ {print $0}’ grade.txt

< > >= <= ==

/[Gg]reen/ /^…a/ /(Yellow|Brown)/

/ && / / || /

Awk(3)Built-in variables :

AGC,ARGV,ENVIRON,FIELNAME,FNR,FS,NF,OFS,ORS,RS

Operator:= += *= /= ^= ? || && !< <= == != >=

+ - * / % ^++ --

Awk(4)Add Fields:

awk ‘if($6<$7) {$8=$7-$6;print $1”,”$8}’ grade.txt

File Length calculation:

ls –l | awk ‘/^[^d]/ {print $9”\t”$5}{tot+=$5}

Awk(5)Built-in string operator:

gsub(r,s) awk ‘gsub(/4842/,4899) {print $0}’ grade.txt

awk ‘{print index(“Bunny”,”ny”)}’

gsub(r,s,t),index(s,t),length(s)

match(s,r),split(s,a,fs),sprint(fmt,exp)

sub(r,s) substr(s,p) substr(s,p,n)

Awk(6)mask

\b \n \f \r

Awk(7)printf

- Width .prec

%cdefgosx

awk ‘{printf “%c\n”,$0}’

awk ‘{printf “%-15s”,$0}’

Awk(8)Array:

awk ‘BEGIN {print split(“123#456#678”,myarray,”#”)}’

for (element in array) print array[element]

Example

Rm $(ls –l |awk ‘ { if ($5 >=1000000) print $9 }’ )

If condition is true then print, why is not the “false”?

sedsed –e ‘/foo/d’ myfilesed –e ‘/50d’ myfilesed –e ‘/50,$d’ myfilesed –e ‘s/foo/bar/’ myfilesed –e ‘s/foo/bar/g’ myfilesed –e ‘s/foo/s/***/’ myfile

:do not redirect to >myfilesed –f {command; } <file1 >file2Other command: =(lineno),w(write address),q

(quit)

--

Make and makefile

GCC(G++)Gcc is a common C compiler from GNU

C program compiling four steps: Preprocessing Compiling Assembly Linking

GCC OptionsGcc -o –g –c –O –O2 –I –L –l –static –shared -DDEBUGeg: gcc –DDEBUG …In file.c #ifdefine DEBUG loginfo(char * str) {.. };#endif ? Explain what is preprocessing man gcc for detailed usage

GCC(G++)Gcc is a common C compiler in Linux

C program compiling four steps: Preprocessing Compiling Assembly Linking

makeWhy make?

1

2

makefileMake

Default : GNUmakefile, Makefile, makefile

make –f makefile

Make –d display debug info

Make –h --help

Make –n only parse the command

Make –s no info displayed

example

power: main.o input.o compute.o

gcc main.o input.o compute.o –o power -lm

main.o: main.c main.h input.h compute.h

gcc -c main.c

input.o: input.c input.h

gcc –c input.c

compute.o: compute.c compute.h

gcc –c compute.c

ruleSUFFIXES ： .o .c .s.c.o:

$(CC) $(CFLAGS) –c $<.s.o:

$(AS) $(ASFLAGS) –o $@ $< #power:main.o input.o compute.o

gcc main.o input.o compute.oMain.o : main.h input.h compute.hInput.o : input.hCompute.o : compute.h

macro

Pre-defined symbols

$@(target name) $*(%.*)

$?(newer dependency files)

$<(first depen..) $^(all depen..list)

virtual target

Some target does not exist, you do not have to create it, but some actions can be taken

Special targets

• .defaults

• Ignore

• .PHONY

• .DILENT

• SUFFIXES

LINUX 安装和启动对硬件的要求 (1G Linux 7.2, depends …)分区 (ext2, swap)安装引导过程SetupX WINDOW