The shell is one of the main interfaces to unix that a Systems Administrator uses. The shell is one of the main interfaces to unix that a Systems Administrator uses

The shell is one of the main interfaces to UNIX that a Systems Administrator uses.

• The shell is one of the main interfaces to UNIX that a Systems Administrator uses.

• Interpreter
• Process the commands you enter
• Programming language
• Process groups of commands stored in a file called shell scripts.
• Like other languages, shells have
• Variables
• Control flow commands

There are many different types

• There are many different types

• Bourne shell, sh The original shell from AT&T.
• Korn Shell, ksh
• A superset of Bourne shell that lets you edit the command line.
• C shell, csh Shell for BSD UNIX. Which uses C syntax and has many conveniences.
• modern updates – bash, tcsh modern updates of the Bourne and C shell
• bash is the default Linux shell.
• Most examples used in this lecture are from bash.

Shells are just executable programs.

• Shells are just executable programs.

• Different shells use different syntax and provide different services.

• You can start any shell at anytime

• Example:
• $sh
• $csh

• You exit a shell with

• logout, exit or CTRL-D

• Each user has an entry in the /etc/passwd file which includes the name of the shell to execute.

As a command interpreter, the shell performs the following tasks.

• As a command interpreter, the shell performs the following tasks.

• Wait for the user to enter a command
• Parse the command line
• This is the step this lecture concentrates on
• Find the executable file for the command
• a shell function
• a built-in shell command
• an executable program.
• If the command can't be found generate an error message
• If it is found, fork off a child process to execute the command
• Wait until the command is finished
• Return to step 1

When parsing the command line, bash performs three major steps

• When parsing the command line, bash performs three major steps

• I/O redirection
• Expansion
• Tilde ~, Variable, Command, Arithmetic, Filename , Brace {} etc.
• Remove quote characters

• Much of the above process is achieved using characters with special meanings.

When you enter a command, the shell searches for special characters, and it then performs some special tasks to replace the special characters.

• When you enter a command, the shell searches for special characters, and it then performs some special tasks to replace the special characters.

• Example:
• echo * will not display a *.
• To actually use a special character (e.g. *) as itself you have to quote it
• using a pair of single quotes. echo '*'
• using a single back slash. Echo \*

• Check the actual command after the replacing:

• Turn on: set –x
• Turn off: set +x

Keeping track of special characters and their meanings can be a hassle

• Keeping track of special characters and their meanings can be a hassle

• Exercise – run the following command

• [root@faile 6]# echo a \" is a special character
• [root@faile 6]# echo a “ is a special character
• [root@faile 6]# echo a “’’” is a special character
• [root@faile 6]# echo showing multiple \\\\\\\\\ is tricky
• [root@faile 6]# echo then again maybe not '\\\\'
• [root@faile 6]# echo then again maybe not "\\\\"

Any command line consists of

• Any command line consists of

• Command and any number of arguments
• All separated by white space
• When the shell parses the command line it removes the white space

Control operators are a collection of characters (; & && || ) which change the operation of the command line.

• Control operators are a collection of characters (; & && || ) which change the operation of the command line.

• The ; character can be used to place more than one command on a single line.
• Example: ls ; echo now in root directory ; cd / ; ls
• Place the command in the background: &
• Example: firefox &

&& characters tell the shell to execute the 2nd command only if the first command has an exit status 0 (and)

• && characters tell the shell to execute the 2nd command only if the first command has an exit status 0 (and)
• Example:
• grep root /etc/passwd && echo it works
• grep not_there /etc/passwd && echo it works
• || characters tell shell to execute the 2nd command only if the first command has an exit status of not 0
• Example:
• grep root /etc/passwd || echo it does not work
• grep not_there /etc/passwd || echo it does not work
• Example: On Linux, /etc/init.d/sshd has a line
• $SSHD $OPTIONS && success || failure

How to type a long command

• How to type a long command

• Continue a Command using \ at the end of line
• NEWLINE is the end of a command
• \ escape the meaning of the next character

• Group Commands ()

• User parentheses to group commands
• The shell create a subshell for each group
• Each subshell has its own env
• Ex:
• #pwd; (cd /); pwd
• #pwd; cd /; pwd
• (cd $1; tar –cf - . ) | (cd $2 ; tar –xvf - )

A way to send input and output (I/O) to different places.

• A way to send input and output (I/O) to different places.

• Used to

• Read data from a file
• EX: cat < /etc/passwd
• Write data to a file
• EX: ls –lR > all.my.files
• Join Unix commands together
• EX: grep “/bin/sh” /etc/passwd | cut –f1 –d:

Every process has a table of file descriptors (one for each file).

• Every process has a table of file descriptors (one for each file).

• Every process has three standard file descriptors which are used by default.

Log in to wopr.csl.mtu.edu

• Log in to wopr.csl.mtu.edu

• Start a bash
• %bash
• Create a work dir and change to it
• Generate the “ls –R /etc” output to etc.files
• $ls –lR /etc > etc.files
• Save the error output from the above command to file errors also.
• $ls –lR /etc > etc.files 2> errors
• Put output and error output in the samefile
• $ls –R / > output.and.errors 2>& 1

• Here documents – commonly used in script

• $cat << the_end
• >This is a test of output of ls
• >`ls`
• >the_end

• If no input file is provided, some commands will wait for the input from keyboard, until ctrl-D (EOF) is pressed.

• cat, awk, grep, mailx, …

UNIX treats everything (devices, processes, the kernel) as files

• UNIX treats everything (devices, processes, the kernel) as files

• Sending information to these files sends the information to the device.

• Example:
• Send the contents of a file to terminal 1
• $ls > /dev/tty4
• Send a file to the bit bucket in the sky
• $cat file > /dev/null
• $ls –lR /etc > etc.files 2> /dev/null

#

• #

• #15 2 * * * /usr/sbin/quot /fortran > /fortran/disk.usage 2>&1

• #

• # check quotas once every hour

• # quotacheck is run on demand by e-mailing quotacheck@cslserver from login

• # scripts now so only run quotacheck once a day

• #

• #30 3 * * * /usr/sbin/quotacheck /fortran > /dev/null 2>&1

• #

• # delete old apache logs and compress new ones

• #

• 0 1 * * * /usr/bin/find /var/adm-httpd -type f -mtime +14 -name "*log*gz" -exec /usr/bin/rm {} \; > /dev/null 2>&1

• 5 1 * * * /usr/bin/find /var/adm-httpd -type f -mtime +1 -name "*log*" \! -name "*gz" -exec /usr/bin/gzip {} \; > /dev/null 2>&1

• #

• # check number of files in /var/spool/mqueue for CEC nagios monitor

• #

• 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58 * * * * /usr/bin/ls /var/spool/mqueue | /usr/bin/wc -l | /usr/bin/awk '{print $1}' - >/usr/host/nagios/mqf 2>/dev/null

A shell defines variables to control the behavior of your UNIX session

• A shell defines variables to control the behavior of your UNIX session

• Pre-defined variables
• Example:
• User defined variables
• Name - must start with a letter or underscore character
• followed by 0 or more letters, numbers or underscores

Define a variable or Assign a new value

• Define a variable or Assign a new value

• variable_name=[value]
• no space around =
• Bash: % name=“david jones”
• Tcsh: % set name=“values”

• Using the value

• %echo ${name}
• %echo $name

• Check what variables are set

• The set command (actually a built-in shell command) will show all the current shell variables and their values.

• To Unset a variable

• %unset variablename

By default, user defined variables are local.

• By default, user defined variables are local.

• Environment variables are passed to sub-processes. Local variables are not.

• To promote a local variables to environment variables

• Bash:
• $CLASSPATH=$NETSCAPE_HOME/classes
• $export CLASSPATH
• Tcsh:
• $setenv CLASSPATH $NETSCAPE_HOME/classes

• To check environment variables

• Tcsh: setenv
• Bash: export –n

Environment variables should be set at login time in startup files.

• Environment variables should be set at login time in startup files.

• Bourne shell

• System wide /etc/profile
• .profile in your home dir
• Example: /etc/profile in undergrad lab.

Make the change take effect

• Make the change take effect

• Log out and log back in
• Running .profile with . (DOT) built-in
• Ex: $. .profile
• Command runs the script as the part of the current process
• Changes will affect the login shell.
• If without the first ., the new variable would be in effect only in the subshell running the script.
• You can change the values of the environment variables just like you would to any other variable variable_name=[value]
• What will happen if you do:
• %PATH=
• %ls

Change your prompt for the bash

• Change your prompt for the bash

• Start a bash %bash
• Find the value for the environment variable $PS1
• ‘%man bash ‘ and looking for PS1
• Set PS1 to be like [username@hostname:pwd%]

• About $PATH

• Login to icu0.csl.mtu.edu as youself
• What is the value of PATH?
• Which startup files set its value?
• $man tcsh – looking for startup and shutdown section
• /etc/csh.cshrc
• /etc/csh.login
• ~/.tcshrc
• ~/.cshrc
• ~/.history
• ~/.login

Example

• Example

• dinbig:~$ myName=
• dinbig:~$ echo my name is ${myName:-"NO NAME"}
• my name is NO NAME
• dinbig:~$ echo my name is $myName
• my name is
• dinbig:~$ echo my name is ${myName:="NO NAME"}
• my name is NO NAME
• dinbig:~$ echo my name is $myName
• my name is NO NAME

dinbig:~$ herName=

• dinbig:~$ herName=

• dinbig:~$ echo her name is ${herName:?"she hasn't got a name"}

• bash: herName: she hasn't got a name

• dinbig:~$ echo her name is ${herName:?}

• bash: herName: parameter null or not set

Command substitution allows you to insert the output of one command into the command line of another.

• Command substitution allows you to insert the output of one command into the command line of another.

• command substitution uses back quote character `.
• BE CAREFUL: Many people confuse the back quote with the single quote character. They are different.

• Exercise:

• %echo “there are `wc –l /etc/passwd` lines in the passwd file.
• Create a file called hostname.log
• Create a file named yyyy_mm_dd.log

filename substitution or globbing

• filename substitution or globbing

• A way of specifying a number of filenames with a small number of characters.
• %ls -l *.doc
• Some special characters are used to specify which filenames to match.
• *
• Any string including the null string
• ?
• any single character
• [ ]
• Matches any one character within the [ ] Two characters separated by a - indicates a range If ! or ^ are the first character then any character NOT between [ ] are matched.

The ~ (tilde) character expands to either user's home directory, current working directory or previous working directory depending on following character.

• The ~ (tilde) character expands to either user's home directory, current working directory or previous working directory depending on following character.

• Examples
• echo my home directory is ~
• echo working directory is ~+
• echo previous working directory is ~-

bash supports a number of other expansions which can be useful

• bash supports a number of other expansions which can be useful

• brace expansion
• Similar to pathname expansion
• Example
• ls -ld /etc/rc.d/{init,rc1,rc2}.d
• arithmetic expansion
• Evaluation of arithmetic expressions.
• Examples
• echo $[(5+4)-3*2]

Why shell script?

• Why shell script?

• Simply and quickly initiate a complex series of tasks or a repetitive procedure.

• Creating a simple shell script

• A shell script is a file that contains commands that the shell can execute.
• Any commands you enter in response to a shell prompt.
• A utility
• A compiled program
• Another shell script
• Control flow commands

Make the file executable

• Make the file executable

• When you create a shell script using a editor, does it have execute permission typically?
• Example
• [ruihong@dafinn ~/cs3451]$ ./test
• ./test: Permission denied.
• [ruihong@dafinn ~/cs3451]$ ls -l test
• -rw------- 1 ruihong csdept 22 Jan 28 09:33 test
• [ruihong@dafinn ~/cs3451]$ chmod +x test
• [ruihong@dafinn ~/cs3451]$ ./test
• this is a test

Enter the script filename as the command

• Enter the script filename as the command

• The shell forks a process
• Which creates a duplicate of the shell process (subshell)
• The new process attempt to exec the command
• If the command is a executable program
• Exec succeeds
• System overlays the newly created subshell with the executables programs
• If the the command is a shell script
• Exec failed
• The command is assumed to be a shell script
• The subshell runs the commands in the shell one after another

Pass the shell script as a parameter to a shell program

• Pass the shell script as a parameter to a shell program

• Run a shell script which does not have execution permission
• Ex: $sh filename
• Run the script with different shell other than your interactive shell
• Ex: $ksh filename

Put special characters on the first line of a shell script

• Put special characters on the first line of a shell script

• To tell OS checks what kind of file it is before attempting to exec it
• To tell which utility to use (sh, csh, tcsh, …)
• The firsts two character of a script are #!
• Then followed by the absolute pathname of the program that should execute the script
• sh-2.05b$ more /etc/init.d/sshd
• #!/bin/bash
• #
• # Init file for OpenSSH server daemon
• #

Comments make shell scripts easier to read and maintain

• Comments make shell scripts easier to read and maintain

• Pound sign (#) start a comment line until the end of that line, except

• #! In the first line.
• Or inside quotes

A shell parameter is associated with a value that is accessible to the user.

• A shell parameter is associated with a value that is accessible to the user.

• Shell variables
• Keyword shell variables
• Has special meaning to the shell
• Being created and initialized by the startup file
• User created variables (create and assign value)
• Local variables
• Environment variables
• Positional parameters
• Allow you to access command line arguments
• Special parameters
• Useful others

The command name and arguments

• The command name and arguments

• Why are they called positional parameters?
• Because they can by referenced by their position on the command line
• $0 : Name of the calling program
• $1 - $9 : Command-line Arguments
• The first argument is represented by $1
• The second argument is represented by $2

Example:

• Example:

How to access the parameter after $9?

• How to access the parameter after $9?

• shift
• Built-in command shift promotes each of the command-line arguments.
• The first argument ( which was $1) is discarded
• The second argument ( which was $2) becomes $1
• The third becomes the second
• And so on

• Repeatedly using shift is a convenient way to loop over all the command-line arguments

Example:

• Example:

Initialize arguments outside of the command line

• Initialize arguments outside of the command line

• set (sh/ksh only)
• Set the positional parameters starting from $1, …

• Use quote for variable reference

• Example:
• what’s the difference if $1 is null
• $ display_4args $1 a b c d
• $ display_4args “$1” a b c d
• What will happen if a is null
• if [ $a = 3 ]; then
• echo a is 3
• fi

Special parameters give you some useful values

• Special parameters give you some useful values

• Number of the Command-line arguments
• Return status of the execution of shell commands
• Etc..

• Special parameters’ value can not be changed directly, like positional parameters

Value of Command-line arguments: $* and $@

• Value of Command-line arguments: $* and $@

• $* and $@ represent all the command_line arguments ( not just the first nine)
• “$*” : treat the entire list of arguments as a single argument
• “$@” : produce a list of separate arguments.

sh-2.05b$ more for_test

• sh-2.05b$ more for_test

• echo "using \$@ "

• for arg in "$@"

• do

• echo "$arg"

• done

• echo "using \$* "

• for arg in "$*"

• do

• echo "$arg"

• Done

• sh-2.05b$ ./for_test 1 2 3

• using $@

• 1

• 2

• 3

• using $*

• 1 2 3

The number of arguments: $#

• The number of arguments: $#

• The current shell’s PID number: $$

• Ex:
• sh-2.05b$ echo $$
• 11896

• The PID number of last process that you ran in the background: $!

• Ex:
• sh-2.05b$ sleep 1000 &
• [1] 11962
• sh-2.05b$ echo $!
• 11962

Exit status: $?

• Exit status: $?

• When a process stops executing for any reason, it returns an exit status to its parent process.
• By convention,
• Nonzero represents a false value that the command failed.
• A zero value is true and means that the command was successful
• You can specify the exit status that a shell script returns by using the exit built-in followed by a number
• Otherwise, the exit status of the script is the exit status of the last command the script ran.

A shell is both a command interpreter and a programming language

• A shell is both a command interpreter and a programming language

• Local variables and environment variables

• Command line expansion

• Parameter expansion
• variable expansion
• Command substitution
• pathname expansion

• Special parameters