Shell scripting involves chaining several UNIX commands together to accomplish a task. For example, you might run the 'date' command and then use today's date as part of a file name. I'll show you how to do this below.
Some of the tools of the trade are variables, backquotes and pipes. First we'll study these topics and also quickly review a few other UNIX topics.
Variables
Topics covered: storing strings in variables
Utilities covered: echo, expr
After we log in to remote1.unl.csi.cuny.edu, a Bourne shell is started up automatically. (If shell is not started, you can use command /bin/sh to start it.)
A variable stores a string (try running these commands in a Bourne shell)
(1) Step 1: type the following command:
name="John"
echo $name
What do you get?
Now type the following command:
echo name
What do you get?
Now type the following command:
name="John Doe"
echo $name
What do you get?
Now type the following command:
name=John Doe
What do you get?
The quotes are required in the example above because the string contains a special character (the space)
(2) Step 2:
A variable may store a number
num=137
The shell stores this as a string even though it appears to be a number
A few UNIX utilities will convert this string into a number to perform arithmetic
Now type the following command:
expr $num + 3 (make sure you have space before + and 3)
What do you get?
Try defining num as '7m8' and try the expr command again.What happens when num is not a valid number?
I/O Redirection
Topics covered: specifying the input or capturing the output of a command in a file
Utilities covered: wc, sort
(3) Step 3:
The wc command counts the number of lines, words, and characters in a file
wc /etc/passwd
wc -l /etc/passwd
What do you get from these two commands?
You can save the output of wc (or any other command) with output redirection
wc /etc/passwd > wc.file
then type the command: cat wc.file
What is the result?
(4) Step 4:
You can specify the input with input redirection
wc < /etc/passwd
What is the output?
Many UNIX commands allow you to specify the input file by name or by input redirection
sort /etc/passwd
sort < /etc/passwd
You can also append lines to the end of an existing file with output redirection
wc -l /etc/passwd >> wc.file
then type the command cat wc.file, what do you get?
Backquotes
Topics covered: capturing output of a command in a variable
Utilities covered: date
The backquote character looks like the single quote or apostrophe, but slants the other way
It is used to capture the output of a UNIX utility
A command in backquotes is executed and then replaced by the output of the command
(5) Step 5:
Execute these commands
date
save_date=`date`
echo The date is $save_date
What is the output?
Notice how echo prints the output of 'date', and gives the time when you defined the save_date variable
(6) Step 6:
Store the following in a file named backquotes.sh and execute it (right click and save in a file)
#!/bin/sh
# Illustrates using backquotes
# Output of 'date' stored in a variable
Today="`date`"
echo Today is $Today
Execute the script with
bash backquotes.sh
What is the output?
The example above shows you how you can write commands into a file and execute the file with a Bourne Again shell
Backquotes are very useful, but be aware that they slow down a script if you use them hundreds of times
(7) Step 7:
You can save the output of any command with backquotes. Try this:
LS=`ls -l`
echo $LS
Pipes
Topics covered: using UNIX pipes
Utilities covered: sort, cat, head
Pipes are used for post-processing data
One UNIX command output is introduced as input to another command
(8) Step 8: type the following command:
sort /etc/passwd | head -5
What is the output?
Notice that this pipe can be simplified
cat /etc/passwd | head -5
What is the output for this?
You could accomplish the same thing more efficiently with either of the two commands:
head -5 /etc/passwd
head -5 < /etc/passwd
(9) Step 9:
The command displays all the files in the current directory sorted by file size
ls -al | sort -n -r +4
What is the output? What are the outputs sorting according to? Which column is this?
The command ls -al writes the file size in the fifth column, which is why we skip the first four columns using +4.
The options -n and -r request a numeric sort (which is different than the normal alphabetic sort) in reverse order
awk
Topics covered: processing columnar data
Utilities covered: awk
The awk utility is used for processing columns of data
(10) Step 10:
The following example shows how to extract column 5 (the file size) from the output of ls -l
ls -l | awk '{print $5}'
What is the output?
Cut and paste this line into a Bourne shell and you should see a column of file sizes, one per file in your current directory.
(11) Step 11:
The following example shows how to sum the file sizes and print the result at the end of the awk run
ls -al | awk '{sum = sum + $5} END {print sum}'
In this example you should see printed just one number, which is the sum of the file sizes in the current directory.
Shell Scripts
Topics covered: storing commands in a file and executing the file
Utilities covered: date, cal, last (shows who has logged in recently)
(12) Step 12:
Store the following in a file named simple.sh and execute it
#!/bin/sh
# Show some useful info at the start of the day
date
echo Good morning $USER
cal
last | head -6
What is the result after you execute it?(Shows current date, calendar, and a six of previous logins Notice that the commands themselves are not displayed, only the results )
last command - display login and logout information about users and
terminals
(13) Step 13:
To display the commands verbatim as they run, execute with
sh -v simple.sh
Another way to display the commands as they run is with -x
sh -x simple.sh
What is the difference between -v and -x? Notice that with -v you see '$USER' but with -x you see your login name
Run the command 'echo $USER' at your terminal prompt and see that the variable $USER stores your login name
With -v or -x (or both) you can easily relate any error message that may appear to the command that generated it
(14) Step 14:
When an error occurs in a script, the script continues executing at the next command
Verify this by changing 'cal' to 'caal' to cause an error, and then run the script again
Run the 'caal' script with 'sh -v simple.sh' and with 'sh -x simple.sh' and verify the error message comes from cal
Other standard variable names include: $HOME, $PATH, $PRINTER. Use echo to examine the values of these variables
Storing File Names in Variables
Topics covered: variables store strings such as file names, more on creating and using variables
Utilities covered: echo, ls, wc
A variable is a name that stores a string
It's often convenient to store a filename in a variable
Step 1: Store the following in a file named variables.sh and execute it
#!/bin/sh
# An example with variables
filename="/etc/passwd"
echo "Check the permissions on $filename"
ls -l $filename
echo "Find out how many accounts there are on this system"
wc -l $filename
What is the output?
Now change the value of $filename to /etc/link, the change is automatically propagated throughout the entire script
What is the output?
Scripting With sed
Topics covered: global search and replace, input and output redirection
Utilities covered: sed
Step 2: Here's how you can use sed to modify the contents of a variable:
echo "Hello Jim" | sed -e 's/Hello/Bye/'
Step 3: create a file named nlanr.txt with the following contents:
The National Laboratory for Applied Network Research (NLANR) is a
collaboration among NSF-supported supercomputer sites. NLANR was
created in 1995 to provide technical and engineering support and
overall coordination of the vBNS connections at the five NSF-supported
supercomputer centers. The vBNS has evolved to become a "leading edge
but stable" platform to enable the development and use of high
performance applications by the broader academic research community.
Consequently, NLANR's focus has expanded.
notice how the word 'vBNS' appears in it several times
Step 4: Change 'vBNS' to 'NETWORK' with the following command:
sed -e 's/vBNS/NETWORK/g' < nlanr.txt
open the file nlanr.txt, what do you find?
Step 5: open file nlanr.txt, and change NETWORK back to vBNS, then type the following command:
sed -e 's/vBNS/NETWORK/g' < nlanr.txt > nlanr.new
open the file nlanr.new, what do you find?
Sed can be used for many complex editing tasks, we have only scratched the surface here
Performing Arithmetic
Topics covered: integer arithmetic, preceding '*' with backslash to avoid file name wildcard expansion
Utilities covered: expr
Arithmetic is done with expr
Step 6: type the following command:
expr 5 + 7
expr 5 \* 7
Backslash required in front of '*' since it is a filename wildcard and would be translated by the shell into a list of file names
You can save arithmetic result in a variable
Step 7: Store the following in a file named arith.sh and execute it
#!/bin/sh
# Perform some arithmetic
x=24
y=4
Result=`expr $x \* $y`
echo "$x times $y is $Result"
What is the output?
Translating Characters
Topics covered: converting one character to another, translating and saving string stored in a variable
Utilities covered: tr
Step 8: create a file with the name of sdsc.txt with the following contents:
San Diego Supercomputer Center
A National Laboratory for Computational Science and Engineering
The San Diego Supercomputer Center (SDSC) -- a campus research unit of
the University of California, San Diego -- is the focus of
computational activities within NPACI. In operation since 1986, SDSC
has provided the national research community with access to the
highest performance computers available; conducted applications and
technology research/development projects; trained researchers,
educators, and students; and facilitated technology transfer to
industry and government. Within NPACI, SDSC will broaden its focus to
include partnership with 36 other academic and research institutions.
Step 9: type the following command(The utility tr translates characters):
tr 'a' 'Z' < sdsc.txt
This example shows how to translate the contents of a variable and display the result on the screen with tr
Step 10: Store the following in a file named tr1.sh and execute it
#!/bin/sh
# Translate the contents of a variable
Cat_name="Piewacket"
echo $Cat_name | tr 'a' 'i'
what is the output?
This example shows how to change the contents of a variable
Step 11: Store the following in a file named tr2.sh and execute it
#!/bin/sh
# Illustrates how to change the contents of a variable with tr
Cat_name="Piewacket"
echo "Cat_name is $Cat_name"
Cat_name=`echo $Cat_name | tr 'a' 'i'`
echo "Cat_name has changed to $Cat_name"
You can also specify ranges of characters.
Step 12: Type the following commands and see what happened?(This example converts upper case to lower case)
tr 'A-Z' 'a-z' < file
Now you can change the value of the variable and your script has access to the new value
Processing Multiple Files
Topics covered: executing a sequence of commands on each of several files with for loops
Utilities covered: no new utilities
Step 13: Store the following in a file named loop1.sh and execute it
#!/bin/sh
# Execute ls and wc on each of several files
# File names listed explicitly
for filename in simple.sh variables.sh loop1.sh
do
echo "Variable filename is set to $filename..."
ls -l $filename
wc -l $filename
done
This executes the three commands echo, ls and wc for each of the three file names
You should see three lines of output for each file name
filename is a variable, set by "for" statement and referenced as $filename
Now we know how to execute a series of commands on each of several files
Using File Name Wildcards in For Loops
Topics covered: looping over files specified with wildcards
Utilities covered: no new utilities
Step 14: Store the following in a file named loop2.sh and execute it
#!/bin/sh
# Execute ls and wc on each of several files
# File names listed using file name wildcards
for filename in *.sh
do
echo "Variable filename is set to $filename..."
ls -l $filename
wc -l $filename
done
You should see three lines of output for each file name ending in '.sh'
The file name wildcard pattern *.sh gets replaced by the list of filenames that exist in the current directory
Step 15: For another example with filename wildcards try this command
echo *.sh
Search and Replace in Multiple Files
Topics covered: combining for loops with utilities for global search and replace in several files
Utilities covered: mv
Step 16: type the following command (Sed performs global search and replace on a single file):
sed -e 's/application/APPLICATION/g' sdsc.txt > sdsc.txt.new
open file sdsc.txt.new and see what are changed?
The original file sdsc.txt is unchanged
How can we arrange to have the original file over-written by the new version?
Step 17: Store the following in a file named s-and-r.sh and execute it
#!/bin/sh
# Perform a global search and replace on each of several files
# File names listed explicitly
for text_file in sdsc.txt nlanr.txt
do
echo "Editing file $text_file"
sed -e 's/application/APPLICATION/g' $text_file > temp
mv -f temp $text_file
done
First, sed saves new version in file 'temp'
Then, use mv to overwrite original file with new version
Accessing Command-line Arguments
Topics covered: accessing command-line arguments
Utilities covered: no new utilities
Step 18: Store the following in a file named args1.sh
#!/bin/sh
# Illustrates using command-line arguments
# Execute with
# sh args1.sh On the Waterfront
echo "First command-line argument is: $1"
echo "Third argument is: $3"
echo "Number of arguments is: $#"
echo "The entire list of arguments is: $*"
Execute the script with
sh args1.sh -x On the Waterfront
Words after the script name are command-line arguments
Arguments are usually options like -l or file names
Looping Over the Command-line Arguments
Topics covered: using command-line arguments in a for loop
Utilities covered: no new utilities
Step 19: Store the following in a file named args2.sh and execute it
#!/bin/sh
# Loop over the command-line arguments
# Execute with
# sh args2.sh simple.sh variables.sh
for filename in "$@"
do
echo "Examining file $filename"
wc -l $filename
done
This script runs properly with any number of arguments, including zero
The shorter form of the for statement shown below does exactly the same thing
for filename
do
...
Don't use
for filename in $*
Fails if any arguments include spaces
Also, don't forget the double quotes around $@
The read Command
Topics covered: reading a line from the standard input
Utilities covered: no new utilities
stdin is the keyboard unless input redirection used
Step 20: type the following command(Read one line from stdin, store line in a variable):
read variable_name
Step 20: Store the following in a file named read.sh and execute it (This file is to ask the user if he wants to exit the script):
#!/bin/sh
# Shows how to read a line from stdin
echo "Would you like to exit this script now?"
read answer
if [ "$answer" = y ]
then
echo "Exiting..."
exit 0
fi
Dostları ilə paylaş: |