Task Automation with Scripts

Overview

Teaching: 15 min
Exercises: 10 min

Questions

• How can we repeat the same or similar set of commands over and over?

Objectives

• Writing a simple script by chaining one or more commands

• Learning some programming constructs such as If and for

What is a Script?

Shell scripting is the process of combining a sequence of commands in a single file to be executed in the prescribed order. It gives the benefit of not having to retype commands again and again, but having to execute them just by calling the file in wich they reside. The file containing the commands is called a script. A script can be defined as a program (sequence of instructions) that is executed (interpreted) by and interpreter. This is different for compiled programs. Compiled languages are transformed into a more machine readable code before being executed. Interpreted languages are scripts that are directly executed, line by line. In this section, you will learn some basic concepts of shell scripting.

What can you put in a script?

A shell script is a sequence of commands. Think of all the commands you have learned so far, you can write a script containing all of them and run them one after the other by invoking the script file. This sounds great, but there is so much more. Shell scripting is just like any other programming language. You can use constructs like conditional statement if, loops for and while, and case statements. You can document your script using comments just as in other programming languages. To add a comment in on the terminal or in your script, just start the line with # everything that follows to the end of the line will be ignored. Some exceptions apply. In the next section you will see that there is a special sequence starting with # and is not ignored. Shell scripts also can have variables of different types, used to house values such as integers and strings.

Your first script, Hello world!

All you need for your first script is to start your favorite text editor (nano, vi, emacs…) and type in the following:

#!/bin/bash

echo "Hello world!"

In the above script, notice the first line: #!/bin/bash. This is not a comment, but an instruction to the operating system. What makes a difference between this line and a comment is the the two characters #!. This line tells the operating system which program (interpreter) to use in order to execute all that follow after. In this case, the program named bash located in the /bin directory will be used as ‘interpret’ the script. There are several other interpreters for shell scripting each with its own syntax. Examples are bash(the one we are using), Bourne shell sh, C shell csh… The line echo "Hello world!" is the first and only command to execute. The echo command takes a string argument and displays it on the following line.

How do you execute a script

To run a script, all you need is to call the script file. However, you need to make sure you have granted yourself the permission to execute it, using chmod.

$ chmod u+x hello.sh #grant user the execute permission
$ ./hello.sh

Hello world!

Shell variables

Variables are important in all programming languages. They are used in order to store values and access them. Bash variables are not typed and do not need to be declared. All you have to do is to assign a value to a named variable:

#!/bin/bash

a=1
b=t
c="string of characters"

Notice

Notice that there is no spacing on either side of the =.

To access a variable’s contain, you need to use the $ special character.

#!/bin/bash

# Assign values to variables
a=1
b=t
c="string of characters"

# Print variable values
echo $a
echo $b
echo $c

1
t
string of characters

The $ tells the interpreter to evaluate the content of the variable following it. Another use for $ is to evaluate a command. For example try the the following:

#!/bin/bash

echo ls
echo $(ls)

The line ‘echo ls’ will treat ls as a string of character to display. The second ‘echo $(ls)’ will evaluate the expression within the parenthesis (here ls) and then display its content.

Special variables

• $0 - The name of the Bash script.
• $1 - $9 - The first 9 arguments to the Bash script.
• $# - How many arguments were passed to the Bash script.
• $@ - All the arguments supplied to the Bash script.
• $? - The exit status of the most recently run process.
• $$ - The process ID of the current script.
• $USER - The username of the user running the script.
• $HOSTNAME - The hostname of the machine the script is running on.
• $SECONDS - The number of seconds since the script was started.
• $RANDOM - Returns a different random number each time is it referred to.
• $LINENO - Returns the current line number in the Bash script.

Shell arguments

Shell scripts take arguments just as any other command. Arguments are a way to provide information which is not known at the time when the script is being written, but available right before execution. It could be a file to process, a list of folders to work with… It could pretty much be anything you want. To feed an argument to a shell script you just add it after the script’s name at run time:

$ ./script_name.sh [ARG1] [ARG2] [ARG3] [...]

The order in which your arguments appear is important as this is how you access them from within your script. Try the following:

#!/bin/bash

echo "Argument 0 is :"
echo $0

echo "Argument 1 is :"
echo $1

echo "Argument 2 is :"
echo $2

Save the the above script as script_args.sh then add executable permission and then run the script with different arguments:

$ chmod u+x script_args.sh
$ ./script_args.sh arg1 arg2

Argument 0 is :
./script_args.sh
Argument 1 is :
arg1
Argument 2 is :
arg2

You can reference arguments from 0 to 9 using $0 through $9. Beyond nine arguments you have to use curly brackets in order to directly reference an argument by its order. For example if you want to access arguments 12 and 15 from your argument list, your script will look like this:

#!/bin/bash

echo "Argument 12 is:"
echo ${12}

echo "Argument 15 is :"
echo ${15}

There are other ways to access arguments. You can for example iterate through the argument list (given by special variable $@) using a loop.

Test operators

Test operators are used in order to determine equality relationship between two entities (expressions). Expressions can be output from a command, variables, constants… The tests have several uses such as conditional execution with if statements and conditional loop while. There are several operators available in shell scripting. Some of them test two strings, some test numerical values. Tests in Bash shell scripting are of the following form:

[ EXPRESSION1 operator EXPRESSION2 ]

Square brackets have two main uses in Bash: the first, globbing is used as pattern representation to match string such as file names. In this case, there is no space between the brackets and their content. The second use is for tests. Here the brackets are treated as a command. You have to insert spaces between the brackets and their content to differentiate from globbing.

Test operators by operand type

• ! EXPRESSION - The EXPRESSION is false.
• -n STRING - The length of STRING is greater than zero.
• -z STRING - The lengh of STRING is zero (ie it is empty).
• STRING1 == STRING2 - STRING1 is equal to STRING2.
• STRING1 != STRING2 - STRING1 is not equal to STRING2.
• INTEGER1 -eq INTEGER2 - INTEGER1 is numerically equal to INTEGER2.
• INTEGER1 -gt INTEGER2 - INTEGER1 is numerically greater than INTEGER2.
• INTEGER1 -lt INTEGER2 - INTEGER1 is numerically less than INTEGER2.
• -d FILE - FILE exists and is a directory.
• -e FILE - FILE exists.
• -r FILE - FILE exists and the read permission is granted. *-s FILE - FILE exists and it’s size is greater than zero (ie. it is not empty).
• -w FILE - FILE exists and the write permission is granted. *-x FILE - FILE exists and the execute permission is granted.

The if statement

The if statement is used to execute a command under conditional terms. It uses a test to see if an expression fulfils a certain requirement and if it does then execute a given command. The if statement allows for an alternative command to be executed in case the condition is not fulfilled as provided by the statement.

The general form of the if statement is as follows:

#!/bin/bash

if conditional_test;
then
    command_1;
    command_2;
       ...
    command_n;
fi

In the above listing, there is no alternative in case the conditional test fails. To have an alternative execution in case the test fails, you need to add an else clause.

#!/bin/bash

if conditional_test;
then
    command_1;
    command_2;
       ...
    command_n;
else
    alt_command_1;
    alt_command_2;
       ...
    alt_command_m;
fi

A practical example to the if then else follows:

#!/bin/bash

if [ "a" = "b" ];
then
    echo "a = b";
else
    echo "a != b";
fi

Upon running this script you should get output:

a != b

The for Statement

The for statement is a loop execution with iteration through a list. The statement iterates through the list element by element and for each execute the provided set of commands:

#!/bin/bash

for i in list;
do
   command1;
   command2;
       ...
   command3;
done

As example, let say you want to print numbers from 0 to 10, one number per line:

#!/bin/bash

for i in 0 1 2 3 4 5 6 7 8 9 10;
do
    echo $i;
done

Yields:

0
1
2
3
4
5
6
7
8
9
10

Remember the shell arguments from before? Here is how you could access each argument and do something with them:

#!/bin/bash

i=0

for arg in $@;
do
   let "i++";
   echo "Argument " $i "is : " $arg;
done

If you run this script with arguments ‘one two three orange banana mango’ you will get:

Argument  1 is :  one
Argument  2 is :  two
Argument  3 is :  three
Argument  4 is :  orange
Argument  5 is :  banana
Argument  6 is :  mango

Key Points

• A script is a text file containing a sequence of commands

• If statements are used to execute commands based on given conditions

• The for statement takes a list and run commands for each of the elements in the list by iterating through that list