Showing posts with label Shell Scripting. Show all posts
Showing posts with label Shell Scripting. Show all posts

Tuesday, 20 October 2020

Linux shell script - while loop and sleep example

Linux Shell Script, LPI Exam Prep, LPI Learning, LPI Certification, LPI Prep

Linux shell script FAQ: Can you share a Linux shell script while loop example? While you’re at it, can you show how to use the sleep command in the shell script while loop?

Sure. As a little background, I’ve written a program I call an Email Agent that periodically scans my email inbox, and does a lot of things to the inbox, including deleting the over 6,000 spams that I receive in a typical day. A recent problem with the Agent is that it runs too fast, apparently overwhelming the sendmail process on the machine that it runs on.

To help keep sendmail alive, I slow my program down in two ways. First, I try to get it to go through messages more slowly. Because the program is written in Java, I accomplish that with the Thread.sleep call in my Java code. Second, I pause the program by calling the Unix/Linux sleep command from my Bourne shell script.

Sleeping in a shell script while loop

As an example of both a while loop and sleep command, here is how my Email Agent program is now run from inside a Bourne shell script:

i=1

while [ "$i" -ne 0 ]

do

  i=./runEmailAgent

  sleep 10

done

Linux Shell Script, LPI Exam Prep, LPI Learning, LPI Certification, LPI Prep
Basically what happens is that my Email Agent program is called by my shell script, and then the Email Agent program returns a numeric value when it is finished running. A value of zero means it is finished doing all it can do, while any other number means it still has work to do, but stopped itself to give sendmail a break. After that, I give sendmail a second break by calling the Linux sleep command. I tell the sleep command to take a break for 10 seconds. When ten seconds has come and gone, the test in the while block is run again.

Linux shell script while loop and sleep example

If you ever wanted to write a while loop in the Bourne shell, I hope this serves as a simple example. Please note that it is very easy to create an infinite loop here, so be careful that the program you call will actually return a zero value at some time (assuming you decide to implement the loop exactly as shown above).

Wednesday, 5 December 2018

How to define and use functions in Linux Shell Script

Linux Tutorial and Material, Linux Study Materials, Linux Guides

In this article we’ll discuss more about functions and recipes. For demonstration purpose I’ll be using Bourne Again SHell(Bash) on Ubuntu machine.

Calling function


In Shell calling function is exactly same as calling any other command. For instance, if your function name is my_func then it can be execute as follows:

$ my_func

If any function accepts arguments then those can be provided from command line as follows:

$ my_func arg1 arg2 arg3

Defining function


We can use below syntax to define function:

 function function_name {
            Body of function
 }

Body of function can contain any valid command, loop constrain, other function or script. Now let us create simple function which displays message on screen.

 function print_msg {
       echo "Hello, World"
 }

Now let us execute this function:

 $ print_msg
 Hello, World

As expected, this function displays message on screen.

In above example we have created function directly on terminal. We can store this function in file as well. Below example demonstrates this.

 #! /bin/bash
 function print_msg {
       echo "Hello, World"
 }
 print_msg

We have defined this function inside function.sh file. Now let us execute this script:

 $ chmod +x function.sh
 $ ./function.sh
 Hello, World

If you observe, above output is exactly identical to previous one.

More about functions


In previous section we have defined very basic function. However during software development we need more advanced functions which can accept various parameters and return values. In this section we’ll discuss such functions.

Passing arguments to function

We can provide arguments to function same as other commands. We can access these arguments from function using dollar($) symbol. For instance, $1 represents first argument, $2 represents second argument and so on.

Let us modify above function to accept message as an argument. Our modified function will look like this:

 function print_msg {
       echo "Hello $1"
 }

In above function we are accessing first argument using $1. Let us execute this function:

 $ print_msg "LPICentral"

When you execute this function, it will generate following output:

 Hello LPICentral

Returning value from function

Like other programming languages, Bash provides return statement using that we can return value to the caller. Let us understand this with example:

function func_return_value {
      return 10
 }

Above function returns value 10 to its caller. Let us execute this function:

 $ func_return_value
 $ echo "Value returned by function is: $?"

When you execute above function, it will generate following output:

 Value returned by function is: 10

NOTE: In bash we have to use $? to capture return value of function

Function recipes


So far we got fair idea about bash functions. Now let us create some useful bash functions which can be used to make our lives easier.

Logger

Let us create logger function which will print date and time along with log message.

 function log_msg {
        echo "[`date '+ %F %T'` ]: $@"
 }

Let us execute this function:

 $ log_msg "This is sample log message"

When you execute this function, it will generate following output:

 [ 2018-08-16 19:56:34 ]: This is sample log message

Display system information

Let us create a function to display information about GNU/Linux system

 function system_info {
       echo "### OS information ###"
       lsb_release -a

       echo
       echo "### Processor information ###"
       processor=`grep -wc "processor" /proc/cpuinfo`
       model=`grep -w "model name" /proc/cpuinfo  | awk -F: '{print $2}'`
       echo "Processor = $processor"
       echo "Model     = $model"

       echo
       echo "### Memory information ###"
       total=`grep -w "MemTotal" /proc/meminfo | awk '{print $2}'`
       free=`grep -w "MemFree" /proc/meminfo | awk '{print $2}'`
       echo "Total memory: $total kB"
       echo "Free memory : $free kB"
 }

When you execute above function it will generate following output:

### OS information ###
No LSB modules are available.
Distributor ID:           Ubuntu
Description:   Ubuntu 18.04.1 LTS
Release:         18.04
Codename:    bionic

### Processor information ###
Processor = 1
Model     =  Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz

### Memory information ###
Total memory: 4015648 kB
Free memory : 2915428 kB

Find file or directory from current directory

Linux Tutorial and Material, Linux Study Materials, Linux Guides

Below function searches file or directory from current directory:

 function search {
      find . -name $1
 }

Let us search directory namely dir4 using below command:

 $ search dir4

When you execute above command, it will generate following output:

 ./dir1/dir2/dir3/dir4

Digital clock

Below function creates a simple digital clock on terminal

 function digital_clock {
            clear
            while [ 1 ]
            do
                  date +'%T'
                  sleep 1
                  clear
            done
 }

Creating library


Library is a collection of functions. To create library – define functions in a file and import that file in current environment.

Let us suppose we have defined all functions in utils.sh file then use below command to import functions in current environment:

$ source utils.sh

Hereafter you can execute any function from library just like any other bash command.

Friday, 2 November 2018

Execute Mysql Command in Bash / Shell Script

Q) How to connect to mysql database from a bash script in unix or linux and run sql queries?


Bash scripting helps in automating things. We can automate running sql queries by connecting to the mysql database through a shell script in unix or linux system.

Linux Tutorial and Materials, LPI Guides, LPI Learning, LPI Study Material, LPI Learning

Here we will see how to run a small sql query in mysql database through a script. The bash script code is shown below:

#!/usr/bin/bash

#Script to run automated sql queries

#Declaring mysql DB connection 

MASTER_DB_USER='username'
MASTER_DB_PASSWD='password'
MASTER_DB_PORT=3160
MASTER_DB_HOST='mysql.hostname'
MASTER_DB_NAME='mysqlDbName'

#Prepare sql query

SQL_Query='select * from tablename limit 10'

#mysql command to connect to database

MYSQL -u$MASTER_DB_USER -p$MASTER_DB_PASSWD -P$MASTER_DB_PORT -h$MASTER_DB_HOST -D$MASTER_DB_NAME <<EOF 
$SQL_Query
EOF
echo "End of script"

Here in the above script, the first part declares the mysql db variables and assigns the DB details. The second part prepares sql query. And the final part executes the mysql command.

Saturday, 20 October 2018

File Test Operators / Operations Examples in Unix / Linux Shell Script

In linux and unix operating systems every thing is a file. When you are using files in your shell or bash script, it is a good idea to do some tests on the file before using it.

LPI Certification, LPI Guides, LPI Learning, LPI Study Materials, LPI Tutorial and Material

The file tests include:

◈ Checking for existence of the file.
◈ File is readable, writeable or executable.
◈ Type of the file and so on.

The file test operators are mostly used in the if clause of the bash script. The syntax is shown below:

if [ -option filename ]
then
  do something
else
  do something
fi

The different file test operators are listed below:

◈ a : True if the file exists.
◈ b : True if the file exists and is a block special file.
◈ c : True if the file exists and is a character special file.
◈ d : True if the file exists and is a directory.
◈ e : True if the file exists.
◈ f : True if the file exists and is a regular file.
◈ g : True if the file exists and its SGID bit is set.
◈ h : True if the file exists and is a symbolic link.
◈ k : True if the file exists and its sticky bit is set.
◈ p : True if the file exists and is a named pipe (FIFO).
◈ r : True if the file exists and is readable.
◈ s : True if the file exists and has a size greater than zero.
◈ t : True if file descriptor is open and refers to a terminal.
◈ u : True if the file exists and its SUID (set user ID) bit is set.
◈ w : True if the file exists and is writable.
◈ x : True if the file exists and is executable.
◈ O : True if the file exists and is owned by the effective user ID.
◈ G : True if the file exists and is owned by the effective group ID.
◈ L : True if the file exists and is a symbolic link.
◈ N : True if the file exists and has been modified since it was last read.
◈ S : True if the file exists and is a socket.

File Test Operator Example:


The following shell script checks for the existence of a regular file:

#!/bin/bash
#assign file name to the variable
FILE="linux-server.dat"

if [ -f $FILE ]
then
  echo "$FILE exists and is a regular file"
else
  echo "Either $FILE does not exist or is not a regular file"
fi 

Thursday, 4 October 2018

Debugging Shell Scripts in Linux

Linux Tutorial and Material, Linux Certification, Linux Guide, Linux Learning

In most of the programming languages debugger tool is available for debugging. A debugger is a tool that can run a program or script that enables you to examine the internals of the script or program as it runs. In the shell scripting we don”t have any debugger tool but with the help of command line options (-n, -v and -x ) we can do the debugging.

Disabling the Shell ( -n option)


The -n option, shot for noexec ( as in no execution), tells the shell to not run the commands. Instead, the shell just checks for syntax errors. This option will not convince the shell to perform any more checks. Instead the shell just performs the normal syntax check. With -n option, the shell doesn’t execute your commands, so you have a safe way to test your scripts if they contain syntax error.

The follow example shows how to use -n option.

Let us consider a shell script with a name debug_quotes.sh

#!/bin/bash
echo "USER=$USER
echo "HOME=$HOME"
echo "OSNAME=$OSNAME"

Now run the script with -n option

$ sh -n debug_quotes
debug_quotes: 8: debug_quotes: Syntax error: Unterminated quoted string

As the above outputs shows that there is syntax error , double quotes are missing.

Displaying the Scripts Commands ( -v option )


The -v option tells the shell to run in verbose mode. In practice , this means that shell will echo each command prior to execute the command. This is very useful in that it can often help to find the errors.

Let us create a shell script with the name “listusers.sh” with below contents

lpicentral@localhost:~$ cat listusers.sh

#!/bin/bash

cut -d : -f1,5,7 /etc/passwd | grep -v sbin | grep sh | sort > /tmp/users.txt
awk -F':' ' { printf ( "%-12s %-40s\n", $1, $2 ) } ' /tmp/users.txt

#Clean up the temporary file.
/bin/rm -f /tmp/users.txt

Now execute the script with -v option

lpicentral@localhost:~$ sh -v listusers.sh

#!/bin/bash

cut -d : -f1,5,7 /etc/passwd | grep -v sbin | grep sh | sort > /tmp/users.txt
awk -F':' ' { printf ( "%-12s %-40s\n", $1, $2 ) } ' /tmp/users.txt
guest-k9ghtA Guest,,,
guest-kqEkQ8 Guest,,,
guest-llnzfx Guest,,,
pradeep pradeep,,,
mail admin Mail Admin,,,

#Clean up the temporary file.
/bin/rm -f /tmp/users.txt

lpicentral@localhost:~$

In the above output , script output gets mixed with commands of the scripts. But however , with -v option , at least you get a better view of what the shell is doing as it runs your script.

Combining the -n & -v Options


We can combine the command line options ( -n & -v ). This makes a good combination because we can check the syntax of a script while seeing the script output.

Let us consider a previously used script “debug_quotes.sh”

lpicentral@localhost:~$ sh -nv debug_quotes.sh

#!/bin/bash
#shows an error.

echo "USER=$USER
echo "HOME=$HOME"
echo "OSNAME=$OSNAME"

debug_quotes: 8: debug_quotes: Syntax error: Unterminated quoted string

lpicentral@localhost:~$

Tracing Script Execution ( -x option )


The -x option, short for xtrace or execution trace, tells the shell to echo each command after performing the substitution steps. Thus , we can see the values of variables and commands. Often, this option alone will help to diagnose a problem.

In most cases, the -x option provides the most useful information about a script, but it can lead to a lot of output. The following example show this option in action.

lpicentral@localhost:~$ sh -x listusers.sh

+ cut -d :+ -f1,5,7 /etc/passwd
grep -v sbin
+ sort
+ grep sh
+ awk -F: { printf ( "%-12s %-40s\n", $1, $2 ) } /tmp/users.txt
guest-k9ghtA Guest,,,
guest-kqEkQ8 Guest,,,
guest-llnzfx Guest,,,
pradeep pradeep,,,
mail admin Mail Admin,,,
+ /bin/rm -f /tmp/users.txt

lpicentral@localhost:~$

In the above output, shell inserted a + sign in front of the commands.

Sunday, 30 September 2018

How to use variables in shell Scripting

In every programming language variables plays an important role , in Linux shell scripting we are using two types of variables : System Defined Variables & User Defined Variables.

A variable in a shell script is a means of referencing a numeric or character value. And unlike formal programming languages, a shell script doesn’t require you to declare a type for your variables

Shell Scripting, LPI Guides, LPI Learning, LPI Tutorial and Materials, LPI Certification

In this article we will discuss variables, its types and how to set & use variables in shell scripting.

System Defined Variables :


These are the variables which are created and maintained by Operating System(Linux) itself. Generally these variables are defined in CAPITAL LETTERS. We can see these variables by using the command “$ set“. Some of the system defined variables are given below :

System Defined Variables Meaning
BASH=/bin/bash  Shell Name 
BASH_VERSION=4.1.2(1)  Bash Version
COLUMNS=80   No. of columns for our screen 
HOME=/home/lpicentral   Home Directory of the User 
LINES=25   No. of columns for our screen 
LOGNAME=LPICentral LPICentral Our logging name 
OSTYPE=Linux   OS type 
PATH=/usr/bin:/sbin:/bin:/usr/sbin   Path Settings 
PS1=[\u@\h \W]\$   Prompt Settings 
PWD=/home/lpicentral Current Working Directory 
SHELL=/bin/bash   Shell Name
USERNAME=lpicentral User name who is currently login to system 

To Print the value of above variables, use echo command as shown below :

# echo $HOME
# echo $USERNAME

We can tap into these environment variables from within your scripts by using the environment variable’s name preceded by a dollar sign. This is demonstrated in the following script:

$ cat myscript

#!/bin/bash
# display user information from the system.
echo “User info for userid: $USER”
echo UID: $UID
echo HOME: $HOME

Notice that the environment variables in the echo commands are replaced by their current values when the script is run. Also notice that we were able to place the $USER system variable within the double quotation marks in the first string, and the shell script was still able to figure out what we meant. There is a drawback to using this method, however. Look at what happens in this example:

$ echo “The cost of the item is $15”
The cost of the item is 5

That is obviously not what was intended. Whenever the script sees a dollar sign within quotes, it assumes you’re referencing a variable. In this example the script attempted to display the variable $1 (which was not defined), and then the number 5. To display an actual dollar sign, you must precede it with a backslash character:

$ echo “The cost of the item is \$15”
The cost of the item is $15

That’s better. The backslash allowed the shell script to interpret the dollar sign as an actual dollar sign, and not a variable.

User Defined Variables:


These variables are defined by users. A shell script allows us to set and use our own variables within the script. Setting variables allows you to temporarily store data and use it throughout the script, making the shell script more like a real computer program.

User variables can be any text string of up to 20 letters, digits, or an underscore character. User variables are case sensitive, so the variable Var1 is different from the variable var1. This little rule often gets novice script programmers in trouble.

Values are assigned to user variables using an equal sign. No spaces can appear between the variable, the equal sign, and the value (another trouble spot for novices). Here are a few examples of assigning values to user variables:

var1=10
var2=-57
var3=testing
var4=“still more testing”

The shell script automatically determines the data type used for the variable value. Variables defined within the shell script maintain their values throughout the life of the shell script but are deleted when the shell script completes.

Just like system variables, user variables can be referenced using the dollar sign:

$ cat test3
#!/bin/bash
# testing variables
days=10
guest=”Katie”
echo “$guest checked in $days days ago”
days=5
guest=”Jessica”
echo “$guest checked in $days days ago”
$

Running the script produces the following output:

$ chmod u+x test3
$ ./test3
Katie checked in 10 days ago
Jessica checked in 5 days ago
$

Each time the variable is referenced, it produces the value currently assigned to it. It’s important to remember that when referencing a variable value you use the dollar sign, but when referencing the variable to assign a value to it, you do not use the dollar sign. Here’s an example of what I mean:

$ cat test4
#!/bin/bash
# assigning a variable value to another variable
value1=10
value2=$value1
echo The resulting value is $value2
$

When you use the value of the value1 variable in the assignment statement, you must still use the dollar sign. This code produces the following output:

$ chmod u+x test4
$ ./test4
The resulting value is 10
$

If you forget the dollar sign, and make the value2 assignment line look like:

value2=value1
you get the following output:
$ ./test4
The resulting value is value1
$

Without the dollar sign the shell interprets the variable name as a normal text string, which is most likely not what you wanted.

Use of Backtick symbol (`) in shell variables :


The backtick allows you to assign the output of a shell command to a variable. While this doesn’t seem like much, it is a major building block in script programming.You must surround the entire command line command with backtick characters:

testing=`date`

The shell runs the command within the backticks and assigns the output to the variable testing. Here’s an example of creating a variable using the output from a normal shell command:

$ cat test5
#!/bin/bash
# using the backtick character
testing=`date`
echo “The date and time are: ” $testing
$

The variable testing receives the output from the date command, and it is used in the echo statement to display it. Running the shell script produces the following output:

$ chmod u+x test5
$ ./test5
The date and time are: Mon Jan 31 20:23:25 EDT 2011

Note: In bash you can also use the alternative $(…) syntax in place of backtick (`),which has the advantage of being re-entrant.

Example : $ echo ” Today’s date & time is :” $(date)
Today’s date & time is : Sun Jul 27 16:26:56 IST 2014