Tuesday, 30 October 2018

Examples of Awk Command in Unix - Part 2

Awk Command, LPI Study Material, LPI Guides, LPI Learning, LPI Tutorial and Material

1. Inserting a new line after every 2 lines


We will see how to implement this using the awk command with an example. 
The input "file.txt" contains the below data:

1 A
2 B
3 C
4 D
5 E
6 F

Let say, we want to insert the new line "9 Z" after every two lines in the input file. The required output data after inserting a new line looks as

1 A
2 B
9 Z
3 C
4 D
9 Z
5 E
6 F
9 Z

The awk command for getting this output is 

awk '{ 
 if(NR%2 == 0) 
 {
  print $0"\n9 Z";
 } 
 else 
 {
  print $0
 }
     }' file.txt

2. Replace the Nth occurrence of a pattern


The input file contains the data.

AAA 1
BBB 2
CCC 3
AAA 4
AAA 5
BBB 6
CCC 7
AAA 8
BBB 9
AAA 0

Now we want to replace the fourth occurrence of the first filed "AAA" with "ZZZ" in the file.
The required output is:

AAA 1
BBB 2
CCC 3
AAA 4
AAA 5
BBB 6
CCC 7
ZZZ 8
BBB 9
AAA 0

The awk command for getting this output is

awk 'BEGIN {count=0} 
 { 
  if($1 == "AAA") 
  { 
   count++
  } 
  if(count == 4) 
  { 
   sub("AAA","ZZZ",$1) 
  } 
 } 
 {
  print $0
 }' file.txt

3. Find the sum of even and odd lines separately


The input file data:

A 10
B 39
C 22
D 44
E 75
F 89
G 67

You have to get the second field and then find the sum the even and odd lines.
The required output is

174, 172

The awk command for producing this output is

awk '{ 
 if(NR%2 == 1) 
 {
  sum_e = sum_e + $2
 } 
 else 
 {
  sum_o = sum_o + $2
 }
      } 
      END { print sum_e,sum_o }' file.txt

4. Fibonacci series using awk command


Now we will produce the Fibonacci series using the awk command.

awk ' BEGIN{ 
 for(i=0;i<=10;i++) 
 { 
  if (i <=1 )
  {
   x=0;
   y=1;
   print i;
  }
  else
  {
   z=x+y;
   print z; 
   x=y;
   y=z;
  }
 } 
    }'

The output is 

0
1
1
2
3
5
8
13
21
34
55

5. Remove leading zeros from a file using the awk command. The input file contains the below data. 


0012345
05678
01010
00001

After removing the leading zeros, the output should contain the below data. 

12345
5678
1010
1

The awk command for this is. 

awk '{print $1 + 0}' file.txt
awk '{printf "%d\n",$0}' file.txt

Monday, 29 October 2018

5 tricks for using the sudo command

The sudoers file can provide detailed control over user privileges, but with very little effort, you can still get a lot of benefit from sudo. In this post, we're going to look at some simple ways to get a lot of value out of the sudo command in Linux.

Trick 1: Nearly effortless sudo usage


The default file on most Linux distributions makes it very simple to give select users the ability to run commands as root. In fact, you don’t even have to edit the /etc/sudoers file in any way to get started. Instead, you just add the users to the sudo or admin group on the system and you’re done.

sudo Command, Linux Tutorial and Material, Linux Guides, Linux Certification, LPI Study Material

Adding users to the sudo or admin group in the /etc/group file gives them permission to run commands using sudo.

$ grep sudo /etc/group
sudo:x:27:shs,jdoe,peanut

Assuming the standard /etc/sudoers setup, they should immediately be able to start using sudo commands once this change has been made. The privileges are derived through a line like this in the /etc/sudoers file:

%sudo   ALL=(ALL:ALL) ALL

The $sudo (or %admin) part of this line is a reference to the sudo (or admin) group. The rest of the line allows members of this group to run any command as any user. This much is built in. If you don’t want anyone to have this ability, don’t put anyone in the sudo (or admin) group on your system and this privilege level will not be implemented.

$ sudo whoami
[sudo] password for shs:
root

Trick 2: Running commands as other users — not just root


While most people use sudo access to run commands as root, it also allows you to run commands as other users. Just use the -u option with the sudo command and specify the username.

$ sudo -u jdoe whoami
[sudo] password for shs:
jdoe

Trick 3: Changing the default editor for the visudo command


The /etc/sudoers file should always be modified using the visudo command because this command helps to keep you from causing configuration errors that might make the resultant file unusable (i.e., it can break sudo). If the editor used by default is one that you’re not comfortable using, you can change it with this command:

sudo update-alternatives --config editor

This command will display a list of editors with the current one marked with an asterisk and allows you to select the one that you prefer.

$ sudo update-alternatives --config editor
[sudo] password for shs:
There are 3 choices for the alternative editor (providing /usr/bin/editor).

  Selection    Path               Priority   Status
------------------------------------------------------------
* 0           /bin/nano           40        auto mode
  1            /bin/ed              -100       manual mode
  2            /bin/nano           40        manual mode
  3            /usr/bin/vim.tiny   10        manual mode

Press <enter> to keep the current choice[*], or type selection number:

Trick 4: Switching to root


There are times when prefacing every command with "sudo" gets in the way of getting your work done. With a default /etc/sudoers configuration and membership in the sudo (or admin) group, you can assume root control using the command sudo su -. Extra care should always be taken when using the root account in this way.

$ sudo -i -u root
[sudo] password for jdoe:
root@stinkbug:~#


Trick 5: Fixing a corrupt /etc/sudoers file


A corrupt /etc/sudoers file can keep sudo from working and really mess up your day. Fortunately, there are some ways around this that don't involve a lot of work, and the visudo command provides some details on the problems needing to be fixed.

sudo Command, Linux Tutorial and Material, Linux Guides, Linux Certification, LPI Study Material

The problem

shs@stinkbug:/etc$ sudo date
>>> /etc/sudoers: syntax error near line 3 <<<
sudo: parse error in /etc/sudoers near line 3
sudo: no valid sudoers sources found, quitting
sudo: unable to initialize policy plugin

The fix

shs@stinkbug:/etc$ pkexec visudo
>>> /etc/sudoers: syntax error near line 3 <<<

Wednesday, 24 October 2018

5 Ways to Send Email From Linux Command Line

We all know the importance of emails these days for information transfer. There are many free emails service providers which we used for use like Gmail, Yahoo, RediffMail etc, which provides a web interface for sending and receiving emails. But this is not enough, sometimes we also required to send emails from system command line. This tutorial will provide you multiple ways to send emails from the Linux command line. This is useful for sending email through our shell scripts, cronjobs etc.

Linux Tutorial and Material, Linux Certification, Linux Study Material, Mail Command

There are various ways to send emails from the command line but here I am sharing few options used by most users. You can use anyone option given below to send email from Linux command line.

1. Using ‘sendmail’ Command


Sendmail is a most popular SMTP server used in most of Linux/Unix distribution. Sendmail allows sending email from command line. Use below instructions to send email using ‘sendmail‘ command.

Created a file with email content:

$ cat /tmp/email.txt

Subject: Terminal Email Send

Email Content line 1
Email Content line 2
Subject: line will be used as subject for email.

Now send email using the following command.

$ sendmail user@example.com  < /tmp/email.txt

2. Using ‘mail’ Command


mail command is most popular command to send emails from Linux terminal. Use few of below examples to send an email.

$ mail -s "Test Subject" user@example.com < /dev/null

◈ -s is used for defining subject for email.

Also, you can send an attachment with this command. Use -a for mailx and -A for mailutils.

$ mail -a /opt/backup.sql -s "Backup File" user@example.com < /dev/null

◈ Here -a is used for attachments. Use -A for debian based systems, which uses mailutils package.

Also, we can add comma separated emails to send the email to multiple recipients together.

$ mail -s "Test Email"  user@example.com,user2@example.com < /dev/null

3. Using 'mutt' command


Mutt is basically used for reading emails from Linux terminal from local user mailboxes, also useful to read emails from POP/IMAP servers. Mutt command is little similar to mail command. Use few of below examples to send an email.

$ mutt -s "Test Email" user@example.com < /dev/null

Send an email including an attachment

$ mutt  -s "Test Email" -a /opt/backup.sql user@example.com < /dev/null

4. Using 'SSMTP' Command


sSMTP allows users to send emails from SMTP server from Linux command line. For example to send an email to user admin@example.com use following command. Now type your subject of the email as below with keyword Subject. After that type your message to be sent to the user, After finishing your message press CTRL+d (^d) to send the email.

$ ssmtp admin@example.com
Subject: Test SSMTP Email
Email send test using SSMTP
via SMTP server.
^d

5. Using 'telnet' Command


As per my experience, all system administrators use telnet command to test remote port connectivity test or login to the server remotely. Most of the newbie in Linux doesn't know that we can send email using telnet also, which is the better way to troubleshoot email sending problems. Below is an example of email sending.

Red marked text is the user input and remaining is the responses of that commands.

$ telnet localhost smtp

Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
220 fbreveal.com ESMTP Sendmail 8.13.8/8.13.8; Tue, 22 Oct 2013 05:05:59 -0400
HELO yahoo.com
250 lpicentral.blogspot.in Hello lpicentral.blogspot.in [127.0.0.1], pleased to meet you
mail from: sender@lpicentral.blogspot.in
250 2.1.0 sender@lpicentral.blogspot.in... Sender ok
rcpt to: myemail@ymail.com
250 2.1.5 myemail@ymail.com... Recipient ok
data
354 Enter mail, end with "." on a line by itself
Hey
This is test email only

Thanks
.
250 2.0.0 r9M95xgc014513 Message accepted for delivery
quit
221 2.0.0 fbreveal.com closing connection
Connection closed by foreign host.

Saturday, 20 October 2018

File Test Operators / Operations Examples in Unix / Linux Shell Script

In linux and unix operating systems every thing is a file. When you are using files in your shell or bash script, it is a good idea to do some tests on the file before using it.

LPI Certification, LPI Guides, LPI Learning, LPI Study Materials, LPI Tutorial and Material

The file tests include:

◈ Checking for existence of the file.
◈ File is readable, writeable or executable.
◈ Type of the file and so on.

The file test operators are mostly used in the if clause of the bash script. The syntax is shown below:

if [ -option filename ]
then
  do something
else
  do something
fi

The different file test operators are listed below:

◈ a : True if the file exists.
◈ b : True if the file exists and is a block special file.
◈ c : True if the file exists and is a character special file.
◈ d : True if the file exists and is a directory.
◈ e : True if the file exists.
◈ f : True if the file exists and is a regular file.
◈ g : True if the file exists and its SGID bit is set.
◈ h : True if the file exists and is a symbolic link.
◈ k : True if the file exists and its sticky bit is set.
◈ p : True if the file exists and is a named pipe (FIFO).
◈ r : True if the file exists and is readable.
◈ s : True if the file exists and has a size greater than zero.
◈ t : True if file descriptor is open and refers to a terminal.
◈ u : True if the file exists and its SUID (set user ID) bit is set.
◈ w : True if the file exists and is writable.
◈ x : True if the file exists and is executable.
◈ O : True if the file exists and is owned by the effective user ID.
◈ G : True if the file exists and is owned by the effective group ID.
◈ L : True if the file exists and is a symbolic link.
◈ N : True if the file exists and has been modified since it was last read.
◈ S : True if the file exists and is a socket.

File Test Operator Example:


The following shell script checks for the existence of a regular file:

#!/bin/bash
#assign file name to the variable
FILE="linux-server.dat"

if [ -f $FILE ]
then
  echo "$FILE exists and is a regular file"
else
  echo "Either $FILE does not exist or is not a regular file"
fi 

Thursday, 18 October 2018

Bash Shell Script to Read / Parse Comma Separated (CSV) File - Unix / Linux

Q) How to parse CVS files and print the contents on the terminal using the bash shell script in Unix or Linux system?


It is the most common operation in Unix system to read the data from a delimited file and applying some operations on the data. Here we see how to read the Comma separated value (CSV) file using the while loop in shell script and print these values on the Unix terminal.

Bash Shell Script, Linux Tutorial and Material, Unix Certification, Linux Certification, Linux CSV

Consider the below CSV file as an example:

> cat os_server.csv
Unix, dedicated server
Linux, virtual server

This file contains two fields. First field is operating system and the second field contains the hosting server type. Let see how to parse this CVS file with simple bash script shown below:

#!/usr/bin/bash

INPUT_FILE='unix_file.csv'

IFS=','

while read OS HS
do

echo "Operating system - $OS"
echo "Hosting server type - $HS"

done < $INPUT_FILE

Here IFS is the input field separator. As the file is comma delimited, the IFS variable is set with comma. The output of the above script is

Operating system - Unix
Hosting server type - dedicated server
Operating system - Linux
Hosting server type - virtual server

Here in the code, the fourth line (IFS=',') and sixth line (while) can be merged into a single statement as shown below:

while IFS=',' read OS HS

Saturday, 13 October 2018

Crontab in Linux with 20 Useful Examples to Schedule Jobs

The crontab is used for running specific tasks on a regular interval. Linux crontab is similar to windows task schedules. Crontab is very useful for routine tasks like scheduling system scanning, daily backups etc. Crontab executes jobs automatically in the backend on a specified time and interval. In this tutorial, you will learn to uses of crontab with 20 useful examples for scheduling jobs. You can also use crontab for the tasks to run once in future only, but for any tasks to run once we recommends to use Linux at command.

Linux Crontab Syntax


Linux crontab has six fields. 1-5 fields defines the date and time of execution. The 6’th fields are used for command or script to be executed.The Linux crontab syntax are as following:

[Minute] [hour] [Day_of_the_Month] [Month_of_the_Year] [Day_of_the_Week] [command]

Linux Tutorial and Material, Linux Study Material, Linux Certification, Linux Crontab

◈ Astrics (*) – Matches anything
◈ Define range – You can define range using the hypen like: 1-10 or 20-30 or sun-fri or feb-apr
◈ Define multiple range – You can define multiple ranges with command seprated like: jan-mar,jul-sep

How to Add/Edit Crontab


To add or update job in crontab, use below command. It will open crontab file in the editor where a job can be added/updated.

Linux Tutorial and Material, Linux Study Material, Linux Certification, Linux Crontab

crontab -e

By default, it will edit crontab entries of current logged in user. To edit other user crontab use command as below

crontab -u username -e

Change EDITOR environment variable to change your default editor.

How to List Crontab


To view crontab entries of current user use the following command.

crontab -l

Use -u followed by username to view crontab entries of the specified user.

crontab -u username -l

20 Useful Crontab Examples


1. Schedule a cron to execute at 2am daily.


This will be useful for scheduling database backup on daily basis.

0 2 * * * /bin/sh backup.sh

◈ are used for matching all the records.

2. Schedule a cron to execute twice a day.


Below example command will execute at 5 AM and 5 PM daily. You can specify multiple time stamp by comma separated.

0 5,17 * * * /scripts/script.sh

3. Schedule a cron to execute on every minutes.


Generally, we don’t require any script to execute on every minute but in some case, you may need to configure it.

* * * * *  /scripts/script.sh

4. Schedule a cron to execute on every Sunday at 5 PM.


This type of cron is useful for doing weekly tasks, like log rotation etc.

0 17 * * sun  /scripts/script.sh

5. Schedule a cron to execute on every 10 minutes.


If you want to run your script on 10 minutes interval, can configure like below. These type of crons are useful for monitoring.

*/10 * * * * /scripts/monitor.sh

*/10: means to run on every 10 minutes. Same as if you want to execute on every 5 minutes use */5.

6. Schedule a cron to execute on selected months.


Sometimes we required scheduling a task to be executed for selected months only. Below example script will run in January, May and August months.

* * * jan,may,aug *  /script/script.sh

7. Schedule a cron to execute on selected days.


If you required scheduling a task to be executed for selected days only. Below example will run on each Sunday and Friday at 5 PM.

0 17 * * sun,fri  /script/script.sh

8. Schedule a cron to execute on first sunday of every month.


To schedule a script to execute a script on first Sunday only is not possible by time parameter, But we can use the condition in command fields to do it.

0 2 * * sun  [ $(date +%d) -le 07 ] && /script/script.sh

9. Schedule a cron to execute on every four hours.


If you want to run a script on 4 hours interval. It can be configured like below.

0 */4 * * * /scripts/script.sh

10. Schedule a cron to execute twice on every Sunday and Monday.


To schedule a task to execute twice on Sunday and Monday only. Use following settings to do it.

0 4,17 * * sun,mon /scripts/script.sh

11. Schedule a cron to execute on every 30 Seconds.


To schedule a task to execute on every 30 seconds is not possible by time parameters, But it can be done by schedule same cron twice like below.

* * * * * /scripts/script.sh
* * * * *  sleep 30; /scripts/script.sh

12. Schedule a multiple tasks in single cron.


To configure multiple tasks with single cron, Can be done by separating tasks by the semicolon ( ; ).

* * * * * /scripts/script.sh; /scripts/scrit2.sh

13. Schedule tasks to execute on yearly ( @yearly ).


@yearly timestamp is similar to “0 0 1 1 *”. It will execute task on the first minute of every year, It may useful to send new year greetings.

@yearly /scripts/script.sh

14. Schedule tasks to execute on monthly ( @monthly ).


@monthly timestamp is similar to “0 0 1 * *”. It will execute a task in the first minute of the month. It may useful to do monthly tasks like paying the bills and invoicing to customers.

@monthly /scripts/script.sh

15. Schedule tasks to execute on Weekly ( @weekly ).


@weekly timestamp is similar to “0 0 1 * mon”. It will execute a task in the first minute of the week. It may useful to do weekly tasks like the cleanup of system etc.

@weekly /bin/script.sh

16. Schedule tasks to execute on daily ( @daily ).


@daily timestamp is similar to “0 0 * * *”. It will execute a task in the first minute of every day, It may useful to do daily tasks.

@daily /scripts/script.sh

17. Schedule tasks to execute on hourly ( @hourly ).


@hourly timestamp is similar to “0 * * * *”. It will execute a task in the first minute of every hour, It may useful to do hourly tasks.

@hourly /scripts/script.sh

18. Schedule tasks to execute on system reboot ( @reboot ).


@reboot is useful for those tasks which you want to run on your system startup. It will be same as system startup scripts. It is useful for starting tasks in the background automatically.

@reboot /scripts/script.sh

19. Redirect Cron Results to specified email account.


By default, cron sends details to the current user where cron is scheduled. If you want to redirect it to your other account, can be done by setup MAIL variable like below

# crontab -l
MAIL=bob
0 2 * * * /script/backup.sh

20. Taking backup of all crons to plain text file.


I recommend keeping a backup of all jobs entry in a file. This will help you to recover crons in case of accidental deletion.

Check current scheduled cron:

# crontab -l
MAIL=rahul
0 2 * * * /script/backup.sh

Backup cron to text file:

# crontab -l > cron-backup.txt
# cat cron-backup.txt
MAIL=rahul
0 2 * * * /script/backup.sh

Removing current scheduled cron:

# crontab -r
# crontab -l
no crontab for root

Restore crons from text file:

# crontab cron-backup.txt
# crontab -l
MAIL=rahul
0 2 * * * /script/backup.sh

Thursday, 11 October 2018

Git Command with Practical Examples on Linux – Part 2

Git Command, LPI Tutorial and Material, LPI Certification, LPI Learning

In previous article we have learned basic Git workflow. In this article we’ll be focusing on some advanced features like miscellaneous repository actions, branching and tagging.

Miscellaneous repository actions


Rename

As name suggests rename operation instructs Git that traced file has been modified. To rename file execute following command:

$ git mv README NEW-README

Now let us check the repository status:

$ git status -s
R  README -> NEW-README

In above output letter R appears before README, which indicates file has been renamed. Next column shows the old and new file name.

NOTE: To make these changes in remote repository execute git push command.

Move

Move operation is used to move file from one directory location to another. To move file execute following command:

$ mkdir new-dir
$ git mv README new-dir/

In this example, we have created new directory and move file to that directory. Now let us check the repository status:

$ git status -s
R  README -> new-dir/README

Above output shows that file has been moved to new directory.

NOTE: To make these changes in remote repository execute git push command.

Delete

As name suggests, delete operation removes file/directory from Git. To remove file execute following command:

$ git rm README

Now let us check the repository status:

$ git status -s
 D README

In above output letter D appears before the README which indicates that file has been removed from repository.

NOTE: To make these changes in remote repository execute git push command.

RESET

This operation reset current HEAD to the specified state. In Git, HEAD is the reference pointer which points to the latest commit.

Let us understand this with example:

$ touch AUTHORS
$ git add AUTHORS
$ git status -s
A  AUTHORS

In above example, we have created AUTHORS file and added it to changeset. Now this file will be part of next commit. Git reset command will adjust HEAD reference and remove file from changeset

$ git reset HEAD AUTHORS

Now, let us check the repository status:

$ git status -s
?? AUTHORS

Git Command, LPI Tutorial and Material, LPI Certification, LPI Learning

As expected, above output shows that file has been removed from changeset hence ?? symbol appears before filename.

Working with branches


Branch in version control system is a independent line of development. Unlike other version control system branching is really lightweight in Git. In this section we’ll discuss various branch related features.

Create branch

To create new branch execute following command:

$ git branch my-feature-branch

In above command my-feature-branch is branch name. This command will create local branch.

To create remote branch execute git push command as follows:

$ git push origin my-feature-branch

List branch

To list branch execute branch command without any argument.

$ git branch
* master
  my-feature-branch

For instance, above output lists local branches. Asterisk symbol represents current branch. In our case it is master branch.

To list local as well as remote branches execute following command:

$ git branch -a
* master
  my-feature-branch
  remotes/origin/HEAD -> origin/master
  remotes/origin/master

In above output, branches highlighted in red colour are remote branches.

Delete branch

To delete local branch use -d option as follows:

$ git branch -d my-feature-branch

Deleted branch my-feature-branch (was 220bf4d).

To delete remote branch execute following command:

$ git push origin :my-feature-branch

Note that we have use colon(:) with branch name which instructs git to delete remote branch

Switch branch

To switch in between branches execute checkout command as follows:

$ git checkout my-feature-branch

Switched to branch ‘my-feature-branch’

Now let us check the current branch:

$ git branch
  master
* my-feature-branch

Checkout branch

So far we have used separate command to create and switch to branch. However, we can achieve this is single command as follows:

$ git checkout -b my-feature-branch

Switched to a new branch ‘my-feature-branch’

Now let us check the current branch:

$ git branch
  master
* my-feature-branch

Restore working tree


Git allows us to discard uncommitted changes. We can achieve this using checkout command.  Let us understand this with example.

First modify existing file and check repository status:

$ echo "Update README" >> README
$ git status -s
 M README

Now to discard changes of README file, execute following command:

$ git checkout -- README
$ git status -s

As expected our working tree is clean hence last command is not showing any output.

Merge branch

As name suggests merge operation applies changes from one branch to another. Let us understand this step by step.

First create a new branch and commit changes in this branch:

$ git checkout -b my-feature-branch

Switched to a new branch ‘my-feature-branch’

$ echo "Update README" >> README.md
$ git add README.md
$ git commit -m "Updated README"
[my-feature-branch 42e28aa] Updated README
 1 file changed, 1 insertion(+)

Let us switch to master branch and apply changes to master branch

$ git checkout master

Switched to branch ‘master’

Your branch is up to date with ‘origin/master’.

$ git merge my-feature-branch
Updating 220bf4d..42e28aa
Fast-forward
 README.md | 1 +
 1 file changed, 1 insertion(+)

Now let us verify that changes are applied to master branch:

$ git log -1 --oneline
42e28aa (HEAD -> master, my-feature-branch) Updated README

Working with tags


Tag is a reference pointer to a particular commit in Git history. Often tag is used to mark release version of product. In this section we’ll discuss basic tag operations like create, list and delete

Create tag

Creating tag is really simple. To create tag execute following command:

$ git tag my-tag

Above command will create tag with name  my-tag. To create tag in remote repository execute push command as follows:

$ git push origin my-tag

List tag

To list tag execute Git tag command without any argument as follows:

$ git tag
my-tag

Delete tag

To delete tag execute Git tag command with -d argument as follows:

$ git tag -d my-tag
Deleted tag 'my-tag' (was 220bf4d)

To delete tag from remote repository execute following command:

$ git push origin :my-tag

Tuesday, 9 October 2018

Linux Professional Institute Rolls Out its Global Training Partner Program to Japan

LPI’s global partner program has been expanded to meet the needs of training companies in Japan.

LPI Guides, LPI Learning, LPI Study Material, LPI Tutorial and Material

Linux Professional Institute (LPI), the world’s largest organization focusing on certification and support for people working with open source, is expanding and enhancing its globally-tested training partner program in Japan.

The launch of LPI’s global training partner program expands customer support services being provided by LPI’s new Tokyo office, and builds on the success of LPI programs operated elsewhere in the world.

The Japan partner programs supports both commercial and academic educational organizations:

◈ LPI Approved Training Partner Program (LPI-ATP) for commercial training companies
◈ LPI Approved Academic Partner Program (LPI-AAP) for nonprofit and public sector schools and colleges

“Our education partners are a critical part of our community,” said Kenji Ito, Director of Communications for the Tokyo office of LPI. “We have received numerous inquiries from academic and training organizations who are eager to work with us, and we are advancing relationships with several partners already. I am eager to launch our new initiatives which combine the strength of LPI’s global program with a sensitivity to what is needed in Japan.”

Kenji Ito announced that LPI will be holding a series of seminars in November in Tokyo for organizations interested in becoming partners. “We are especially interested to work with partners in support of the updated LPIC-1 program, as well as the Linux Essentials program which is new in Japan.”

Approved partners must meet LPI’s quality and training criteria. Those who qualify are able to participate in co-marketing opportunities with LPI and receive discounted pricing for exams taken by their students. They are also invited to participate in the development of LPI’s certification programs, both in the creation of new programs and the updating of old ones.

Participation in both of the LPI educational partner programs is offered at no cost to those organizations which meet the quality standards. Partners who train and certify larger numbers of students may qualify for Silver, Gold and Platinum levels of sponsorship which offer more benefits and higher levels of discounted exam costs.

LPI, which celebrates its 20th anniversary in 2019, is a global organization with a presence in more than 180 countries. Its certification program has been trusted by more than a half-million people worldwide. In Japan, more than 320,000 LPI exams have been delivered to date.

How to Take an Exam


■ Those who do not have LPI-ID, please register at:
https://cs.lpi.org/caf/Xamman/register

■ For details of certification, please refer to the following URL.
https://www.lpi.org/ja/our-certifications/summary-of-certifications

About the Linux Professional Institute:


Established in 1999, the Linux Professional Institute (LPI) is the global certification standard and career support organization for open source professionals. With more than 600,000 exams delivered, it's the world’s first and largest vendor-neutral Linux and open source certification body. LPI has certified professionals in over 180 countries, delivers exams in 9 languages, and has hundreds of training partners.

LPI advances the Linux and Open Source movement through strategic partners, sponsorships, innovative programs and community development activities. LPI's major founding financial sponsors are Platinum Sponsors IBM, Linux Journal, Linux Magazine, As well as Gold Sponsors HP and IDG.

Saturday, 6 October 2018

Mastering the “Kill” Command in Linux

Kill Command, LPI Guides, LPI Tutorial and Material, LPI Certification, LPI Study Material

It doesn’t matter which operating system you are using – you will surely come across a misbehaving application that locks itself up and refuses to close. In Linux (and Mac) there is a “kill” command that you can use to terminate the application forcefully. In this post we will show you the various ways you can make use of the “kill” command to terminate an application.

Kill Commands and Signals


When you execute a “kill” command, you are in fact sending a signal to the system to instruct it to terminate the misbehaving app. There is a total of sixty signals that you can use, but all you really need to know is SIGTERM (15) and SIGKILL (9).

You can view all the signals with the command:

kill -l

Kill Command, LPI Guides, LPI Tutorial and Material, LPI Certification, LPI Study Material

◈ SIGTERM – This signal requests that a process stop running. This signal can be ignored. The process is given time to gracefully shut down. When a program gracefully shuts down, that means it is given time to save its progress and release resources. In other words, it is not forced to stop.

◈ SIGKILL – The SIGKILL signal forces the process to stop executing immediately. The program cannot ignore this signal. Unsaved progress will be lost.

The syntax for using kill is:

kill [signal or option] PID(s)

The default signal (when none is specified) is SIGTERM. When that doesn’t work, you can use the following to kill a process forcefully:

kill SIGKILL PID

or

kill -9 PID

where the -9 flag refers to SIGKILL signal.

If you are not aware of the PID of the application, simply run the command:

ps ux

and it will display all the running applications together with its PID.

Kill Command, LPI Guides, LPI Tutorial and Material, LPI Certification, LPI Study Material

For example, to kill the Chrome app, run the command:

kill -9 3629

Do also note that you can kill multiple processes at the same time.

kill -9 PID1 PID2 PID3

PKill


The pkill command allows the use of extended regular expression patterns and other matching criteria. Instead of using PID, you can now kill applications by entering their process names. For example, to kill the Firefox browser, just run the command:

pkill firefox

As it matches a regular expression pattern, you can also enter a partial name of the process, such as:

pkill fire

To avoid killing the wrong processes, you might want to do a “pgrep -l [process name]” to list the matching processes.

Kill Command, LPI Guides, LPI Tutorial and Material, LPI Certification, LPI Study Material

Killall


Killall uses the process name as well instead of PID, and it kills all instances of the process with the same name. For example, if you are running multiple instances of the Firefox browser, you can kill them all with the command:

killall firefox

xkill


Xkill is a graphical way to kill an application. When you type xkill in the terminal, your mouse cursor will instantly become a “cross.” All you have to do is click the “cross” on the misbehaving app, and it will kill the application instantly.

Thursday, 4 October 2018

Debugging Shell Scripts in Linux

Linux Tutorial and Material, Linux Certification, Linux Guide, Linux Learning

In most of the programming languages debugger tool is available for debugging. A debugger is a tool that can run a program or script that enables you to examine the internals of the script or program as it runs. In the shell scripting we don”t have any debugger tool but with the help of command line options (-n, -v and -x ) we can do the debugging.

Disabling the Shell ( -n option)


The -n option, shot for noexec ( as in no execution), tells the shell to not run the commands. Instead, the shell just checks for syntax errors. This option will not convince the shell to perform any more checks. Instead the shell just performs the normal syntax check. With -n option, the shell doesn’t execute your commands, so you have a safe way to test your scripts if they contain syntax error.

The follow example shows how to use -n option.

Let us consider a shell script with a name debug_quotes.sh

#!/bin/bash
echo "USER=$USER
echo "HOME=$HOME"
echo "OSNAME=$OSNAME"

Now run the script with -n option

$ sh -n debug_quotes
debug_quotes: 8: debug_quotes: Syntax error: Unterminated quoted string

As the above outputs shows that there is syntax error , double quotes are missing.

Displaying the Scripts Commands ( -v option )


The -v option tells the shell to run in verbose mode. In practice , this means that shell will echo each command prior to execute the command. This is very useful in that it can often help to find the errors.

Let us create a shell script with the name “listusers.sh” with below contents

lpicentral@localhost:~$ cat listusers.sh

#!/bin/bash

cut -d : -f1,5,7 /etc/passwd | grep -v sbin | grep sh | sort > /tmp/users.txt
awk -F':' ' { printf ( "%-12s %-40s\n", $1, $2 ) } ' /tmp/users.txt

#Clean up the temporary file.
/bin/rm -f /tmp/users.txt

Now execute the script with -v option

lpicentral@localhost:~$ sh -v listusers.sh

#!/bin/bash

cut -d : -f1,5,7 /etc/passwd | grep -v sbin | grep sh | sort > /tmp/users.txt
awk -F':' ' { printf ( "%-12s %-40s\n", $1, $2 ) } ' /tmp/users.txt
guest-k9ghtA Guest,,,
guest-kqEkQ8 Guest,,,
guest-llnzfx Guest,,,
pradeep pradeep,,,
mail admin Mail Admin,,,

#Clean up the temporary file.
/bin/rm -f /tmp/users.txt

lpicentral@localhost:~$

In the above output , script output gets mixed with commands of the scripts. But however , with -v option , at least you get a better view of what the shell is doing as it runs your script.

Combining the -n & -v Options


We can combine the command line options ( -n & -v ). This makes a good combination because we can check the syntax of a script while seeing the script output.

Let us consider a previously used script “debug_quotes.sh”

lpicentral@localhost:~$ sh -nv debug_quotes.sh

#!/bin/bash
#shows an error.

echo "USER=$USER
echo "HOME=$HOME"
echo "OSNAME=$OSNAME"

debug_quotes: 8: debug_quotes: Syntax error: Unterminated quoted string

lpicentral@localhost:~$

Tracing Script Execution ( -x option )


The -x option, short for xtrace or execution trace, tells the shell to echo each command after performing the substitution steps. Thus , we can see the values of variables and commands. Often, this option alone will help to diagnose a problem.

In most cases, the -x option provides the most useful information about a script, but it can lead to a lot of output. The following example show this option in action.

lpicentral@localhost:~$ sh -x listusers.sh

+ cut -d :+ -f1,5,7 /etc/passwd
grep -v sbin
+ sort
+ grep sh
+ awk -F: { printf ( "%-12s %-40s\n", $1, $2 ) } /tmp/users.txt
guest-k9ghtA Guest,,,
guest-kqEkQ8 Guest,,,
guest-llnzfx Guest,,,
pradeep pradeep,,,
mail admin Mail Admin,,,
+ /bin/rm -f /tmp/users.txt

lpicentral@localhost:~$

In the above output, shell inserted a + sign in front of the commands.

Monday, 1 October 2018

10 ‘rm’ command examples for beginners

rm stands for ‘remove‘ as the name suggests rm command is used to delete or remove files and directory in UNIX like operating system. If you are new to Linux then you should be very careful while running rm command because once you delete the files then you can not recover the contents of files and directory. Though there are some tools and commands through which deleted files can be recovered but for that you need expert skills.

In this post i will demonstrate 10 Linux rm command examples. Below is the basic syntax of rm command.

# rm <options> {files}

options used in rm command.

rm Command, LPI Tutorial and Material, LPI Guides, LPI Learning, LPI Certification

Example: 1 Remove or delete a file.


Let’s delete a file with name “linuxstufff.log”

[lpicentral@cloud ~]$ rm linuxstufff.log
[lpicentral@cloud ~]$

Delete multiple files at once.

Let’s assume that i want to delete four text files at once. Use the below syntax

# rm {file1} {file2}] {file3} {file4}

[lpicentral@cloud ~]$ rm file1.txt file2.txt file3.txt file4.txt
[lpicentral@cloud ~]$

Example: 2 Delete the files interactively.


‘-i‘ option in rm command will prompt before deleting a file, example is shown below.

[lpicentral@cloud ~]$ rm -i linuxstufff.log
rm: remove regular file ‘linuxstufff.log’? y
[lpicentral@cloud ~]$

Example: 3 Delete a empty directory


use ‘-d‘ option in rm command to delete a empty directory.

[lpicentral@cloud ~]$ ls -R appdata/
appdata/:
[lpicentral@cloud ~]$ rm -d appdata/
[lpicentral@cloud ~]$

we can also use ‘rmdir‘ command to delete empty folder or directory.

[lpicentral@cloud ~]$ ls -R appdata/
appdata/:
[lpicentral@cloud ~]$ rmdir appdata
[lpicentral@cloud ~]$

Example: 4 Deleting a directory recursively using ‘-r’ option


‘-r‘ option in rm command will delete all the files and sub-directories recursively of the parent directory.

[lpicentral@cloud ~]$ ls -lR dbstore/
dbstore/:
total 0
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 26 23:59 file1.log
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 26 23:59 file2.log
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 26 23:59 file3.log
drwxrwxr-x. 2 lpicentral lpicentral 6 Mar 26 23:59 service

dbstore/service:
total 0
[lpicentral@cloud ~]$ rm -r dbstore/
[lpicentral@cloud ~]$

Example: 5 Delete the files and sub-directories interactively.


Use ‘-ri‘ option in rm command to delete file and sub-directories interactively, Let’s assume we want to all files and directories of ‘dbstore’ directory interactively.

[lpicentral@cloud ~]$ ls -lR dbstore/
dbstore/:
total 0
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 27 00:02 file1.log
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 27 00:02 file2.log
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 27 00:02 file3.log
drwxrwxr-x. 2 lpicentral lpicentral 6 Mar 27 00:02 service

dbstore/service:
total 0
[lpicentral@cloud ~]$ rm -ri dbstore/
rm: descend into directory ‘dbstore/’? y
rm: remove regular empty file ‘dbstore/file1.log’? y
rm: remove regular empty file ‘dbstore/file2.log’? y
rm: remove regular empty file ‘dbstore/file3.log’? y
rm: remove directory ‘dbstore/service’? y
rm: remove directory ‘dbstore/’? y
[lpicentral@cloud ~]$

Example: 6 Deleting files forcefully using ‘-f’ option


‘-f‘ option in rm command will remove or delete the files forcefully regardless of its permissions and will also ignore non-existing files.

Let’s delete a write-protected file ‘tech.txt’

[lpicentral@cloud ~]$ ls -l tech.txt
-r--r--r--. 1 lpicentral lpicentral 0 Mar 27 00:23 tech.txt
[lpicentral@cloud ~]$
[lpicentral@cloud ~]$ rm tech.txt
rm: remove write-protected regular empty file ‘tech.txt’?

As we can see above that when we try to delete a write-protected file using rm command without ‘-f’ option , it gives us a prompt to delete write-protected file.

Now try to delete a file using ‘-f’ option.

[lpicentral@cloud ~]$ rm -f tech.txt
[lpicentral@cloud ~]$

Also try to delete a non-existing file.

[lpicentral@cloud ~]$ rm -f nonexist.txt
[lpicentral@cloud ~]$

Note : ‘-f’ option of rm command will not work for write-protect directories,

Let’s take an example , directory ‘/home/lpicentral/location/ ‘ is write protected and file (‘db_stuff‘) inside this directory is non-protected.

[lpicentral@cloud ~]$ ls -ld /home/lpicentral/location/
drwxrwxr-x. 2 root root 29 Mar 27 00:43 /home/lpicentral/location/

[lpicentral@cloud ~]$ ls -l /home/lpicentral/location/db_stuff
-rw-rw-r--. 1 lpicentral lpicentral 17 Mar 27 00:43 /home/lpicentral/location/db_stuff

[lpicentral@cloud ~]$ rm -f /home/lpicentral/location/db_stuff
rm: cannot remove ‘/home/lpicentral/location/db_stuff’: Permission denied
[lpicentral@cloud ~]$

Example 7: Prompt once before deleting more than three files or recursive delete.


‘-I‘ option in rm command will prompt once before deleting more than three files or recursive delete.

Suppose i want to delete all log files which starts with the name ‘app’ under the directory ‘linux_store’.

[lpicentral@cloud ~]$ ls -l linux_store/
total 0
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 27 01:07 app1.log
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 27 01:07 app2.log
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 27 01:07 app3.log
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 27 01:07 app4.log
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 27 01:07 app5.log
[lpicentral@cloud ~]$ rm -I linux_store/app*
rm: remove 5 arguments? y
[lpicentral@cloud ~]$

Example: 8 Regular expression in rm command


We can use regular expression in the rm command, some of the examples are shown below :

Let’s delete 5 log files starting from log1 to log5 under the directory ‘linux_store‘.

[lpicentral@cloud linux_store]$ pwd
/home/lpicentral/linux_store
[lpicentral@cloud linux_store]$ ll
total 0
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 27 01:15 log1.txt
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 27 01:15 log2.txt
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 27 01:15 log3.txt
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 27 01:15 log4.txt
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 27 01:15 log5.txt
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 27 01:15 log6.txt
........................................
[lpicentral@cloud linux_store]$
[lpicentral@cloud linux_store]$ rm -f log{1..5}.txt
[lpicentral@cloud linux_store]$

Delete all the files of the current directory that ends with ‘.txt’

[lpicentral@cloud linux_store]$ rm -f *.txt
[lpicentral@cloud linux_store]$

Delete all files of present working directory which has 3 characters in extension.

[lpicentral@cloud linux_store]$ rm -f *.???
[lpicentral@cloud linux_store]$

Example: 9 Delete large number files using rm command.


If your are trying to delete large number of files using rm command then you will get an error message ‘Argument list too long’

In the below example i am trying to delete all the files (around ‘300001’) of the directory ‘/home/lpicentral/linux_store‘ at once.

[lpicentral@cloud linux_store]$ ls -l | wc -l
300001
[lpicentral@cloud linux_store]$ rm *.log
-bash: /bin/rm: Argument list too long
[lpicentral@cloud linux_store]$

To resolve this issue , use the below find command.

[lpicentral@cloud ~]$ find ~/linux_store/ -type f -exec rm {} \;
[lpicentral@cloud ~]$

Example: 10 Delete a file which starts with hyphen symbol (-)


Let’s assume that we have a file with name ‘-store‘ in our current working directory and we want to delete this file.

[lpicentral@cloud linux_store]$ ll
total 0
-rw-rw-r--. 1 lpicentral lpicentral 0 Mar 27 02:05 -store
[lpicentral@cloud linux_store]$ rm -store
rm: invalid option -- 's'
Try 'rm --help' for more information.
[lpicentral@cloud linux_store]$

Use below either of the command to delete such files.

[lpicentral@cloud linux_store]$ rm -- \ -store
[lpicentral@cloud linux_store]$

OR

[lpicentral@cloud linux_store]$ rm ./\ -store
[lpicentral@cloud linux_store]$