Friday, 29 June 2018

Add Job to Cron (Crontab Command Examples) - Unix / Linux

Unix or Linux operating system provides a feature for scheduling the jobs. You can setup command or scripts which will run periodically at the specified time. The Crontab is command used to add or remove jobs from the cron. The cron service is a daemon runs in the background and checks for /etc/crontab file, /etc/con.*/ directories and /var/spool/cron/ directory for any scheduled jobs.

Unix Command, Linux Command, Crontab Command, LPI Study Material, LPI Certifications

Each user has a separate /var/spool/cron/crontab file. Users are not allowed directly to modify the files. The crontab command is used for setting up the jobs in the cron.

The format of crontab command is

* * * * * command to be executed

You can easily remember this command in the below format

MI HH DOM MON DOW command

The field descriptions of the crontab are explained below:

MI  : Minutes      from 0 to 59
HH  : Hours        from 0 to 23
DOM : Day of month from 0 to 31
MON : Months       from 1 to 12
DOW : Day of week  from 0 to 7   (0 or 7 represents Sunday)

Command: Any command or script to be scheduled

Let see the usage of crontab command with examples.

1. List crontab entries


You can list out all the jobs which are already scheduled in cron. Use "crontab -l" for listing the jobs.

crontab -l
0 0 * * *  /usr/local/bin/list_unix_versions.sh

The above contab command displays the cron entries. Here the shell script for listing the unix versions (list_unix_version.sh) is scheduled to run daily at midnight.

2. List crontab entries of other users


To list the corntab entries of other user in the unix, use the -u option with crontab. The syntax is shown below:

crontab -u username -l

3. Removing all crontab entries


You can un-schedule all the jobs by removing them from the crontab. The syntax for removing all the crontab entries is

crontab -r

For removing other user’s crontab entries:
crontab -u username -r

4. Editing the crontab


You can edit the crontab and add a new job to it. You can also remove an existing job from the crontab. Use the -e option for editing the crontab.

crontab -e

For editing other user’s crontab entries:
crontab -u username -e

This will open a file in VI editor. Now use the VI commands for adding, removing the jobs and for saving the crontab entries.

5. Schedule a job to take oracle backup on every Sunday at midnight


Edit crontab using "crontab -e" and append the following entry in the file.

0 0 * * 0 /usr/local/bin/oracle_backup.sh

6. Schedule a job to run every six hours in a day


You can schedule a job to run more than once in a day. As an example the following crontab entry takes the mysql backup more than once in a day.

0 0,6,12,18 * * * /usr/bin/mysql_backup.sh

Here the list 0,6,12,18 indicates midnight, 6am, 12pm and 6pm respectively.

7. Schedule job to run for the first 15 days of the month.


You can schedule a job by specifying the range of values for a field. The following example takes the sql server backup daily at midnight for the first 15 days in a month.

0 0 * 1-15 * /usr/bin/sql_server_backup.sh

8. Schedule job to run every minute.


The following crontab command runs the command to send emails to group of users for every minute.

* * * * * /bin/batch_email_send.sh

9. Taking backup of cron entries


Before editing the cron entries, it is good to take backup of the cron entries. So that even if you do mistake you can get back those entries from the backup.

crontab -l > /var/tmp/cron_backup.dat

10. Restoring the cron entries


You can restore the cron entries from the backup as

crontab cron_backup.dat

Understanding the Operators:

There are three operators allowed for specifying the scheduling times. They are:

◈ Asterisk (*) : Indicates all possible values for a field. An asterisk in the month field indicates all possible months (January to December).
◈ Comma (,) : Indicates list of values. See example 6 above.
◈ Hyphen (-): Indicates range of values. See example 7 above.

Disabling Emails:


By default the crontab sends emails to the local user if the commands or scripts produce any output. To disable sending of emails redirect the output of commands to /dev/null 2>&1.

0 0 * 20 * /usr/bin/online_backup.sh > /dev/null 2>&1

Note: you cannot schedule a job to run at second’s level as the minimum allowed scheduling is at minute level.

Wednesday, 27 June 2018

Date Command in Unix and Linux Examples

Date Command, Unix Command, Linux Command, LPI Study Materials, LPI Learning

Date command is used to print the date and time in unix. By default the date command displays the date in the time zone that the unix operating system is configured.

Now let see the date command usage in unix

Date Command Examples:


1. Write a unix/linux date command to print the date on the terminal?


>date
Mon Jan 23 01:37:51 PST 2012

This is the default format in which the date command print the date and time. Here the unix server is configured in pacific standard time.

2. Write a unix/linux date command to print the date in GMT/UTC time zone?


>date -u
Mon Jan 23 09:40:21 UTC 2012

The -u option to the date command tells it to display the time in Greenwich Mean Time.

3. Write a unix/linux date command to sett the date in unix?


You can change the date and time by using the -s option to the date command.

>date -s "01/01/2000 12:12:12"

4. Write a unix/linux date command to display only the date part and ignore the time part?


>date '+%m-%d-%Y'
01-23-2012

You can format the output of date command by using the %. Here %m for month, %d for day and %Y for year.

5. Write a unix/linux date command to display only the time part and ignore the date part?


>date '+%H-%M-%S'
01-48-45

Here %H is for hours in 24 hour format, %M is for minutes and %S for seconds

6. Write a unix/linux date command to format both the date and time part.


>date '+%m-%d-%Y %H-%M-%S'
01-23-2012 01-49-59

7. Write a unix/linux date command to find the number of seconds from unix epoch.


>date '+%s'
1327312228

Unix epoch is the date on January 1st, 1970. The %s option is used to find the number of seconds between the current date and unix epoch.

Saturday, 23 June 2018

Tail Command Examples in Unix / Linux

Tail Command, LPI Study Materials, LPI Guides, LPI Learning, LPI Certifications, Linux and Unix

The tail command in unix or linux system is used to print the last N lines from the file on the terminal. Tail command is especially used with log files to read the last few lines to know about the error messages. The syntax of tail command is

tail [options] [files]

The tail command options are:

◈ c : Prints the last N bytes of file; With leading +, prints the characters from the N byte in the file.
◈ n : Prints last N lines; With leading + prints lines from the Nth line in the file.
◈ f : Prints the appended lines on the terminal as the file grows.

Tail Command Examples


Create the following file in your linux or unix operating system for practising the examples:

> cat example.txt
virtual storage
oracle virtual instance
mysql backup
dedicated hosting server
cloud servers

1. Display last 10 lines


By default, the tail command prints the last 10 lines from the file.

> tail example.txt

2. Display last N lines


Use the -n option to print the last n lines from the file. The following example prints the last 2 lines from the file:

> tail -n2 example.txt
dedicated hosting server
cloud servers

3. Print lines from the Nth line


You can print lines starting from the Nth line in a file. The following example prints lines from the 2nd line.

> tail -n+2 example.txt
oracle virtual instance
mysql backup
dedicated hosting server
cloud servers

4. Print the last n bytes.


use the -c option to print the last N bytes from the file. The following example prints the last 8 bytes from the file.

> tail -c8 example.txt
servers

5. Print characters from the Nth byte.


Use the leading "+" with -c option to print the characters from the Nth byte. The following example prints the characters from the 79th byte.

> tail -c+79 example.txt
cloud servers

6. Print last lines from dynamically changing file.


The -f option print the lines from file that is growing dynamically. When you run the tail -f filename command, it prints the last 10 lines and waits for new lines to be added to the file. Whenever the new lines are appended to the file, the tail command also appends the new lines on the standard output. The -f option is useful when debugging applications. In general, the applications writes error messages to log files. You can use the -f option to check for the error messages as and when they appear in the log file.

> tail -f logfile

Wednesday, 20 June 2018

Hostname Command Examples in Unix / Linux

Hostname Command, Linux Command, Unix Command, LPI Study Materials

Hostname is the name of the system or server you are logged into. The hostname can also refer to the sitename or computer name. As an example, if an organization domain name is "google.com" and a specific computer name in that doman is "unix-box", then the hostname of the computer is "unix-box.google.com".

The syntax of hostname command in unix or linux system is

hostname [options] [file]

The options of hostname command are:

-a : Prints the alisa name of the host if created any.
-d : prints the domain name
-i : prints the ip address of the host
-s : prints the shortname of the host.
-v : verbose data
-V : version information
-h : help about hostname command

Hostname Command Examples: 


1. Print the hostname of the system The basic functionality of the hostname command is to display the name of the system on the terminal. Just type the hostname on the unix terminal and press enter to print the hostname.

> hostname
unix-box.google.com

2. Ip address of the computer You can find the ip address of the computer by using the -i option with hostname command.

> hostname -i
125.20.223.69

3. Print the domain name To know the domain name where the computer resides, use the -d option with hostname command.

> hostname -d
google.com

4. Short hostname By default the hostname command prints the complete name of the computer. You can print a short name by using the -s option. This prints the name upto the first dot in the full hostname.

> hostname -s
unix-box

5. Getting help To get help about the hostanme command either use the man command or the -h option with hostname command.

> man hostname
> hostname -h

Saturday, 16 June 2018

Paste Command Examples in Unix / Linux

Unix Command, Linux Command, Paste Command, LPI Certifications

Paste command is one of the useful commands in unix or linux operating system. The paste command merges the lines from multiple files. The paste command sequentially writes the corresponding lines from each file separated by a TAB delimiter on the unix terminal.

The syntax of the paste command is

paste [options] files-list

The options of paste command are:

-d : Specify of a list of delimiters.
-s : Paste one file at a time instead of in parallel.
--version : version information
--help : Help about the paste command.

Paste Command Examples:


Create the following three files in your unix or linux servers to practice to practice the examples:

> cat file1
Unix
Linux
Windows

> cat file2
Dedicated server
Virtual server

> cat file3
Hosting
Machine
Operating system

1. Merging files in parallel


By default, the paste command merges the files in parallel. The paste command writes corresponding lines from the files as a tab delimited on the terminal.

> paste file1 file2
Unix    Dedicated server
Linux   Virtual server
Windows

> paste file2 file1
Dedicated server  Unix
Virtual server    Linux
                  Windows

2. Specifying the delimiter


Paste command uses the tab delimiter by default for merging the files. You can change the delimiter to any other character by using the -d option.

> paste -d"|" file1 file2
Unix|Dedicated server
Linux|Virtual server
Windows|

In the above example, pipe delimiter is specified

3. Merging files in sequentially.


You can merge the files in sequentially using the -s option. The paste command reads each file in sequentially. It reads all the lines from a single file and merges all these lines into a single line.

> paste -s file1 file2
Unix    Linux   Windows
Dedicated server        Virtual server

The following example shows how to specify a delimiter for sequential merging of files:

> paste -s -d"," file1 file2
Unix,Linux,Windows
Dedicated server,Virtual server

4. Specifying multiple delimiters.


Multiple delimiters come in handy when you want to merge more than two files with different delimiters. For example I want to merge file1, file2 with pipe delimiter and file2, file3 with comma delimiter. In this case multiple delimiters will be helpful.

> paste -d"|," file1 file2 file3
Unix|Dedicated server,Hosting
Linux|Virtual server,Machine
Windows|,Operating system

5. Combining N consecutive lines


The paste command can also be used to merge N consecutive lines from a file into a single line. The following example merges 2 consecutive lines into a single line

> cat file1 | paste - -
Unix    Linux
Windows

Friday, 15 June 2018

Objectives: LPIC-3 Exam 304 - Virtualization

LPIC-3 Certifications, LPIC-3, LPI Certifications, LPI Guides, LPI Study Materials

LPIC-3 Exam 304: Virtualization


Exam Objectives Version: Version 2.0

Exam Code: 304-200

About Objective Weights: Each objective is assigned a weighting value. The weights indicate the relative importance of each objective on the exam. Objectives with higher weights will be covered in the exam with more questions.

Topic 330: Virtualization


330.1 Virtualization Concepts and Theory

Weight: 8

Description: Candidates should know and understand the general concepts, theory and terminology of Virtualization. This includes Xen, KVM and libvirt terminology.

Key Knowledge Areas:

◈ Terminology
◈ Pros and Cons of Virtualization
◈ Variations of Virtual Machine Monitors
◈ Migration of Physical to Virtual Machines
◈ Migration of Virtual Machines between Host systems
◈ Cloud Computing

The following is a partial list of the used files, terms and utilities:

◈ Hypervisor
◈ Hardware Virtual Machine (HVM)
◈ Paravirtualization (PV)
◈ Container Virtualization
◈ Emulation and Simulation
◈ CPU flags
◈ /proc/cpuinfo
◈ Migration (P2V, V2V)
◈ IaaS, PaaS, SaaS

330.2 Xen

Weight: 9

Description: Candidates should be able to install, configure, maintain, migrate and troubleshoot Xen installations. The focus is on Xen version 4.x.

Key Knowledge Areas:

◈ Xen architecture, networking and storage
◈ Xen configuration
◈ Xen utilities
◈ Troubleshooting Xen installations
◈ Basic knowledge of XAPI
◈ Awareness of XenStore
◈ Awareness of Xen Boot Parameters
◈ Awareness of the xm utility

Terms and Utilities:

◈ Domain0 (Dom0), DomainU (DomU)
◈ PV-DomU, HVM-DomU
◈ /etc/xen/
◈ xl
◈ xl.cfg
◈ xl.conf
◈ xe
◈ xentop

330.3 KVM

Weight: 9

Description: Candidates should be able to install, configure, maintain, migrate and troubleshoot KVM installations.

Key Knowledge Areas:

◈ KVM architecture, networking and storage
◈ KVM configuration
◈ KVM utilities
◈ Troubleshooting KVM installations

Terms and Utilities:

◈ Kernel modules: kvm, kvm-intel and kvm-amd
◈ /etc/kvm/
◈ /dev/kvm
◈ kvm
◈ KVM monitor
◈ qemu
◈ qemu-img

330.4 Other Virtualization Solutions

Weight: 3

Description: Candidates should have some basic knowledge and experience with alternatives to Xen and KVM.

Key Knowledge Areas:

◈ Basic knowledge of OpenVZ and LXC
◈ Awareness of other virtualization technologies
◈ Basic knowledge of virtualization provisioning tools

Terms and Utilities:

◈ OpenVZ
◈ VirtualBox
◈ LXC
◈ docker
◈ packer
◈ vagrant

330.5 Libvirt and Related Tools

Weight: 5

Description: Candidates should have basic knowledge and experience with the libvirt library and commonly available tools.

Key Knowledge Areas:

◈ libvirt architecture, networking and storage
◈ Basic technical knowledge of libvirt and virsh
◈ Awareness of oVirt

Terms and Utilities:

◈ libvirtd
◈ /etc/libvirt/
◈ virsh
◈ oVirt

330.6 Cloud Management Tools

Weight: 2

Description: Candidates should have basic feature knowledge of commonly available cloud management tools.

Key Knowledge Areas:

◈ Basic feature knowledge of OpenStack and CloudStack
◈ Awareness of Eucalyptus and OpenNebula

Terms and Utilities:

◈ OpenStack
◈ CloudStack
◈ Eucalyptus
◈ OpenNebula

Topic 334: High Availability Cluster Management


334.1 High Availability Concepts and Theory

Weight: 5

Description: Candidates should understand the properties and design approaches of high availability clusters.

Key Knowledge Areas:

◈ Understand the most important cluster architectures
◈ Understand recovery and cluster reorganization mechanisms
◈ Design an appropriate cluster architecture for a given purpose
◈ Application aspects of high availability
◈ Operational considerations of high availability

Terms and Utilities:

◈ Active/Passive Cluster, Active/Active Cluster
◈ Failover Cluster, Load Balanced Cluster
◈ Shared-Nothing Cluster, Shared-Disk Cluster
◈ Cluster resources
◈ Cluster services
◈ Quorum
◈ Fencing
◈ Split brain
◈ Redundancy
◈ Mean Time Before Failure (MTBF)
◈ Mean Time To Repair (MTTR)
◈ Service Level Agreement (SLA)
◈ Disaster Recovery
◈ Replication
◈ Session handling

334.2 Load Balanced Clusters

Weight: 6

Description: Candidates should know how to install, configure, maintain and troubleshoot LVS. This includes the configuration and use of keepalived and ldirectord. Candidates should further be able to install, configure, maintain and troubleshoot HAProxy.

Key Knowledge Areas:

◈ Understanding of LVS / IPVS
◈ Basic knowledge of VRRP
◈ Configuration of keepalived
◈ Configuration of ldirectord
◈ Backend server network configuration
◈ Understanding of HAProxy
◈ Configuration of HAProxy

Terms and Utilities:

◈ ipvsadm
◈ syncd
◈ LVS Forwarding (NAT, Direct Routing, Tunneling, Local Node)
◈ connection scheduling algorithms
◈ keepalived configuration file
◈ ldirectord configuration file
◈ genhash
◈ HAProxy configuration file
◈ load balancing algorithms
◈ ACLs

334.3 Failover Clusters

Weight: 6

Description: Candidates should have experience in the installation, configuration, maintenance and troubleshooting of a Pacemaker cluster. This includes the use of Corosync. The focus is on Pacemaker 1.1 for Corosync 2.x.

Key Knowledge Areas:

◈ Pacemaker architecture and components (CIB, CRMd, PEngine, LRMd, DC, STONITHd)
◈ Pacemaker cluster configuration
◈ Resource classes (OCF, LSB, Systemd, Upstart, Service, STONITH, Nagios)
◈ Resource rules and constraints (location, order, colocation)
◈ Advanced resource features (templates, groups, clone resources, multi-state resources)
◈ Pacemaker management using pcs
◈ Pacemaker management using crmsh
◈ Configuration and Management of corosync in conjunction with Pacemaker
◈ Awareness of other cluster engines (OpenAIS, Heartbeat, CMAN)

Terms and Utilities:

◈ pcs
◈ crm
◈ crm_mon
◈ crm_verify
◈ crm_simulate
◈ crm_shadow
◈ crm_resource
◈ crm_attribute
◈ crm_node
◈ crm_standby
◈ cibadmin
◈ corosync.conf
◈ authkey
◈ corosync-cfgtool
◈ corosync-cmapctl
◈ corosync-quorumtool
◈ stonith_admin

334.4 High Availability in Enterprise Linux Distributions

Weight: 1

Description: Candidates should be aware of how enterprise Linux distributions integrate High Availability technologies.

Key Knowledge Areas:

◈ Basic knowledge of Red Hat Enterprise Linux High Availability Add-On
◈ Basic knowledge of SUSE Linux Enterprise High Availability Extension

Terms and Utilities:

◈ Distribution specific configuration tools
◈ Integration of cluster engines, load balancers, storage technology, cluster filesystems, etc.

Topic 335: High Availability Cluster Storage


335.1 DRBD / cLVM

Weight: 3

Description: Candidates are expected to have the experience and knowledge to install, configure, maintain and troubleshoot DRBD devices. This includes integration with Pacemaker. DRBD configuration of version 8.4.x is covered. Candidates are further expected to be able to manage LVM configuration within a shared storage cluster.

Key Knowledge Areas:

◈ Understanding of DRBD resources, states and replication modes
◈ Configuration of DRBD resources, networking, disks and devices
◈ Configuration of DRBD automatic recovery and error handling
◈ Management of DRBD using drbdadm
◈ Basic knowledge of drbdsetup and drbdmeta
◈ Integration of DRBD with Pacemaker
◈ cLVM
◈ Integration of cLVM with Pacemaker

Terms and Utilities:

◈ Protocol A, B and C
◈ Primary, Secondary
◈ Three-way replication
◈ drbd kernel module
◈ drbdadm
◈ drbdsetup
◈ drbdmeta
◈ /etc/drbd.conf
◈ /proc/drbd
◈ LVM2
◈ clvmd
◈ vgchange, vgs

335.2 Clustered File Systems

Weight: 3

Description: Candidates should know how to install, maintain and troubleshoot installations using GFS2 and OCFS2. This includes integration with Pacemaker as well as awareness of other clustered filesystems available in a Linux environment.

Key Knowledge Areas:

◈ Understand the principles of cluster file systems
◈ Create, maintain and troubleshoot GFS2 file systems in a cluster
◈ Create, maintain and troubleshoot OCFS2 file systems in a cluster
◈ Integration of GFS2 and OCFS2 with Pacemaker
◈ Awareness of the O2CB cluster stack
◈ Awareness of other commonly used clustered file systems

Terms and Utilities:

◈ Distributed Lock Manager (DLM)
◈ mkfs.gfs2
◈ mount.gfs2
◈ fsck.gfs2
◈ gfs2_grow
◈ gfs2_edit
◈ gfs2_jadd
◈ mkfs.ocfs2
◈ mount.ocfs2
◈ fsck.ocfs2
◈ tunefs.ocfs2
◈ mounted.ocfs2
◈ o2info
◈ o2image
◈ CephFS
◈ GlusterFS
◈ AFS

Tuesday, 12 June 2018

FTP (File Transfer Protocol) Command Examples

FTP Command, FTP Learning, LPI Guides, LPI Study Materials, LPI Tutorials and Materials

The FTP (file transfer program) utility is used to transfer files between a local machine and remote network machine Using the File Transfer protocol. In simple terms it transfers / copies files between two computers. You can transfer files between unix systems and also non-unix systems like windows operating system using FTP.

The FTP command is simple to use and easy to learn. Let see useful examples of FTP command in detail.

FTP Command Examples:


If you are using windows operating system, open the command prompt and practice the below FTP commands. If you are using unix or linux operating systems, just simply type the ftp command on the terminal.

1. Connecting to Remote Host


First you need to connect to a remote host before doing any operations. You can use any one of the following methods to connect to a remote host. First method is

> ftp remote-server-name
connected to remote-server-name
User-Name:
Password:
ftp>

Once the ftp connects to the remote server name, it will prompt you to enter the user name and password. After successful login, your terminal or prompt changes to "ftp>".

Another method is to use the open option with ftp command. This is shown below:

>ftp
ftp>open remote-server-name
connected to remote-server-name
User-Name:
Password:
ftp>

If the ftp command fails to connect to the remote server, then you will get the below error:

ftp: connect: Connection refused

2. Copy file from remote machine to local machine.


The get option is used to download or transfer a file from the remote system to the local system.

ftp> get windows-cleveland.bat

This will download the specified file (windows-cleveland.bat) from the remote systems current directory.

3. Copying multiple files from remote machine to local machine.


You can use the mget to transfer multiple files from the remote host to local host.

ftp>mget *.png

This will download all the png images to the local machine.

4. Transferring file from local server to remote server


The put option is used to copy the file from the local host to the remote host.

ftp>put linux-virtual-server.rpm

This command puts the rpm file into the remote machine.

5. Transferring multiple files to the remote server.


You can use the mput option to transfer more than one file from local system to the remote system.

ftp>put *.rpm

6. Executing commands in remote machine.


After connecting to the remote network machine using the ftp, you can run commands like ls to list the files, cd to change directory and many more.

ftp> ls

This will list the files and directories in the remote machines current directory.

7. Executing commands in local machine.


Once you have connected to the remote host, to run the commands on local machine you need to exit from the ftp connection. Instead of this, there is a way to run commands on local host without exiting from the ftp connection. Use the ! symbol before the command you want to run.

ftp> !ls

Now this will list the files in the local machines current directory.

8. Changing the file transferring mode.


You can change the file transfer modes to ascii and binary modes. Use the below commands to change the mode.

ftp>ascii
ftp>binary

9. Deleting files on remote machine


You can use the delete or mdelete to remove a single file or multiple files in the remote machine.

ftp>delete linux-dedicated-server.dat
ftp>mdelete *.dat

10. Disconnecting from ftp connection.


Use the quit command to close the ftp connection.

ftp>quit

11. Using FTP command in batch scripts


The following script reads the instructions from the dat file and executes them on the remote machine.

echo "Ftp command batch script"
echo "start"
ftp -s:instructions.dat remote-host
echo "End"

The contents of the instructions.dat file is

FTP Command, FTP Learning, LPI Guides, LPI Study Materials, LPI Tutorials and Materials

user
password
cd /var/tmp
put oracle_storage.exe
quit

12. Getting the help about ftp command.


To know more about the ftp command, just type the help on the prompt. It will display the options/commands that you can use with ftp command.

ftp>help
Commands may be abbreviated.  Commands are:

!       disconnect mdelete  preserve  runique
$       edit       mdir     progress  send
account exit       mget     prompt    sendport
append  form       mkdir    proxy     site
ascii   ftp        mls      put       size
bell    get        mode     pwd       sndbuf
binary  gate       modtime  quit      status
bye     glob       more     quote     struct
case    hash       mput     rcvbuf    sunique
cd      help       msend    recv      system
cdup    idle       newer    reget     tenex
chmod   image      nlist    rename    trace
close   lcd        nmap     reset     type
cr      less       ntrans   restart   umask
debug   lpwd       open     rhelp     user
delete  ls         page     rmdir     verbose
dir     macdef     passive  rstatus   ?

Saturday, 9 June 2018

Sort Command Examples in Unix / Linux

Sort Command, Linux Command, Unix Command, LPI Certifications

Sort command in unix or linux system is used to order the elements or text. Sort command has the capability of sorting numerical values and strings. The sort command can order the lines in a text file.

The syntax of sort command is:


sort [options] filename

The options are:

-b : Ignores leading spaces in each line
-d : Uses dictionary sort order. Conisders only spaces and alphanumeric characters in sorting
-f : Uses case insensitive sorting.
-M : Sorts based on months. Considers only first 3 letters as month. Eg: JAN, FEB
-n : Uses numeric sorting
-R : Sorts the input file randomly.
-r : Reverse order sorting
-k : Sorts file based on the data in the specified field positions.
-u : Suppresses duplicate lines
-t : input field separator

Sort Command Examples:


Before practicing the examples create the below two files in your unix system:

> cat order.txt
Unix distributed 05 server
Linux virtual 3 server
Unix distributed 05 server
Distributed processing 6 system

> cat delim_sort.txt
Mayday|4
Janmon|1
Declast|12

1. Sorting lines of text


The default sort command uses alphabetical order (ASCII order) to sort the file. It treats each line as a string and then sorts the lines.

> sort order.txt
Distributed processing 6 system
Linux virtual 3 server
Unix distributed 05 server
Unix distributed 05 server

2. Sorting based on the field positions.


You can specify the field postions using the -k option of sort command. The sort command uses the space or tab as the default delimiter. To sort based on the data in the second field, run the below command:

> sort -k2 order.txt
Unix distributed 05 server
Unix distributed 05 server
Distributed processing 6 system
Linux virtual 3 server

You can also pecify more than field with k option as a comma separated list. The below command uses the second and fourth fields to sort the data.

> sort -k2,4 order.txt

3. Numeric sorting


Instead of the default alphabetical sorting order, you can make the sort command to sort in numeric order using the -n option. This is shown below:

> sort -nk3 order.txt
Linux virtual 3 server
Unix distributed 05 server
Unix distributed 05 server
Distributed processing 6 system

4. Sort in reverse order


By default, the sort command sorts the data in ascending order. You can change this to descending order using the -r option.

> sort -nrk3 order.txt
Distributed processing 6 system
Unix distributed 05 server
Unix distributed 05 server
Linux virtual 3 server

5. Suppressing duplicates or Print only unique values


You can produce only unique values in the output using the - u option of the sort command.

> sort -u order.txt
Distributed processing 6 system
Linux virtual 3 server
Unix distributed 05 server

Another way is piping the output of sort command to uniq command.

> sort order.txt | uniq

6. Delimited file input


In the second, third and fourth examples we have sorted the data based on the field positions. Here the fields are separted by space or tab character. What if the fields are specifed by any other character? In such cases, we have to specify the input delimiter with the -t option. An example is shown below:

> sort -t'|' -nrk2 delim_sort.txt
Declast|12
Mayday|4
Janmon|1

7. Sorting on months.


We can sort the data in the monthwise using the -M option of the sort command. This is shown below:

> sort -M delim_sort.txt
Janmon|1
Mayday|4
Declast|12

Treats the first 3 characters in the string as month and then sorts in months order.

Thursday, 7 June 2018

Zip Command Examples in Unix / Linux

Zip Command, Linux Certification, Unix Certification, Linux LPI

zip is used to compress the files to reduce file size and also used as file package utility. zip is available in many operating systems like unix, linux, windows etc.

If you have a limited bandwidth between two servers and want to transfer the files faster, then zip the files and transfer.

The syntax of zip command is

zip [options] zipfile files_list

The options of zip command are:

-d : Removes the file from the zip archive
-u : Updates the file in the zip archive
-m : Deletes the original files after zipping.
-r : Recursively zips the files in a directory
-x : Exclude the files in creating the zip
-v : verbose mode
-1 : Compresses the files faster
-9 : Compresses the files better
-f : freshen only changed files.
zipfile : creates the zip file with name as zipfile.zip
files_list : list of files to be zipped.

Zip Command Examples:


The files in my current directory are listed below:

docs/linux.pdf
docs/oracle.pdf
docs/unix.pdf
linux-virtual-server.bat
unix-server.dat

Here docs is a directory which contains the files linux.pdf, unix.pdf and oracle.pdf. We will see how to use zip command with examples.

1. Creating a zip file


The zip command in unix or linux system creates an archive with the specified files. This is shown below:

> zip archive linux-virtual-server.bat unix-server.dat
  adding: linux-virtual-server.bat (deflated 80%)
  adding: unix-server.dat (deflated 80%)
> ls
archive.zip  docs  linux-virtual-server.bat  unix-server.dat

The above command creates the zip file with name archive.zip

2. Extracting files from zip


To extract files from the zip, use the unzip command in unix system. This is shown below:

> unzip archive.zip
Archive:  archive.zip
  inflating: linux-virtual-server.bat
  inflating: unix-server.dat
> ls
archive.zip  linux-virtual-server.bat  unix-server.dat

3. Removing file from a zip file


After creating a zip file, you can remove a file from the archive using the -d option. To remove the file unix-server.dat from the archive, run the below zip command:

> zip -d archive.zip unix-server.dat
deleting: unix-server.dat

> unzip archive.zip
Archive:  archive.zip
  inflating: linux-virtual-server.bat

4. Update existing zip file


You can update the files in already created zip file. If any of the files are modified after zipping, you can fresh the zip file with only those modified files using the -f option.

> zip -f archive.zip
freshening: linux-virtual-server.bat (stored 0%)

Another way is using the -u option. This option can be used to update the specified list of files or add new files to the existing zip file.

> zip -u archive.zip  linux-virtual-server.bat temp
updating: linux-virtual-server.bat (deflated 79%)
  adding: temp (stored 0%)

5. Recursively zip files in directory.


To zip a directory recursively, use the -r option with the zip command. This example is shown below:

> zip -r dir_archive docs
  adding: docs/ (stored 0%)
  adding: docs/unix.pdf (stored 0%)
  adding: docs/oracle.pdf (stored 0%)
  adding: docs/linux.pdf (stored 0%)

6. Excluding files in zipping


Let say you are zipping all the files in the current directory and want to exclude some unwanted files. You can exclude these unwanted files using the -x option.

zip exclude_archive * -x linux-virtual-server.bat

The above command zips all the files in the current directory except the file linux-virtual-server.bat

7. Faster compressing


You can compress the files very fast using the -1 option with zip command. An example is shown below with and without using fast compression.

> zip -1 fast_archive linux-virtual-server.bat
  adding: linux-virtual-server.bat (deflated 79%)

>zip normal_archive linux-virtual-server.bat
  adding: linux-virtual-server.bat (deflated 80%)

If you use fast compression, the archive file created will occupy more space (size) when compared to normal compression.

8. Better compression.


To reduce more amount of size the files occupied, you can use the -9 option with the zip command. This gives a better compression.

> zip -9 better_archive linux-virtual-server.bat
  adding: linux-virtual-server.bat (deflated 81%)

Compare the deflated percentages in the example 7 and 8.

Tuesday, 5 June 2018

LPIC-3 Exam 303: Security (Objectives)

LPIC-3 Certifications, LPIC-3 Exam, LPIC-3 Security, LPIC-3 Tips

Exam Objectives Version: Version 2.0

Exam Code: 303-200

About Objective Weights: Each objective is assigned a weighting value. The weights indicate the relative importance of each objective on the exam. Objectives with higher weights will be covered in the exam with more questions.

LPIC-3 Exam 303: Security


Topic 325: Cryptography


325.1 X.509 Certificates and Public Key Infrastructures

Weight: 5

Description: Candidates should understand X.509 certificates and public key infrastructures. They should know how to configure and use OpenSSL to implement certification authorities and issue SSL certificates for various purposes.

Key Knowledge Areas:

◈ Understand X.509 certificates, X.509 certificate lifecycle, X.509 certificate fields and X.509v3 certificate extensions
◈ Understand trust chains and public key infrastructures
◈ Generate and manage public and private keys
◈ Create, operate and secure a certification authority
◈ Request, sign and manage server and client certificates
◈ Revoke certificates and certification authorities

The following is a partial list of the used files, terms and utilities:

◈ openssl, including relevant subcommands
◈ OpenSSL configuration
◈ PEM, DER, PKCS
◈ CSR
◈ CRL
◈ OCSP

325.2 X.509 Certificates for Encryption, Signing and Authentication

Weight: 4

Description: Candidates should know how to use X.509 certificates for both server and client authentication. Candidates should be able to implement user and server authentication for Apache HTTPD. The version of Apache HTTPD covered is 2.4 or higher.

Key Knowledge Areas:

◈ Understand SSL, TLS and protocol versions
◈ Understand common transport layer security threats, for example Man-in-the-Middle
◈ Configure Apache HTTPD with mod_ssl to provide HTTPS service, including SNI and HSTS
◈ Configure Apache HTTPD with mod_ssl to authenticate users using certificates
◈ Configure Apache HTTPD with mod_ssl to provide OCSP stapling
◈ Use OpenSSL for SSL/TLS client and server tests

Terms and Utilities:

◈ Intermediate certification authorities
◈ Cipher configuration (no cipher-specific knowledge)
◈ httpd.conf
◈ mod_ssl
◈ openssl

325.3 Encrypted File Systems

Weight: 3

Description: Candidates should be able to setup and configure encrypted file systems.

Key Knowledge Areas:

◈ Understand block device and file system encryption
◈ Use dm-crypt with LUKS to encrypt block devices
◈ Use eCryptfs to encrypt file systems, including home directories and
◈ PAM integration
◈ Be aware of plain dm-crypt and EncFS

Terms and Utilities:

◈ cryptsetup
◈ cryptmount
◈ /etc/crypttab
◈ ecryptfsd
◈ ecryptfs-* commands
◈ mount.ecryptfs, umount.ecryptfs
◈ pam_ecryptfs

325.4 DNS and Cryptography

Weight: 5

Description: Candidates should have experience and knowledge of cryptography in the context of DNS and its implementation using BIND. The version of BIND covered is 9.7 or higher.

Key Knowledge Areas:

◈ Understanding of DNSSEC and DANE
◈ Configure and troubleshoot BIND as an authoritative name server serving DNSSEC secured zones
◈ Configure BIND as an recursive name server that performs DNSSEC validation on behalf of its clients
◈ Key Signing Key, Zone Signing Key, Key Tag
◈ Key generation, key storage, key management and key rollover
◈ Maintenance and re-signing of zones
◈ Use DANE to publish X.509 certificate information in DNS
◈ Use TSIG for secure communication with BIND

Terms and Utilities:

◈ DNS, EDNS, Zones, Resource Records
◈ DNS resource records: DS, DNSKEY, RRSIG, NSEC, NSEC3, NSEC3PARAM, TLSA
◈ DO-Bit, AD-Bit
◈ TSIG
◈ named.conf
◈ dnssec-keygen
◈ dnssec-signzone
◈ dnssec-settime
◈ dnssec-dsfromkey
◈ rndc
◈ dig
◈ delv
◈ openssl

Topic 326: Host Security


326.1 Host Hardening

Weight: 3

Description: Candidates should be able to secure computers running Linux against common threats. This includes kernel and software configuration.

Key Knowledge Areas:

◈ Configure BIOS and boot loader (GRUB 2) security
◈ Disable useless software and services
◈ Use sysctl for security related kernel configuration, particularly ASLR, Exec-Shield and IP / ICMP configuration
◈ Exec-Shield and IP / ICMP configuration
◈ Limit resource usage
◈ Work with chroot environments
◈ Drop unnecessary capabilities
◈ Be aware of the security advantages of virtualization

Terms and Utilities:

◈ grub.cfg
◈ chkconfig, systemctl
◈ ulimit
◈ /etc/security/limits.conf
◈ pam_limits.so
◈ chroot
◈ sysctl
◈ /etc/sysctl.conf

326.2 Host Intrusion Detection

Weight: 4

Description: Candidates should be familiar with the use and configuration of common host intrusion detection software. This includes updates and maintenance as well as automated host scans.

Key Knowledge Areas:

◈ Use and configure the Linux Audit system
◈ Use chkrootkit
◈ Use and configure rkhunter, including updates
◈ Use Linux Malware Detect
◈ Automate host scans using cron
◈ Configure and use AIDE, including rule management
◈ Be aware of OpenSCAP

Terms and Utilities:

◈ auditd
◈ auditctl
◈ ausearch, aureport
◈ auditd.conf
◈ auditd.rules
◈ pam_tty_audit.so
◈ chkrootkit
◈ rkhunter
◈ /etc/rkhunter.conf
◈ maldet
◈ conf.maldet
◈ aide
◈ /etc/aide/aide.conf

326.3 User Management and Authentication

Weight: 5

Description: Candidates should be familiar with management and authentication of user accounts. This includes configuration and use of NSS, PAM, SSSD and Kerberos for both local and remote directories and authentication mechanisms as well as enforcing a password policy.

Key Knowledge Areas:

◈ Understand and configure NSS
◈ Understand and configure PAM
◈ Enforce password complexity policies and periodic password changes
◈ Lock accounts automatically after failed login attempts
◈ Configure and use SSSD
◈ Configure NSS and PAM for use with SSSD
◈ Configure SSSD authentication against Active Directory, IPA, LDAP, Kerberos and local domains
◈ Kerberos and local domains
◈ Obtain and manage Kerberos tickets

Terms and Utilities:

◈ nsswitch.conf
◈ /etc/login.defs
◈ pam_cracklib.so
◈ chage
◈ pam_tally.so, pam_tally2.so
◈ faillog
◈ pam_sss.so
◈ sssd
◈ sssd.conf
◈ sss_* commands
◈ krb5.conf
◈ kinit, klist, kdestroy

326.4 FreeIPA Installation and Samba Integration

Weight: 4

Description: Candidates should be familiar with FreeIPA v4.x. This includes installation and maintenance of a server instance with a FreeIPA domain as well as integration of FreeIPA with Active Directory.

Key Knowledge Areas:

◈ Understand FreeIPA, including its architecture and components
◈ Understand system and configuration prerequisites for installing FreeIPA
◈ Install and manage a FreeIPA server and domain
◈ Understand and configure Active Directory replication and Kerberos cross-realm trusts
◈ Be aware of sudo, autofs, SSH and SELinux integration in FreeIPA

Terms and Utilities:

◈ 389 Directory Server, MIT Kerberos, Dogtag Certificate System, NTP, DNS, SSSD, certmonger
◈ ipa, including relevant subcommands
◈ ipa-server-install, ipa-client-install, ipa-replica-install
◈ ipa-replica-prepare, ipa-replica-manage

Topic 327: Access Control


327.1 Discretionary Access Control

Weight: 3

Description: Candidates are required to understand Discretionary Access Control and know how to implement it using Access Control Lists. Additionally, candidates are required to understand and know how to use Extended Attributes.

Key Knowledge Areas:

◈ Understand and manage file ownership and permissions, including SUID and SGID
◈ Understand and manage access control lists
◈ Understand and manage extended attributes and attribute classes

Terms and Utilities:

◈ getfacl
◈ setfacl
◈ getfattr
◈ setfattr

327.2 Mandatory Access Control

Weight: 4

Description: Candidates should be familiar with Mandatory Access Control systems for Linux. Specifically, candidates should have a thorough knowledge of SELinux. Also, candidates should be aware of other Mandatory Access Control systems for Linux. This includes major features of these systems but not configuration and use.

Key Knowledge Areas:

◈ Understand the concepts of TE, RBAC, MAC and DAC
◈ Configure, manage and use SELinux
◈ Be aware of AppArmor and Smack

Terms and Utilities:

◈ getenforce, setenforce, selinuxenabled
◈ getsebool, setsebool, togglesebool
◈ fixfiles, restorecon, setfiles
◈ newrole, runcon
◈ semanage
◈ sestatus, seinfo
◈ apol
◈ seaudit, seaudit-report, audit2why, audit2allow
◈ /etc/selinux/*

327.3 Network File Systems

Weight: 3

Description: Candidates should have experience and knowledge of security issues in use and configuration of NFSv4 clients and servers as well as CIFS client services. Earlier versions of NFS are not required knowledge.

Key Knowledge Areas:

◈ Understand NFSv4 security issues and improvements
◈ Configure NFSv4 server and clients
◈ Understand and configure NFSv4 authentication mechanisms (LIPKEY, SPKM, Kerberos)
◈ Understand and use NFSv4 pseudo file system
◈ Understand and use NFSv4 ACLs
◈ Configure CIFS clients
◈ Understand and use CIFS Unix Extensions
◈ Understand and configure CIFS security modes (NTLM, Kerberos)
◈ Understand and manage mapping and handling of CIFS ACLs and SIDs in a Linux system

Terms and Utilities:

◈ /etc/exports
◈ /etc/idmap.conf
◈ nfs4acl
◈ mount.cifs parameters related to ownership, permissions and security modes
winbind
◈ getcifsacl, setcifsacl

Topic 328: Network Security


328.1 Network Hardening

Weight: 4

Description: Candidates should be able to secure networks against common threats. This includes verification of the effectiveness of security measures.

Key Knowledge Areas:

◈ Configure FreeRADIUS to authenticate network nodes
◈ Use nmap to scan networks and hosts, including different scan methods
◈ Use Wireshark to analyze network traffic, including filters and statistics
◈ Identify and deal with rogue router advertisements and DHCP messages

Terms and Utilities:

◈ radiusd
◈ radmin
◈ radtest, radclient
◈ radlast, radwho
◈ radiusd.conf
◈ /etc/raddb/*
◈ nmap
◈ wireshark
◈ tshark
◈ tcpdump
◈ ndpmon

328.2 Network Intrusion Detection

Weight: 4

Description: Candidates should be familiar with the use and configuration of network security scanning, network monitoring and network intrusion detection software. This includes updating and maintaining the security scanners.

Key Knowledge Areas:

◈ Implement bandwidth usage monitoring
◈ Configure and use Snort, including rule management
◈ Configure and use OpenVAS, including NASL

Terms and Utilities:

◈ ntop
◈ Cacti
◈ snort
◈ snort-stat
◈ /etc/snort/*
◈ openvas-adduser, openvas-rmuser
◈ openvas-nvt-sync
◈ openvassd
◈ openvas-mkcert
◈ /etc/openvas/*

328.3 Packet Filtering

Weight: 5

Description: Candidates should be familiar with the use and configuration of packet filters. This includes netfilter, iptables and ip6tables as well as basic knowledge of nftables, nft and ebtables.

Key Knowledge Areas:

◈ Understand common firewall architectures, including DMZ
◈ Understand and use netfilter, iptables and ip6tables, including standard modules, tests and targets
◈ Implement packet filtering for both IPv4 and IPv6
◈ Implement connection tracking and network address translation
◈ Define IP sets and use them in netfilter rules
◈ Have basic knowledge of nftables and nft
◈ Have basic knowledge of ebtables
◈ Be aware of conntrackd

Terms and Utilities:

◈ iptables
◈ ip6tables
◈ iptables-save, iptables-restore
◈ ip6tables-save, ip6tables-restore
◈ ipset
◈ nft
◈ ebtables

328.4 Virtual Private Networks

Weight: 4

Description: Candidates should be familiar with the use of OpenVPN and IPsec.

Key Knowledge Areas:

◈ Configure and operate OpenVPN server and clients for both bridged and routed VPN networks
◈ Configure and operate IPsec server and clients for routed VPN networks using IPsec-Tools / racoon
◈ Awareness of L2TP

Terms and Utilities:

◈ /etc/openvpn/*
◈ openvpn server and client
◈ setkey
◈ /etc/ipsec-tools.conf
◈ /etc/racoon/racoon.conf