The Linux pwd command (and Linux command line prompts)

Linux current directory FAQ: How do I show what directory I'm currently in on a Unix or Linux system?

Just use the Unix/Linux pwd command, like this:

pwd

If you're in a directory named /home/al, the output looks like this:

$ pwd
/home/al

If you're in a directory named /tmp, the output looks like this:

$ pwd
/tmp

That's all there is to the pwd command.

Putting the current directory in your Linux prompt

In the old days we had to type this command all the time, but in these more modern times we can just display the current directory in our prompt symbol by setting it like this:

PS1='$PWD> '

Note that it's very important to use single quotes in this command and not double quotes.

Myself, I prefer a longer, multiline Linux prompt, so I put this PS1 setting in my ~/.bash_profile file on my Unix and Linux systems:

PS1='
$PWD
==> '

This gives me a prompt like this, which I prefer, especially with longer directories:

/Users/al/foo/bar/baz
==> _

Continue reading

Linux cd command examples

Linux FAQ: Can you share some cd command examples?

The Linux cd command stands for "change directory". It is the primary command for moving between directories on a Unix/Linux filesystem.

Examples of the cd command

This first command moves you to the /usr directory. The /usr directory becomes your current working directory:

cd /usr

Similarly, this cd command moves you to the /tmp directory:

cd /tmp

You don't have to move just one directory at a time though. You can easily jump from one directory to another directory that's far, far away, like this:

cd /var/www/html/unix/edu

Linux cd command: Using wildcards

You can also use wildcard characters when moving between directories. This next command works just like the previous command, taking me to the /var/www/html/unix/edu directory on my Linux system:

cd /v*/w*/h*/u*/e*

The * wildcard characters can be interpreted as "any number of any character", so the Linux system expands the /v*/w*/h*/u*/e* that I typed into /var/www/html/unix/edu.

Linux cd command and auto-complete

An even better way to cd to that same directory is to use the auto-complete functionality in the Bash shell of all modern Linux systems. Using auto-complete, you can hit the [Tab] key any time you might normally use the asterisk, so I could move to that previous directory using the following keystrokes:

cd /v[Tab]/w[Tab]/h[Tab]/u[Tab]/e[Tab]

That doesn't look great in text, but trust me, you'll come to love the Bash auto-complete functionality. (As they would say on the tv show Monk, "You'll thank me later.")

The cd command: Going home

On all Unix and Linux systems that I'm aware of, you can always get back to your home directory by typing the cd command without any arguments, like this:

cd

As far as I know, that works with Bash, the Korn shell, the C shell, etc.

cd command: Back to the previous directory

Again, with every Unix and Linux shell I've worked with, you can also always move back to your previous directory by adding a hyphen after the cd command, like this:

cd -

This is a great feature for when you're moving back and forth between two different directories while working on a project.

Continue reading

The Linux copy command (cp)

Linux file copy FAQ: How do I copy Linux files and directories? (Or, Can you share some cp command examples?)

You use the cp command to copy files and directories on Linux systems. Let's look at some copy examples to see how this works.

Using Linux cp to copy files

At its most basic, here's how you copy a Linux file:

cp Chapter1 Chapter1.bak

This cp command copies the original file named Chapter1 to the new file named Chapter1.bak. After issuing this command both your original file and the new file will be in the current directory.

It's also easy to copy a Linux file to a different directory. Here's how you copy the same file to the /tmp directory:

cp Chapter1 /tmp

and here's how you copy a file one level up in the directory hierarchy:

cp Chapter1 ..

As a final note about copying files, you can also copy multiple files at one time, like this:

cp file1 file2 file3 /tmp

That command copies those three files to the /tmp directory.

How to copy directories with the Linux cp command

You can also copy directories with the Linux cp command, and when you do this, you just need to remember to use the -r option. For example, this command copies the directory named Foo to the /tmp directory:

cp -r Foo /tmp

If you try to copy a directory without using the -r argument you'll get an error like this:

$ cp Foo /tmp
cp: Foo is a directory (not copied).

so it's not too hard to remember to use that option.

Continue reading

The Linux more command

The Linux more command lets you view text files or other output in a scrollable manner. It displays the text one screenful at a time, and lets you scroll backwards and forwards through the text, and even lets you search the text.

Looking at a Linux file with the more command

A common way to use the Linux more command is to display the contents of a text file. Where you might normally "cat out" the contents of a text file with the cat command, like this:

cat /etc/passwd

a problem with this approach is that when the file is long, all the output scrolls off the top of your screen. The Linux more command solves this problem by letting you scroll the output one screenful of data at a time. For this use, just use the more command instead of the cat command, like this:

more /etc/passwd

Now you can see the file contents on screen, and you can also scroll through the file output. On modern Linux systems you can use the [UpArrow] and [DownArrow] keys to scroll through the display. You can also use these keys to move through the output:

◉ [Space] - scrolls the display, one screenful of data at a time
◉ [Enter] - scrolls the display one line
◉ [b] - scrolls the display backwards one screenful of data
◉ [/] - lets you search the text, just like you would in the vi/vim editor

Use the Linux more command in a pipeline

It's also very common to use the Linux more command in a command pipeline. For instance, let's say you want to look at a list of system processes, but don't want them to all scroll off the top of your screen. In this case you use the more command with the ps command, like this:

ps auxwww | more

You can use the more command at the end of any Linux command pipeline, for example, something like this:

grep 'foo' myfile.txt | grep bar | more

The 'less' command

Most Unix and Linux systems also now include a less command. It works similarly to the more command, but has a few improved options. Check it out, or read the man (manual) pages for both commands to find out which one you prefer.

Continue reading

Examples of the Unix mkdir command

Linux directory FAQ: How do I create (make) a directory on Linux or Unix?

The Unix/Linux mkdir command is used to create new Unix/Linux directories (sub-directories). Let's take a look at some mkdir command examples.

How to create one directory

This first example creates a new directory named tmp in your current directory:

mkdir tmp

This example assumes that you have the proper permissions to create a new sub-directory in your current working directory.

Linux mkdir example - How to create multiple directories at one time

This command creates three new sub-directories (memos, letters, and e-mail) in the current directory:

mkdir memos letters e-mail

Linux mkdir example - How to create several subdirectories at one time

Use the -p option of the mkdir command to create multiple levels of subdirectories with one command. This example creates the directory /home/joe/customer/acme/foo/bar, and makes all intermediate subdirectories, as needed:

mkdir -p /home/joe/customer/acme/foo/bar

As you can imagine, that's a lot easier than typing these equivalent commands:

cd /home/joe
mkdir customer

cd customer
mkdir acme

cd acme
mkdir foo

cd foo
mkdir bar

Linux mkdir command: "Permission denied" errors

As a final note, if you try to create a directory like this:

mkdir baz

and you get an error message like this:

mkdir: cannot create directory 'baz': Permission denied

as the message implies, you don't have permission to create this directory. You can use the ls command to figure out what permission you have in this directory.

Continue reading

tar command in Linux with examples

The Linux ‘tar’ stands for tape archive, is used to create Archive and extract the Archive files. tar command in Linux is one of the important command which provides archiving functionality in Linux. We can use Linux tar command to create compressed or uncompressed Archive files and also maintain and modify them.

Syntax:

tar [options] [archive-file] [file or directory to be archived]

Options:

-c : Creates Archive
-x : Extract the archive
-f : creates archive with given filename
-t : displays or lists files in archived file
-u : archives and adds to an existing archive file
-v : Displays Verbose Information
-A : Concatenates the archive files
-z : zip, tells tar command that create tar file using gzip
-j : filter archive tar file using tbzip
-W : Verify a archive file
-r : update or add file or directory in already existed .tar file

What is an Archive file?

An Archive file is a file that is composed of one or more files along with metadata. Archive files are used to collect multiple data files together into a single file for easier portability and storage, or simply to compress files to use less storage space.

Examples:

1. Creating an uncompressed tar Archive using option -cvf: This command creates a tar file called file.tar which is the Archive of all .c files in current directory.

$ tar cvf file.tar *.c

Output :

os2.c
os3.c
os4.c

2. Extracting files from Archive using option -xvf: This command extracts files from Archives.

$ tar xvf file.tar

Output :

os2.c
os3.c
os4.c

3. gzip compression on the tar Archive, using option -z: This command creates a tar file called file.tar.gz which is the Archive of .c files.

$ tar cvzf file.tar.gz *.c

4. Extracting a gzip tar Archive *.tar.gz using option -xvzf: This command extracts files from tar archived file.tar.gz files.

$ tar xvzf file.tar.gz

5. Creating compressed tar archive file in Linux using option -j: This command compresses and creates archive file less than the size of the gzip. Both compress and decompress takes more time then gzip.

$ tar cvfj file.tar.tbz example.cpp

Output :

$tar cvfj file.tar.tbz example.cpp
example.cpp
$tar tvf file.tar.tbz
-rwxrwxrwx root/root        94 2017-09-17 02:47 example.cpp

6. Untar single tar file or specified directory in Linux: This command will Untar a file in current directory or in a specified directory using -C option.

$ tar xvfj file.tar
or
$ tar xvfj file.tar -C path of file in directoy

7. Untar multiple .tar, .tar.gz, .tar.tbz file in Linux: This command will extract or untar multiple files from the tar, tar.gz and tar.bz2 archive file. For example the above command will extract “fileA” “fileB” from the archive files.

$ tar xvf file.tar "fileA" "fileB"
or
$ tar zxvf file1.tar.gz "fileA" "fileB"
or
$ tar jxvf file2.tar.tbz "fileA" "fileB"

8. Check size of existing tar, tar.gz, tar.tbz file in Linux: The above command will display the size of archive file in Kilobytes(KB).

$ tar czf file.tar | wc -c
or
$ tar czf file1.tar.gz | wc -c
or
$ tar czf file2.tar.tbz | wc -c

9. Update existing tar file in Linux

$ tar rvf file.tar *.c

Output :

os1.c

10. list the contents and specify the tarfile using option -tf : This command will list the entire list of archived file. We can also list for specific content in a tarfile

$ tar tf file.tar

Output :

example.cpp

11. Applying pipe to through ‘grep command’ to find what we are looking for: This command will list only for the mentioned text or image in grep from archived file.

$ tar tvf file.tar | grep "text to find"
or
$ tar tvf file.tar | grep "filename.file extension"

12. We can pass a file name as an argument to search a tarfile: This command views the archived files along with their details.

$ tar tvf file.tar filename

13. Viewing the Archive using option -tvf

$ tar tvf file.tar

Output:

-rwxrwxrwx root/root       191 2017-09-17 02:20 os2.c
-rwxrwxrwx root/root       218 2017-09-17 02:20 os3.c
-rwxrwxrwx root/root       493 2017-09-17 02:20 os4.c

What are wildcards in Linux

Alternatively referred to as a ‘wild character’ or ‘wildcard character’, a wildcard is a symbol used to replace or represent one or more characters. Wildcards are typically either an asterisk (*), which represents one or more characters or question mark (?),which represents a single character.

Example:

14. To search for an image in .png format: This will extract only files with the extension .png from the archive file.tar. The –wildcards option tells tar to interpret wildcards in the name of the files
to be extracted; the filename (*.png) is enclosed in single-quotes to protect the wildcard (*) from being expanded incorrectly by the shell.

$ tar tvf file.tar --wildcards '*.png'

Note: In above commands ” * ” is used in place of file name to take all the files present in that particular directory.

Continue reading

Running Linux and IBM Spectrum Scale on IBM supercomputers

Overview

Almost all of the world’s top 500 supercomputers today run Linux®. Mostly they have batch job submission systems, which partition the supercomputer as required for the applications, and run the applications in sequence in their allocated partitions in an attempt to keep the expensive supercomputer at maximum utilization.

It is also possible to run Linux in the compute fabric as a multiuser operating system. This standard programming environment broadens the set of applications which can run on the leadership hardware and makes it easy to put the supercompute capability in the hands of scientists, engineers, and other business personnel who need it.

This article shows a Linux application running on an IBM® POWER9™ (model 8335-GTW) supercomputer cluster, and presents the software you need if you have a machine like this and want to get started with Linux.

Supercomputers and cloud computers

Having your own supercomputer is like having your own Amazon Elastic Compute Cloud. The benchmarks and test cases that you use to measure previous generations of computers (mainframes, PCs, games consoles, cellphones) don’t really apply in this new world.

Fortunately, some of the software developed for those other types of computers can be pressed in to service to make some basic measurements, to showcase these new computers, and to illustrate who in a modern competitive business needs to have access to these facilities.

Writing this article in five years’ time would be simple; we might most likely have oil reservoir models, airline seat pricing models, gas turbine flow visualizations, and similar techniques to show off; the market would be mature. However, today is today, we’re in at the ground floor of new and growing business, so we’re adapting IBM General Parallel File System (IBM GPFS™) for the purpose.

IBM General Parallel File System is now IBM Spectrum Scale

IBM GPFS, now IBM Spectrum Scale™, started life as the multimedia file system, intended for streaming video at predictable bandwidth from server farms. It is now actively marketed for data management in enterprise data centers.

A typical IBM Spectrum Scale installation consists of maybe 10 servers, each with up to a few hundred disk spindles. These servers provide POSIX file system services for hundreds to thousands of network-connected client systems.

IBM Spectrum Scale provides data replication, volume management, backup/restore, continuous operation in case of disk and server failures, improved serviceability, and scalability. These are features needed by enterprises and are what distinguish this IBM technology from open technology such as Network File System (NFS).

In our scenario with the POWER9 cluster, we allocate a solid-state disk of 1.5 TB on each POWER9 node as if it was a disk spindle. The whole IBM Spectrum Scale system consists of 16 server nodes, each with one disk of size 1.5 TB, providing a coherent POSIX file system image to client applications running on the 16 server nodes. This is an unusual geometry for an IBM Spectrum Scale cluster; but it is viable.

I had access to 16 server nodes; cluster sizes vary from two nodes all the way up to several thousand nodes depending on the intended application.

Interleaved or Random

Interleaved or Random (IOR) is a file system benchmark from the University of California. Figure 1 shows a screen capture of it running on the 16 nodes of the POWER9.

Figure 1. Running IOR

Refer to Listing 1 for the text from Figure 1.

Listing 1. Running IOR

+ jsrun --rs_per_host 1 --nrs 16 -a 1 /gpfs/wscgpfs01/tjcw/workspace/IOR/src/C/IOR -r -w -o /gpfs/ssdfilesys/tjcw//iorfile -b 921G
IOR-2.10.3: MPI Coordinated Test of Parallel I/O

Run began: Fri Nov 16 03:10:53 2018
Command line used: /gpfs/wscgpfs01/tjcw/workspace/IOR/src/C/IOR -r -w -o /gpfs/ssdfilesys/tjcw//iorfile -b 921G
Machine: Linux c699c010

Summary:
        api                = POSIX
        test filename      = /gpfs/ssdfilesys/tjcw//iorfile
        access             = single-shared-file
        ordering in a file = sequential offsets
        ordering inter file= no tasks offsets
        clients            = 16 (1 per node)
        repetitions        = 1
        xfersize           = 262144 bytes
        blocksize          = 921 GiB
        aggregate filesize = 14736 GiB

Operation  Max (MiB)  Min (MiB)  Mean (MiB)   Std Dev  Max (OPs)  Min (OPs)  Mean (OPs)   Std Dev  Mean (s)
---------  ---------  ---------  ----------   -------  ---------  ---------  ----------   -------  --------
write       33291.72   33291.72    33291.72      0.00  133166.87  133166.87   133166.87      0.00 453.25578   EXCEL
read        86441.95   86441.95    86441.95      0.00  345767.81  345767.81   345767.81      0.00 174.56413   EXCEL

Max Write: 33291.72 MiB/sec (34908.90 MB/sec)
Max Read:  86441.95 MiB/sec (90640.96 MB/sec)

Run finished: Fri Nov 16 03:21:21 2018

real    10m28.375s
user    0m0.084s
sys     0m0.017s

This shows a session from a desktop to the supercomputer. c699c010 is one of the 16 nodes allocated to this job, each with 44 POWER9 processors, six NVIDIA Tesla GPUs; a 1.5 TB solid-state disk and 605 GB of RAM, for a total of 704 POWER9 processors, 96 GPUs, 24 TB of solid-state disk, and 9.6 TB of RAM.

Log on to the launch node named c699launch01, and issue the jsrun command to ask for one processor core on each processing node to be joined up over TCP/IP as a Message Passing Interface (MPI) job.

jsrun --rs_per_host 1 --nrs 16 -a 1
/gpfs/wscgpfs01/tjcw/workspace/IOR/src/C/IOR -r -w -o
/gpfs/ssdfilesys/tjcw//iorfile -b 921G

MPI runs IOR, a distributed file system benchmark which you could run over NFS among a cluster of workstations. In this case, IOR is running over the IBM Spectrum Scale File System with its data in solid-state disk, and it achieves an average write data rate of 33.3 GBps and an average read data rate of 86.4 GBps over Mellanox InfiniBand among the 16 nodes. These data rates are limited by the transfer speeds to and from the solid-state disks.

It would be possible to ask jsrun to run the MPI job over all 704 processor cores on the 16 nodes by specifying --rs_per_host 44 –nrs 704, but one core per node is sufficient in this benchmark to use the whole capability of the solid-state disks.

Source: ibm.com

Continue reading

Linux gzip: How to work with compressed files

If you work much with Unix and Linux systems you'll eventually run into the terrific file compression utilities, gzip and gunzip. As their names imply, the first command creates compressed files (by gzip'ing them), and the second command unzip's those files.

In this post I take a quick look at the gzip and gunzip file compression utilities, along with their companion tools you may not have known about: zcat, zgrep, and zmore.

The Unix/Linux gzip command

You can compress a file with the Unix/Linux gzip command. For instance, if I run an ls -l command on an uncompressed Apache access log file named access.log, I get this output:

-rw-r--r--   1 al  al  22733255 Aug 12  2008 access.log

Note that the size of this file is 22,733,255 bytes. Now, if we compress the file using gzip, like this:

gzip access.log

we end up creating a new, compressed file named access.log.gz. Here's what that file looks like:

-rw-r--r--   1 al  al  2009249 Aug 12  2008 access.log.gz

Notice that the file has been compressed from 22,733,255 bytes down to just 2,009,249 bytes. That's a huge savings in file size, roughly 10 to 1(!).

There's one important thing to note about gzip: The old file, access.log, has been replaced by this new compressed file, access.log.gz. This might freak you out a little the first time you use this command, but very quickly you get used to it. (If for some reason you don't trust gzip when you first try it, feel free to make a backup copy of your original file.)

The Linux gunzip command

The gunzip ("g unzip") command works just the opposite of gzip, converting a gzip'd file back to its original format. In the following example I'll convert the gzip'd file we just created back to its original format:

gunzip access.log.gz

Running that command restores our original file, as you can see in this output:

-rw-r--r--   1 al  al  22733255 Aug 12  2008 access.log

The Linux file compress utilities (zcat, zmore, zgrep)

I used to think I had to uncompress a gzip'd file to work on it with commands like cat, grep, and more, but at some point I learned there were equivalent gzip versions of these same commands, appropriately named zcat, zgrep, and zmore. So, anything you would normally do on a text file with the first three commands you can do on a gzip'd file with the last three commands.

For instance, instead of using cat to display the entire contents of the file, you use zcat to work on the gzip'd file instead, like this:

zcat access.log.gz

(Of course that output will go on for a long time with roughly 22MB of compressed text.)

You can also scroll through the file one page at a time with zmore:

zmore access.log.gz
And finally, you can grep through the compressed file with zgrep:

zgrep '/java/index.html' access.log.gz

There are also two other commands, zcmp and zdiff, that let you compare compressed files, but I personally haven't had the need for them. However, as you can imagine, they work like this:

zmp file1.gz file2.gz

or

zdiff file1.gz file2.gz

Linux gzip / compress summary

As a quick summary, just remember that you don't have to uncompress files to work on them, you can use the following z-utilities to work on the compressed files instead:

◉ zcat
◉ zmore
◉ zgrep
◉ zcmp
◉ zdiff

Continue reading

Good (Free) Training is Not Hard to Come By

So now that we are indoors most of the time in keeping with social distancing guidelines, many of us are searching for ways to make this time productive. For whatever reasons you might find yourself in-between jobs or are already working for someone and want a change of career. If your interests lie within the realm of Linux and open source software, read on for some tips on how to move forward in learning more about this field.

One can find a plethora of lessons and tutorials on how to use and administer Linux throughout the internet. However, this article will focus on complete and coherent courses designed for those that are very new to Linux as well as seasoned system administrators.

There are some industry leaders in the field of technology that fully understand the importance of open source software and of Linux in general. In keeping with that understanding they offer some Linux training for free. Here are a few examples:

Cisco

Cisco Systems, one of the companies whose hardware and software powers a large portion of the internet’s backbone has long offered entry-level Linux training through their Cisco Networking Academy. Here are two free courses offered through this platform for those interested in Linux:

◉ Linux Unhatched - This is the “baby steps” introduction Linux course aimed at those that are completely new to Linux and are just starting out. This is an excellent resource for beginners.

◉ NDG Linux Essentials - Another course that originates from Network Development Group. This one lines up with the objectives for the Linux Professional Institute’s Linux Essentials exam. This course (and associated exam) presumes that you have some small amount of Linux knowledge and then expands on that to provide a more solid foundation. Those who complete this course should be fully prepared to sit for the Linux Essentials exam. As with the previous Cisco course offering, you could also register for this course at Network Development Group’s website.

IBM

Another large technology company that understands the importance of open source technology is IBM. Since the early 2000’s, IBM has maintained free training materials for Linux on their website.

Learn Linux 101 - This site offers updated training for those seeking to earn the Linux Professional Institute’s LPIC-1 Linux system administrator’s certification. These pages are free to use and no registration for a course is required. If you already have experience working with Linux in a production environment, then this course will help you bolster those base skills and prepare you for the first of the Linux Professional Institute’s professional-level certification track.

Community Supplied Training

Not to be outdone by the titans of the industry, many individuals have taken it upon themselves to provide their own quality Linux training to the world for free. Here are some examples:

◉ Guy Hummel’s Linux Survival website offers an in-browser shell simulator for you to follow along with a list of lessons listed on the left side of the page. Each group of lessons is broken down into modules with each ending with a quiz to test your newly-gained knowledge. This course introduces students to the basics of using Linux, all without having to install anything on their own systems.

◉ Ravi Saive has constructed a rather thorough series of tutorials at his site TecMint.com. Each group of tutorials are listed by a topic (such as installing Linux, package management, etc.) and are well organized. Beginners to Linux can definitely obtain a lot from this site, but the tutorials are written with the moderately experienced system administrator in mind. Nonetheless, this site is an excellent resource for those looking to learn how to use and maintain a Linux installation. There are a number of ads on the site, but they do not detract from the content too much.

◉ Last, but certainly not least, the Linux Professional Institute has been working on learning materials for those that are looking to obtain an LPI certification. This project contains lesson plans that have been submitted by community members and peer reviewed for clarity and correctness. These lessons are available in a number of languages (and more are planned for the near future) and are designed to assist an exam candidate in their studies. You can find these lessons, as well as information on joining the project, at the Learning Materials website.

Software Development

Perhaps you are more interested in creating software to run on a system? Here is a quick (and by no means exhaustive) list of sites that offer structured lessons in a particular programming language. There are a plethora of other sites available for each of the programming languages listed, these are just meant to provide you with a starting point.

◉ C and C++ - The C programming language has been around since the 1970’s, but its importance is still present even in modern projects. The Linux kernel, which is the core of any Linux distribution, is written in the C language. It is a very low level language meaning that the source code that the programs are written in speak directly to particular instructions that a computer’s processor adheres to. Oftentimes these low level programming languages are cryptic to type and read, but the benefit is in their speed in which these applications execute. A good starting point for someone wishing to learn about C or its slightly-newer object-oriented (a programming paradigm that you could learn more about in these courses) counterpart C++ can be found at C Programming’s tutorial site.

◉ Python - Likely the most popular programming language regardless of computing platform that is used is Python. Python is found in numerous applications that run on a Linux system (as well as Windows and macOS). Python is used for every day simple administration applications up to full-scale data analysis and artificial intelligence research. Python is easier to learn than a low level programming language as its syntax is less cryptic and much easier to read. There is no need to compile a Python program as it uses an interpreter to execute its code on a given system. This makes Python highly portable and able to run on numerous systems without having to rewrite anything for a given processor or operating system. If you are looking to get started in the exciting world of Python development, have a look at Python Programming’s website. If you are just starting out, go straight to the basics page to get going.

◉ HTML, CSS, JavaScript, PHP, SQL - Are you aspiring to become a web developer? A great resource is the W3Schools tutorial site. This site is more structured in a manner that has tutorials than an actual classroom approach. Nonetheless, this site offers a great springboard into the field of web development. This site covers everything from the skeletal structure of a web page in HTML up to advanced sites that use e-commerce methods with databases storing information behind the website. Take note that there are other websites available that provide further details on a given topic, particularly with web security, than what you would find at W3Schools. Nonetheless, this site is a wonderful resource for those that are just getting started.

As you can see, there are numerous locations on the internet that can assist you with your journey to become a qualified systems administrator, a developer, or both. These sites do not require any payment and are completely free to use at your pace. While you may be at home with extra free time on your hands, take a look at these sites and see what piques your interest. Whatever path you choose, practice whatever it is you are learning to become more proficient at it and ask questions at the site’s forums or mailing lists for assistance. As the saying goes: “We are all in this together.” So let us make the most of it.

Source: lpi.org

Continue reading

The Unix/Linux lpstat command

The Linux lpstat command lets you look at the progress of your print request(s). The name "lpstat" stands for "line printer statistics".

lpstat command examples

The "lpstat" command, used with no options, may return no output if there are no print jobs queued.

lpstat

It's generally better off to use an option with "lpstat". The "lpstat" command with the "-t" option gives you total information about the printer status. This option is generally the most useful, but the printout can be lengthy if you have a large number for printers configured.

lpstat -t

To look at Linux printer queue information for a specific printer, use the -p option. Here's an example of looking at the long list of output information (-l option) for a printer named "Sales":

lpstat -p Sales -l

The -r option shows whether the print scheduler is currently on or off:

lpstat -r

Finally, the -u option shows print request status information for a given user. Here's what the command looks like when looking at the printer queue information for a user named "fred":

lpstat -u fred

Continue reading

The Linux lp printing command

The lp command is used to print files on Unix and Linux systems. The name "lp" stands for "line printer". As with most Unix commands there are a fairly large number of options available to enable flexible printing capabilities.

Let's look at some lp printing commands examples.

Linux lp printing command examples

lp /etc/passwd

This command prints the "/etc/passwd" file to the default printer. If you do not use the "-d" option, the file is printed to the default printer destination.

lp -dSales .profile

This command prints the ".profile" file to the printer named "Sales". The -d option specifies the destination.

lp -dSales file1 file2 file3

This command prints the three files "file1", "file2", and "file3" to the printer named "Sales".

lp -i Sales-101 -H hold

This command places the print request Sales-101 on hold.

lp -i Sales-101 -H resume

This command resumes the print request Sales-101. The print request starts printing from page one unless you instruct it otherwise.

lp -i Sales-101 -H resume -P 4-

This command resumes the print request Sales-101, starting with page 4 of the print job. Note that the final hyphen (following the "4") is required.

The "lp" command can also be used as part of a pipeline. For instance, the following command will print the output of the "ps -ef" command to the default printer:

ps -ef | lp

Print the output of the "ps -ef" command to the default printer.

Continue reading

The Unix and Linux ps command

Unix/Linux processes FAQ: Can you share some examples of the Linux ps command? (Or, how do I use theps command?)

The basic Linux ps command

If you run the ps command by itself, it only shows very basic information about the processes you are currently running. For example, if you issue the basic command like this without any arguments:

ps

you'll see output from this command looks something like this:

  PID   TTY         TIME CMD
 4343  ttys000    0:00.35 -bash
 2617   ttys001    0:00.65 -bash
18201  ttys003    0:00.27 -bash

The PID column shows the process-id, the second column shows the TTY (terminal) the process is attached to, the TIME column shows how much CPU time the process has used, and the CMD column shows the command that is running. In this case I can tell (from experience) that I have three bash shells (terminals) running on my current system.

In practice I never run the ps command without any arguments like this, but I wanted to show this to help us get started.

There are many ways to customize the output of the ps command. For instance, I can add the f argument to get "full" information about each process. Used by itself, the f argument shows "full" information about just my processes. As an example, this command:

ps -f

leads to this output:

  UID   PID  PPID   C     STIME TTY           TIME CMD
  501  4343  4342   0   0:00.18 ttys000    0:00.35 -bash
  501  2617  2616     0   0:00.41 ttys001    0:00.66 -bash
  501 18201 18200   0   0:00.15 ttys003    0:00.27 -bash

As you can see this adds a few more columns of output to my ps command, including UID (user-id), PPID (parent process-id), and a couple of other columns I don't really look at.

Showing information about every process

As mentioned, those two ps command examples just show information about your processes. If you're a Linux system administrator, you're typically interested in information about all the processes running on the system. To show every process running on the system, we add the e argument to our previous ps command, like this:

ps -ef

This leads to much more output:

UID      PID  PPID  C STIME TTY     TIME CMD
root         1     0  0 Oct21 ?        00:00:01 init [3]                                           
root         2     1  0 Oct21 ?        00:00:00 [migration/0]
root         3     1  0 Oct21 ?        00:00:00 [ksoftirqd/0]
root         4     1  0 Oct21 ?        00:00:00 [watchdog/0]
root         5     1  0 Oct21 ?        00:00:00 [migration/1]
root         6     1  0 Oct21 ?        00:00:00 [ksoftirqd/1]
root         7     1  0 Oct21 ?        00:00:00 [watchdog/1]
root         8     1  0 Oct21 ?        00:00:00 [events/0]
root         9     1  0 Oct21 ?        00:00:00 [events/1]
root        10     1  0 Oct21 ?        00:00:00 [khelper]
root        11     1  0 Oct21 ?        00:00:00 [kthread]
root        15    11  0 Oct21 ?        00:00:00 [kblockd/0]
root        16    11  0 Oct21 ?        00:00:00 [kblockd/1]
root        17    11  0 Oct21 ?        00:00:00 [kacpid]
root        91    11  0 Oct21 ?        00:00:00 [cqueue/0]
root        92    11  0 Oct21 ?        00:00:00 [cqueue/1]
root        95    11  0 Oct21 ?        00:00:00 [khubd]
root        97    11  0 Oct21 ?        00:00:00 [kseriod]
root       158    11  0 Oct21 ?        00:00:00 [pdflush]

That was actually just the first 20 lines of output from this ps command on my CentOS Linux test system. This command actually generated 99 lines of output, and I cropped it to just show the first 20 lines (using ps -ef | head -20).

As mentioned, that's the older way to list processes on a Unix system (and it may still be preferred on Unix systems like HP-UX, AIX, and Solaris; I don't really know, I just use Linux and Mac OS X these days). I wanted to show these options to you (a) to help you learn about the ps command, and (b) see that there are other ps command options than what most people use on a day to day basis. Given that background, let's take a look at how the ps command is typically used on Linux systems.

How the Linux ps command is typically used

Now that you've seen some ps command arguments and sample output, here's how I run the ps command about 80% of the time:

ps auxwww | more

The first 20 lines of output from this command look like this:

USER   PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   2064   620 ?    Ss   Oct21   0:01 init [3]                                           
root         2  0.0  0.0      0     0 ?        S<   Oct21   0:00 [migration/0]
root         3  0.0  0.0      0     0 ?        SN   Oct21   0:00 [ksoftirqd/0]
root         4  0.0  0.0      0     0 ?        S<   Oct21   0:00 [watchdog/0]
root         5  0.0  0.0      0     0 ?        S<   Oct21   0:00 [migration/1]
root         6  0.0  0.0      0     0 ?        SN   Oct21   0:00 [ksoftirqd/1]
root         7  0.0  0.0      0     0 ?        S<   Oct21   0:00 [watchdog/1]
root         8  0.0  0.0      0     0 ?        S<   Oct21   0:00 [events/0]
root         9  0.0  0.0      0     0 ?        S<   Oct21   0:00 [events/1]
root        10  0.0  0.0      0     0 ?        S<   Oct21   0:00 [khelper]
root        11  0.0  0.0      0     0 ?        S<   Oct21   0:00 [kthread]
root        15  0.0  0.0      0     0 ?        S<   Oct21   0:00 [kblockd/0]
root        16  0.0  0.0      0     0 ?        S<   Oct21   0:00 [kblockd/1]
root        17  0.0  0.0      0     0 ?        S<   Oct21   0:00 [kacpid]
root        91  0.0  0.0      0     0 ?        S<   Oct21   0:00 [cqueue/0]
root        92  0.0  0.0      0     0 ?        S<   Oct21   0:00 [cqueue/1]
root        95  0.0  0.0      0     0 ?        S<   Oct21   0:00 [khubd]
root        97  0.0  0.0      0     0 ?        S<   Oct21   0:00 [kseriod]
root       158  0.0  0.0      0     0 ?        S    Oct21   0:00 [pdflush]

As you can see, this output is similar to the earlier output, but the columns are different. Before I talk about these ps command arguments, let me show a few more examples. Here's how I look at all httpd processes running on my Linux system:

$ ps auxwww | grep http

root      2928  0.0  0.6  17648  7120 ?           Ss   Oct21   0:00 /usr/local/apache2/bin/httpd -k start
nobody    2949  0.0  0.6  17648  6492 ?        S    Oct21   0:00 /usr/local/apache2/bin/httpd -k start
nobody    2950  0.0  0.6  17648  6492 ?        S    Oct21   0:00 /usr/local/apache2/bin/httpd -k start
nobody    2951  0.0  0.6  17648  6492 ?        S    Oct21   0:00 /usr/local/apache2/bin/httpd -k start
nobody    2952  0.0  0.6  17648  6492 ?        S    Oct21   0:00 /usr/local/apache2/bin/httpd -k start
nobody    2953  0.0  0.6  17648  6492 ?        S    Oct21   0:00 /usr/local/apache2/bin/httpd -k start
root     18508  0.0  0.0   3916   688 pts/0      S+   11:12   0:00 grep http

Here's how I list all my mysql processes:

$ ps auxwww | grep mysql

root      2837  0.0  0.1   4528  1236 ?        S    Oct21   0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid
mysql     2897  0.0  1.7 136700 17952 ?        Sl   Oct21   0:00 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock
root     18510  0.0  0.0   3916   712 pts/0    S+   11:13   0:00 grep mysql

Now that you've seen a few ps command examples, this is what the arguments to this ps command mean:

◉ The a argument means "show all processes", not just my processes. (There's a bit more to it than that, but this is usually close enough.)

◉ The u argument adds "user information" columns to the output. (Try your ps command without the 'u', and you'll see a major difference in the columns that are displayed.)

◉ The x lifts the BSD-style "must have a tty" restriction, meaning it will show processes that are not associated with a terminal (tty)>

◉ The w means "wide output". Use this option twice for unlimited width.

Examples from the Linux ps command man page

My intention for this article was to help get a new Unix or Linux user get started with the ps command. Before going, there are two more things I want to share with you.

First, there are many different variations of the ps command that you can use if you want to. For instance, I just looked at the Linux ps man page, and found these examples:

EXAMPLES
To see every process on the system using standard syntax:
   ps -e
   ps -ef
   ps -eF
   ps -ely

To see every process on the system using BSD syntax:
   ps ax
   ps axu

To print a process tree:
   ps -ejH
   ps axjf

To get info about threads:
   ps -eLf
   ps axms

To get security info:
   ps -eo euser,ruser,suser,fuser,f,comm,label
   ps axZ
   ps -eM

To see every process running as root (real & effective ID) in user format:
   ps -U root -u root u

To see every process with a user-defined format:
   ps -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm
   ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm
   ps -eopid,tt,user,fname,tmout,f,wchan

Print only the process IDs of syslogd:
   ps -C syslogd -o pid=

Print only the name of PID 42:
   ps -p 42 -o comm=

Continue reading

Unix/Linux ‘cut’ command examples

Linux cut command FAQ: Can you share some cut command examples?

The Linux cut command is a really great command filter to know. You can use it to do all sorts of cool things when processing text files and text in command pipelines.

Using the cut command with /etc/passwd

For a first cut command example, I'll use the /etc/passwd file on my Unix system. I use this file because fields in the file are separated by the ":" character, which make it very easy to work with.

Using that file, here's a Linux cut command that prints the first field (first column) of every line in that file:

cut -f1 -d: /etc/passwd

This cut command can be read as:

◉ Print the first field (-f1)
◉ Fields are delimited by the ":" character (-d:)
◉ Use the /etc/passwd file as input

The output of this command will vary by Unix and Linux systems, but it will be the first field of your /etc/passwd file, which contains the name of the users on your system. This will look something like:

nobody
lpi
george
fred

and so on.

Using the cut command in a command pipeline

You can also use cut in a Unix/Linux command pipeline. For example, although you can get this information in other ways, you can use the cut command with the ls command to get a list of filenames in the current directory, like this:

ls -al | cut -c44-

Notice that in this command I'm using the cut command "-c" option. This command can be read as:

◉ Run the "ls -al" command
◉ Pipe the output of that command into cut
◉ The cut command prints everything from each line starting at column 44 through the end of the line (-c44-)

As you can see, the -c option lets you deal with columns or character positions. In this example I wanted all the output from each line starting in column 44 and going to the end of the line, but if I wanted only columns 40 through 50, I could have specified that like this instead:

ls -al | cut -c44-50

That particular command doesn't make much sense in the real world, since filenames can all be different lengths, but it does show how to use a range of columns with the Linux cut command.

More Unix cut command examples

There are many other uses of the Unix cut command, but I wanted to share these with you to get started. The cut command is a great Unix shell script tool, and I highly recommend being familiar with it.

Continue reading