Linux cd command examples

Linux FAQ: Can you share some cd command examples?

The Linux cd command stands for "change directory". It is the primary command for moving between directories on a Unix/Linux filesystem.

Examples of the cd command

This first command moves you to the /usr directory. The /usr directory becomes your current working directory:

cd /usr

Similarly, this cd command moves you to the /tmp directory:

cd /tmp

You don't have to move just one directory at a time though. You can easily jump from one directory to another directory that's far, far away, like this:

cd /var/www/html/unix/edu

Linux cd command: Using wildcards

You can also use wildcard characters when moving between directories. This next command works just like the previous command, taking me to the /var/www/html/unix/edu directory on my Linux system:

cd /v*/w*/h*/u*/e*

The * wildcard characters can be interpreted as "any number of any character", so the Linux system expands the /v*/w*/h*/u*/e* that I typed into /var/www/html/unix/edu.

Linux cd command and auto-complete

An even better way to cd to that same directory is to use the auto-complete functionality in the Bash shell of all modern Linux systems. Using auto-complete, you can hit the [Tab] key any time you might normally use the asterisk, so I could move to that previous directory using the following keystrokes:

cd /v[Tab]/w[Tab]/h[Tab]/u[Tab]/e[Tab]

That doesn't look great in text, but trust me, you'll come to love the Bash auto-complete functionality. (As they would say on the tv show Monk, "You'll thank me later.")

The cd command: Going home

On all Unix and Linux systems that I'm aware of, you can always get back to your home directory by typing the cd command without any arguments, like this:

cd

As far as I know, that works with Bash, the Korn shell, the C shell, etc.

cd command: Back to the previous directory

Again, with every Unix and Linux shell I've worked with, you can also always move back to your previous directory by adding a hyphen after the cd command, like this:

cd -

This is a great feature for when you're moving back and forth between two different directories while working on a project.

Continue reading

The Linux copy command (cp)

Linux file copy FAQ: How do I copy Linux files and directories? (Or, Can you share some cp command examples?)

You use the cp command to copy files and directories on Linux systems. Let's look at some copy examples to see how this works.

Using Linux cp to copy files

At its most basic, here's how you copy a Linux file:

cp Chapter1 Chapter1.bak

This cp command copies the original file named Chapter1 to the new file named Chapter1.bak. After issuing this command both your original file and the new file will be in the current directory.

It's also easy to copy a Linux file to a different directory. Here's how you copy the same file to the /tmp directory:

cp Chapter1 /tmp

and here's how you copy a file one level up in the directory hierarchy:

cp Chapter1 ..

As a final note about copying files, you can also copy multiple files at one time, like this:

cp file1 file2 file3 /tmp

That command copies those three files to the /tmp directory.

How to copy directories with the Linux cp command

You can also copy directories with the Linux cp command, and when you do this, you just need to remember to use the -r option. For example, this command copies the directory named Foo to the /tmp directory:

cp -r Foo /tmp

If you try to copy a directory without using the -r argument you'll get an error like this:

$ cp Foo /tmp
cp: Foo is a directory (not copied).

so it's not too hard to remember to use that option.

Continue reading

The Linux more command

The Linux more command lets you view text files or other output in a scrollable manner. It displays the text one screenful at a time, and lets you scroll backwards and forwards through the text, and even lets you search the text.

Looking at a Linux file with the more command

A common way to use the Linux more command is to display the contents of a text file. Where you might normally "cat out" the contents of a text file with the cat command, like this:

cat /etc/passwd

a problem with this approach is that when the file is long, all the output scrolls off the top of your screen. The Linux more command solves this problem by letting you scroll the output one screenful of data at a time. For this use, just use the more command instead of the cat command, like this:

more /etc/passwd

Now you can see the file contents on screen, and you can also scroll through the file output. On modern Linux systems you can use the [UpArrow] and [DownArrow] keys to scroll through the display. You can also use these keys to move through the output:

◉ [Space] - scrolls the display, one screenful of data at a time
◉ [Enter] - scrolls the display one line
◉ [b] - scrolls the display backwards one screenful of data
◉ [/] - lets you search the text, just like you would in the vi/vim editor

Use the Linux more command in a pipeline

It's also very common to use the Linux more command in a command pipeline. For instance, let's say you want to look at a list of system processes, but don't want them to all scroll off the top of your screen. In this case you use the more command with the ps command, like this:

ps auxwww | more

You can use the more command at the end of any Linux command pipeline, for example, something like this:

grep 'foo' myfile.txt | grep bar | more

The 'less' command

Most Unix and Linux systems also now include a less command. It works similarly to the more command, but has a few improved options. Check it out, or read the man (manual) pages for both commands to find out which one you prefer.

Continue reading

Examples of the Unix mkdir command

Linux directory FAQ: How do I create (make) a directory on Linux or Unix?

The Unix/Linux mkdir command is used to create new Unix/Linux directories (sub-directories). Let's take a look at some mkdir command examples.

How to create one directory

This first example creates a new directory named tmp in your current directory:

mkdir tmp

This example assumes that you have the proper permissions to create a new sub-directory in your current working directory.

Linux mkdir example - How to create multiple directories at one time

This command creates three new sub-directories (memos, letters, and e-mail) in the current directory:

mkdir memos letters e-mail

Linux mkdir example - How to create several subdirectories at one time

Use the -p option of the mkdir command to create multiple levels of subdirectories with one command. This example creates the directory /home/joe/customer/acme/foo/bar, and makes all intermediate subdirectories, as needed:

mkdir -p /home/joe/customer/acme/foo/bar

As you can imagine, that's a lot easier than typing these equivalent commands:

cd /home/joe
mkdir customer

cd customer
mkdir acme

cd acme
mkdir foo

cd foo
mkdir bar

Linux mkdir command: "Permission denied" errors

As a final note, if you try to create a directory like this:

mkdir baz

and you get an error message like this:

mkdir: cannot create directory 'baz': Permission denied

as the message implies, you don't have permission to create this directory. You can use the ls command to figure out what permission you have in this directory.

Continue reading

tar command in Linux with examples

The Linux ‘tar’ stands for tape archive, is used to create Archive and extract the Archive files. tar command in Linux is one of the important command which provides archiving functionality in Linux. We can use Linux tar command to create compressed or uncompressed Archive files and also maintain and modify them.

Syntax:

tar [options] [archive-file] [file or directory to be archived]

Options:

-c : Creates Archive
-x : Extract the archive
-f : creates archive with given filename
-t : displays or lists files in archived file
-u : archives and adds to an existing archive file
-v : Displays Verbose Information
-A : Concatenates the archive files
-z : zip, tells tar command that create tar file using gzip
-j : filter archive tar file using tbzip
-W : Verify a archive file
-r : update or add file or directory in already existed .tar file

What is an Archive file?

An Archive file is a file that is composed of one or more files along with metadata. Archive files are used to collect multiple data files together into a single file for easier portability and storage, or simply to compress files to use less storage space.

Examples:

1. Creating an uncompressed tar Archive using option -cvf: This command creates a tar file called file.tar which is the Archive of all .c files in current directory.

$ tar cvf file.tar *.c

Output :

os2.c
os3.c
os4.c

2. Extracting files from Archive using option -xvf: This command extracts files from Archives.

$ tar xvf file.tar

Output :

os2.c
os3.c
os4.c

3. gzip compression on the tar Archive, using option -z: This command creates a tar file called file.tar.gz which is the Archive of .c files.

$ tar cvzf file.tar.gz *.c

4. Extracting a gzip tar Archive *.tar.gz using option -xvzf: This command extracts files from tar archived file.tar.gz files.

$ tar xvzf file.tar.gz

5. Creating compressed tar archive file in Linux using option -j: This command compresses and creates archive file less than the size of the gzip. Both compress and decompress takes more time then gzip.

$ tar cvfj file.tar.tbz example.cpp

Output :

$tar cvfj file.tar.tbz example.cpp
example.cpp
$tar tvf file.tar.tbz
-rwxrwxrwx root/root        94 2017-09-17 02:47 example.cpp

6. Untar single tar file or specified directory in Linux: This command will Untar a file in current directory or in a specified directory using -C option.

$ tar xvfj file.tar
or
$ tar xvfj file.tar -C path of file in directoy

7. Untar multiple .tar, .tar.gz, .tar.tbz file in Linux: This command will extract or untar multiple files from the tar, tar.gz and tar.bz2 archive file. For example the above command will extract “fileA” “fileB” from the archive files.

$ tar xvf file.tar "fileA" "fileB"
or
$ tar zxvf file1.tar.gz "fileA" "fileB"
or
$ tar jxvf file2.tar.tbz "fileA" "fileB"

8. Check size of existing tar, tar.gz, tar.tbz file in Linux: The above command will display the size of archive file in Kilobytes(KB).

$ tar czf file.tar | wc -c
or
$ tar czf file1.tar.gz | wc -c
or
$ tar czf file2.tar.tbz | wc -c

9. Update existing tar file in Linux

$ tar rvf file.tar *.c

Output :

os1.c

10. list the contents and specify the tarfile using option -tf : This command will list the entire list of archived file. We can also list for specific content in a tarfile

$ tar tf file.tar

Output :

example.cpp

11. Applying pipe to through ‘grep command’ to find what we are looking for: This command will list only for the mentioned text or image in grep from archived file.

$ tar tvf file.tar | grep "text to find"
or
$ tar tvf file.tar | grep "filename.file extension"

12. We can pass a file name as an argument to search a tarfile: This command views the archived files along with their details.

$ tar tvf file.tar filename

13. Viewing the Archive using option -tvf

$ tar tvf file.tar

Output:

-rwxrwxrwx root/root       191 2017-09-17 02:20 os2.c
-rwxrwxrwx root/root       218 2017-09-17 02:20 os3.c
-rwxrwxrwx root/root       493 2017-09-17 02:20 os4.c

What are wildcards in Linux

Alternatively referred to as a ‘wild character’ or ‘wildcard character’, a wildcard is a symbol used to replace or represent one or more characters. Wildcards are typically either an asterisk (*), which represents one or more characters or question mark (?),which represents a single character.

Example:

14. To search for an image in .png format: This will extract only files with the extension .png from the archive file.tar. The –wildcards option tells tar to interpret wildcards in the name of the files
to be extracted; the filename (*.png) is enclosed in single-quotes to protect the wildcard (*) from being expanded incorrectly by the shell.

$ tar tvf file.tar --wildcards '*.png'

Note: In above commands ” * ” is used in place of file name to take all the files present in that particular directory.

Continue reading

Running Linux and IBM Spectrum Scale on IBM supercomputers

Overview

Almost all of the world’s top 500 supercomputers today run Linux®. Mostly they have batch job submission systems, which partition the supercomputer as required for the applications, and run the applications in sequence in their allocated partitions in an attempt to keep the expensive supercomputer at maximum utilization.

It is also possible to run Linux in the compute fabric as a multiuser operating system. This standard programming environment broadens the set of applications which can run on the leadership hardware and makes it easy to put the supercompute capability in the hands of scientists, engineers, and other business personnel who need it.

This article shows a Linux application running on an IBM® POWER9™ (model 8335-GTW) supercomputer cluster, and presents the software you need if you have a machine like this and want to get started with Linux.

Supercomputers and cloud computers

Having your own supercomputer is like having your own Amazon Elastic Compute Cloud. The benchmarks and test cases that you use to measure previous generations of computers (mainframes, PCs, games consoles, cellphones) don’t really apply in this new world.

Fortunately, some of the software developed for those other types of computers can be pressed in to service to make some basic measurements, to showcase these new computers, and to illustrate who in a modern competitive business needs to have access to these facilities.

Writing this article in five years’ time would be simple; we might most likely have oil reservoir models, airline seat pricing models, gas turbine flow visualizations, and similar techniques to show off; the market would be mature. However, today is today, we’re in at the ground floor of new and growing business, so we’re adapting IBM General Parallel File System (IBM GPFS™) for the purpose.

IBM General Parallel File System is now IBM Spectrum Scale

IBM GPFS, now IBM Spectrum Scale™, started life as the multimedia file system, intended for streaming video at predictable bandwidth from server farms. It is now actively marketed for data management in enterprise data centers.

A typical IBM Spectrum Scale installation consists of maybe 10 servers, each with up to a few hundred disk spindles. These servers provide POSIX file system services for hundreds to thousands of network-connected client systems.

IBM Spectrum Scale provides data replication, volume management, backup/restore, continuous operation in case of disk and server failures, improved serviceability, and scalability. These are features needed by enterprises and are what distinguish this IBM technology from open technology such as Network File System (NFS).

In our scenario with the POWER9 cluster, we allocate a solid-state disk of 1.5 TB on each POWER9 node as if it was a disk spindle. The whole IBM Spectrum Scale system consists of 16 server nodes, each with one disk of size 1.5 TB, providing a coherent POSIX file system image to client applications running on the 16 server nodes. This is an unusual geometry for an IBM Spectrum Scale cluster; but it is viable.

I had access to 16 server nodes; cluster sizes vary from two nodes all the way up to several thousand nodes depending on the intended application.

Interleaved or Random

Interleaved or Random (IOR) is a file system benchmark from the University of California. Figure 1 shows a screen capture of it running on the 16 nodes of the POWER9.

Figure 1. Running IOR

Refer to Listing 1 for the text from Figure 1.

Listing 1. Running IOR

+ jsrun --rs_per_host 1 --nrs 16 -a 1 /gpfs/wscgpfs01/tjcw/workspace/IOR/src/C/IOR -r -w -o /gpfs/ssdfilesys/tjcw//iorfile -b 921G
IOR-2.10.3: MPI Coordinated Test of Parallel I/O

Run began: Fri Nov 16 03:10:53 2018
Command line used: /gpfs/wscgpfs01/tjcw/workspace/IOR/src/C/IOR -r -w -o /gpfs/ssdfilesys/tjcw//iorfile -b 921G
Machine: Linux c699c010

Summary:
        api                = POSIX
        test filename      = /gpfs/ssdfilesys/tjcw//iorfile
        access             = single-shared-file
        ordering in a file = sequential offsets
        ordering inter file= no tasks offsets
        clients            = 16 (1 per node)
        repetitions        = 1
        xfersize           = 262144 bytes
        blocksize          = 921 GiB
        aggregate filesize = 14736 GiB

Operation  Max (MiB)  Min (MiB)  Mean (MiB)   Std Dev  Max (OPs)  Min (OPs)  Mean (OPs)   Std Dev  Mean (s)
---------  ---------  ---------  ----------   -------  ---------  ---------  ----------   -------  --------
write       33291.72   33291.72    33291.72      0.00  133166.87  133166.87   133166.87      0.00 453.25578   EXCEL
read        86441.95   86441.95    86441.95      0.00  345767.81  345767.81   345767.81      0.00 174.56413   EXCEL

Max Write: 33291.72 MiB/sec (34908.90 MB/sec)
Max Read:  86441.95 MiB/sec (90640.96 MB/sec)

Run finished: Fri Nov 16 03:21:21 2018

real    10m28.375s
user    0m0.084s
sys     0m0.017s

This shows a session from a desktop to the supercomputer. c699c010 is one of the 16 nodes allocated to this job, each with 44 POWER9 processors, six NVIDIA Tesla GPUs; a 1.5 TB solid-state disk and 605 GB of RAM, for a total of 704 POWER9 processors, 96 GPUs, 24 TB of solid-state disk, and 9.6 TB of RAM.

Log on to the launch node named c699launch01, and issue the jsrun command to ask for one processor core on each processing node to be joined up over TCP/IP as a Message Passing Interface (MPI) job.

jsrun --rs_per_host 1 --nrs 16 -a 1
/gpfs/wscgpfs01/tjcw/workspace/IOR/src/C/IOR -r -w -o
/gpfs/ssdfilesys/tjcw//iorfile -b 921G

MPI runs IOR, a distributed file system benchmark which you could run over NFS among a cluster of workstations. In this case, IOR is running over the IBM Spectrum Scale File System with its data in solid-state disk, and it achieves an average write data rate of 33.3 GBps and an average read data rate of 86.4 GBps over Mellanox InfiniBand among the 16 nodes. These data rates are limited by the transfer speeds to and from the solid-state disks.

It would be possible to ask jsrun to run the MPI job over all 704 processor cores on the 16 nodes by specifying --rs_per_host 44 –nrs 704, but one core per node is sufficient in this benchmark to use the whole capability of the solid-state disks.

Source: ibm.com

Continue reading