Understanding the Linux File System Hierarchy

How can I master the Linux File System Hierarchy?. This is a short explanation of the Linux file system Hierarchy. In a Linux system, all files are stored on file systems. A file-system hierarchy is the organization of these files into a single inverted tree of directories. The tree of directories is said to be inverted since its root at the top of the hierarchy, and the branches of directories and sub-directories stretch below the root.

The below diagram is a Red Hat Enterprise Linux(RHEL) 8 file-system directories.

As seen in the diagram, the / is the root directory at the top of the file system tree. All other directories are separated by the / character. So var is a subdirectory of the root directory (/). How then can we describe the file-system directory contents?.

Linux File System Hierarchy Content types

These are the major types of content stored in a Linux filesystem.

1. Persistent – These are the contents which should be persistent after a reboot, e.g system and applications configuration settings.

2. Runtime – Content generated by a running process; usually deleted by a reboot

3. Variable / Dynamic – These content may be appended or modified by processes running in the Linux system.

4. Static content – This remains unchanged until explicitly edited or reconfigured.

Important Linux Directories – May vary from one distro to another

These are the standard Linux directories – This is pulled from a RHEL 8 server.

System Directory	Purpose
/etc	Contains configuration files used by system services
/root	This is a home directory for the Linux superuser account, root
/boot	Contains all the files needed to start the boot process.
/home	This is where standard users store their personal configurations and data such as Documents, Videos, Music e.t.c.
/var	Has variable data that is required to persist between boots – databases, log files, mails, cache directories, Web data e.t.c.
/tmp	Stores temporary files. All Linux users can write to this directory. Files older than 10 days are deleted automatically.
/usr	This directory contains shared libraries, installed software, and read-only program data. Some of the important subdirectories include: • /usr/bin: Mostly user commands are located here. • /usr/sbin: Hosts System administrative commands that required privilege escalation to run. • /usr/local: For locally customized software.
/dev	This contains special device files used by the system to access hardware.
/run	The processes started since the last boot stores their runtime data here, e.g. process ID files and lock files. These contents are recreated on reboot.

Other directories that may be symlinks to other:

◉ /bin and /usr/bin

◉ /sbin and /usr/sbin

◉ /lib and /usr/lib

◉ /lib64 and /usr/lib6

Continue reading

The value of open source to universities: UC Santa Cruz tests the water

In addition to providing the world with its most enduring and valuable software--Internet protocol stacks, Linux, and more--free and open source software offers immense educational benefits. The gap between producing a nice prime number generator for a professor and contributing to real-life production code is huge, and numerous programmers can attest that free and open source projects helped them cross the chasm.

A few universities have recognized the value they can provide to their own students and to the world at large by sponsoring open source projects. I talked recently to Dr. Carlos Maltzahn, Adjunct Professor for Computer Science & Engineering at University of California, Santa Cruz, and the founder and director of the Center for Research in Open Source Software (CROSS). Our wide-ranging discussion covered many ways that open source works with university programs.

Over its five-year existence, CROSS found that students are eager to join the program, that people around the world with no formal connection to UCSC sign up to work on projects, and that sponsors are willing to support the projects with substantial donations. Sustainability--especially financial stability--is still hard to achieve, and CROSS has been experimenting with multiple ways to raise funds to ensure that projects can show their independence after a few years.

Other universities that reach out and build communities around open source include the University of California, Berkeley (around Spark), Stanford University (around Open vSwitch), University of Indiana, Urbana-Champaign (around LLVM and Clang), and Boston University, through a partnership with Red Hat that I have written about. Red Hat, along with the National Science Foundation, also launched a web resource called Professors' Open Source Software Experience (POSSE) that teaches teachers how to incorporate open source into education. Here I will discuss only CROSS and the implications of its work for other universities.

Education for free software

Every successful computer science student, whether or not they want to work on free software, has to cross the gap I mentioned earlier between class exercises and work environments. The gap encompasses everything the programmer has to do before writing code and after writing code. Before coding, they have to assess user needs, choose useful tasks, and get buy-in from teams. After coding, they have to carry out testing and integration, submit code for review, accept feedback, remain responsible for maintenance, and stay in touch with their team generally. Free software projects provide excellent communities to learn these skills.

Furthermore, computer science programs don't usually teach students how to read other people's code. They don't get to see what professional, production-ready projects look like. The free and open source projects are rich sources of high-quality code, which UCSC students study as part of their immersion in open source.

Some of the training in open source for UCSC computer science students is classroom-based. They learn how to work on Github, the dynamics of free software communities, and the history of two major communities: the Linux kernel, and FreeBSD. But they also get an unusual hands-on project they must complete in order to get a passing grade: they must get a patch to the Linux kernel accepted. Their patch can be trivial, such as a documentation change, but the assignment requires them to engage in the community.

CROSS projects

CROSS awards fellowships for work on research and incubator projects. Research fellows are UCSC Ph.D. students who are working on cutting-edge innovation with a plausible path to generate open source software projects. Incubator fellows are postdocs who are building a developer community around an open source research prototype.

CROSS calls for research and incubator proposals twice a year and tries to start at least two new projects per year, subject to availability of funding. CROSS, together with its industry sponsors and advisory committee (which includes Doug Cutting, Sage Weil, Karen Sandler, Nissa Strottman, and James Davis), reviews all projects twice a year and expects to fund each fellow for 2-4 years. The organization helps incubator fellows seed their developer community through the Open Source Research Experience program, which encourages students to work on project ideas authored and mentored by CROSS fellows. Thus, CROSS is both a research facility and an incubator. The https://cross.ucsc.edu/projects/index.html current list of CROSS projects is available online.

Given that CROSS has been in existence for five years, the oldest incubator project is nearing completion of its fourth year and is facing the challenge of making the project sustainable outside of CROSS. Incubator fellows are doing a lot of grant proposal writing right now, while making their projects attractive to well-established open source software communities.

CROSS support includes paying incubator fellows, giving them time to recruit and mentor new developers, and guiding them as they connect to well-established open source projects and outside sources of financial support. Incubator applicants must demonstrate interest by well-known developer communities outside of UCSC when proposing their incubator project. We'll see in the next section what's in it for these outsiders.

Working on a CROSS project is not lucrative. Maltzahn says that an incubator fellow could probably earn four times as much money getting a typical coding job in the nearby Silicon Valley. So students are working on the projects out of a passion to make them successful, something of a start-up mentality.

The sterling model for a student open source project is Ceph, the most popular open source software for object storage. It was developed by Sage Weil as a UC Santa Cruz grad student. He spun out a company around Ceph, eventually selling it to Red Hat while Ceph remained open source. Weil then became an advisor to UCSC in its creation of CROSS.

Corporate sponsors have played a key role in CROSS from the start. It began with three sponsors, and typically gets about $100,000 from a sponsor each year, for a total earning of $300,000 to $600,000 per year. Companies sign on because open source helps them create and shape new markets, look for opportunities to network with talent and potential recruits, and track and influence education, research, and next-generation open source software ecosystems.

But the money is only one important benefit offered by a sponsor. They advise the students on real-life, business requirements that affect their projects.

For instance, one CROSS project called SkyhookDM adds smart computations (such as distributed queries) to Ceph. Some companies that manufacture computer storage devices support the project because they have narrow profit margins and prefer to outsource high-risk, pre-competitive research to universities. But they can be a reality check for the SkyhookDM developers, by explaining how far they can impose new costs on the storage products. SkyhookDM is furthest along, of all CROSS projects, toward developing a sustainable funding model.

Outside volunteers

CROSS measures the success of an open source project largely by the health of the community that forms around it: not only the number of contributors is important, but also the number of different organizations they come from. As we have seen, the project must have developers unaffiliated with UCSC even before getting CROSS approval. The organization has found that its projects appeal to a lot of programmers around the world: Mexico, Guatemala, Nigeria, and India, for instance.

Some of these are volunteers; others are paid during summer sessions and sometimes continue to be paid if there is left-over money in the Fall. This summer, CROSS is employing 11 students, all undergrad, of which five are at UCSC and six from elsewhere.

Maltzahn believes that money is not the prime motivator, because about half stay on as volunteers after the payments end. They stay because they can learn from illustrious mentors at UCSC, getting training that is not available in their local communities. He has seen the experience these students get in CROSS projects helps them into degree programs.

Getting the university's act together

To adequately support free and open source projects, university lawyers and administrators have to learn a lot more about their licenses and communities. Maltzahn points out that academic institutions have spent a lot on expertise about publishing, patenting, launching businesses, and other laws and logistics in the proprietary world, but have to catch up on learning about open source strategies to amplify their impact on society.

Students and professors report that open source computer science is valuable. For instance, students learn to go find the tools they need, rather than just working in the environment set up by the professor. This lets them become "productively lost," in the phrase attributed to David Humphrey of Seneca College. Graduates report that the skills they learned, particularly how to work with other people, have made it easier to get jobs and success, whether or not the job involves open source.

Maltzahn recommends that research universities create open source program offices (OSPOs), mirroring the "Talk Openly, Develop Openly" network of big corporations' OSPOs. He'd also love to see universities tracking their impact on society through the production of open source software, such as Ceph. I have a sense that such research would attract attention to free and open source software, and prompt a lot more colleges to make it a part of computer science curriculum.

Source: lpi.org

Continue reading

The Linux cd command

The Linux cd command is used to navigate around the Linux filesystem. In this post I'll show the most common uses of the cd command.

To move to another directory on the filesystem just use the Linux cd command to move to the desired directory. For instance, this command:

cd /tmp

moves you to the /tmp directory, and this command:

cd /foo/bar

would move you to a directory named /foo/bar, assuming that directory existed.

cd - going home

Wherever you are in a Unix filesystem if you type the cd command by itself, like this:

cd

you'll move to your home directory. (If you're already in your home directory it may look like nothing happened.)

cd - going back

To move back to whatever your previous directory was, just type this:

cd -

That's a very nice shortcut, and I thank whoever came up with that one initially. :)

Moving up

If you want to move up one directory level type this:

cd ..

Unlike the DOS "cd" command, the space after the cd is important, so make sure you include it, or you'll get a "command not found" error message. To move up two directories you type this:

cd ../..

and to move up three directories you type this:

cd ../../..

and so on.

If you had a directory named html that was up three levels from your current location you could type this to move there in one command:

cd ../../../html

Going down

If instead you want to move down through a set of subdirectories named level1/level2/level3/level4 you can just type this:

cd level1/level2/level3/level4

And don't forget the command-line expansion stuff. Rarely do you need to type something out fully. Often you can just type a few characters and then use the [Tab] key to fill in the rest for you. Instead of typing that previous command out fully, odds are I could have just typed this instead:

cd le[Tab]le[Tab]le[Tab]le[Tab]

Assuming that I have no other subdirectories beginning with the letters le, my Unix system would replace le[Tab] with level1/ the first time I hit the [Tab] key, etc.

Continue reading

Read command line arguments - Unix / Linux Bash Script

Q) How to read the arguments or parameters passed to a shell script from the command line?

In general the command line arguments are passed to the bash or shell script to change the behavior of the script. In this article, I am going to provide a simple shell script to read and print the command line parameters.

Also Read: 201-450: Linux Engineer - 201 (LPIC-2 201)

Take a look at the following unix shell script:

> cat OS_Print.sh
#!/bin/bash
echo "Script execution starts"
echo "$@"
echo "$0"
echo "$1"
echo "$2"
echo "$#"
echo "Script execution ends"

The basic functionality of the above script is to print the values stored in the $ variables. Now we will run the above script by passing some arguments.

> OS_Print.sh unix linux
Script execution starts
unix linux
OS_Print.sh
unix
linux
2
Script execution ends

You can see, the command line arguments passed here are unix and linux. Command line arguments are a list of parameters separated by space delimiters passed to the shell script.

Explanation of $ variables:

◈ $@ : contains all the arguments
◈ $0 : contains script name
◈ $1 : First argument
◈ $2 : Second argument
◈ $n : Nth argument
◈ $# : Count of arguments passed.

Examples:

1. Script to iterate through arguments.

The following script prints the parameters using for loop.

#!/bin/bash

for value in $@
do
  echo $value
done

2. Print only the last argument.

There are many ways to display the last argument. The following script shows the different ways of printing the last argument.

#!/bin/bash

echo "${@: -1}"
echo "${BASH_ARGV[0]}"
echo "${@: $#}"
echo "${!#}"

for value in $@; do :; done
echo $value

Continue reading

Linux and empowerment go together for non-neurotypical

An impressive achievement was reported to Linux Professional Institute (LPI) this past June: all 12 students in the computer program at the Structuurklas Project, in Rotterdam, the Netherlands, had taken the Linux Essentials exam and passed. Even more impressive: the Structuurklas Project has offered this exam every year for the past three years, and all except one student has passed it. Some of the more proficient students also took the LPIC-1 exam.

Oh, one more thing: the Structuurklas Project serves exclusively students diagnosed with autism.

This news offers us a number of encouraging lessons: the achievement potential of diverse populations who don't thrive in conventional institutions, the employment opportunities provided by computing to those who don't easily fit in, and the inspiration that Linux and free/open source software offer to bold individuals willing to explore. We'll explore all these in this article, after a bit of background.

First, autism is extremely prevalent, and few educational institutions know how to deal effectively with the people who live with it. We now know that autism displays itself in many different ways, and that it can be associated with severe developmental problems as well as academic excellence and amazing achievements. (As just one example, climate leader Greta Thunberg, who identifies as autistic, re-energized the global movement for preserving our climate at age 15.) Although our understanding of autism and other "non-neurotypical" mental states has gotten more sophisticated, the fact remains that thousands of people with autism--even in a sophisticated and educationally advanced country such as the Netherlands--can't succeed in standard schools.

This is where the Structuurklas Project comes in. It's part of a network of schools in the Netherlands that serve people from the ages of about 17 through 21. These schools sit between secondary schools and colleges. In official terminology, the schools cover levels 1 through 4. Graduates of the schools enter the workforce in some 700 professions ranging from baking or car repair to network administration. Some go on to level 5, which is the start of a university education.

The 24 students who attend the Structuurklas Project each year are all diagnosed with autism, and were failing or unable to get the support they needed in conventional schools. The Structuurklas Project focuses on finding out what each student needs and making sure they can understand what is expected of them. During their first year, they meet for about six hours a week with social workers.

It's hard for neurotypical people to realize how much we depend, in every interaction, on implicit understandings that we convey through ambiguous statements, hand gestures, and so on. A lot of the support that helps people with autism succeed involves making these implicit communications more explicit. In addition, a school can enlist the students' innate love of taking things apart and looking inside. The managers' attitude is: if a student isn't learning, the fault lies with the school.

The gender and racial diversity at the school matches those of the Netherlands, with fewer girls than boys because girls get diagnosed with autism much less often.

Structuurklas students working on computers.

To give each student intense guidance and attention, the Structuurklas Project keeps class sizes down to 12 students. In the computer sciences classes that have aced the Linux Essentials exam, students are thrown into difficult tasks from day one. They start with a working Microsoft Windows computer--with which most are familiar--but are told to assemble a second computer from components and install the GNU/Linux operating system on it. They go on to learn the wonders of the Bash command line, C programming, and networking. The latter is particularly popular among students, which is fortunate, because the prevalence of data centers--particularly in the Netherlands--makes networking a lucrative profession.

Who put together this intrepid educational program? Nico Berg, the docent of ICT at the Structuurklas Project and now 59 years old, is a fascinating case study of his own. I had a chance to speak to him for this article.

Although he ran a pet shop for seven years, Berg always had a keen interest in computing, and particularly in GNU/Linux. This right away tells us something about the inspiring power of free software, and of GNU/Linux in particular. Inquisitive people gravitate toward these technologies, because there is so much they can learn with just their own imagination and determination.

Berg was also a professional educator for a long time, getting a four-year teacher's degree and teaching social studies.

When regulatory changes forced Berg to close his pet shop, he came into the orbit of the Structuurklas Project and learned, at age 54, that he himself has a diagnosis of autism. Asked to set up their computer program, he knew he would be teaching Linux. In addition to setting up the hands-on challenges for his students, he puts together learning materials from material that is freely available on the Internet.

The change from Windows to Linux meets resistance from many students, who tend not to like change of any sort. But they come to love Linux. They delve into the hidden riches of the command-line tools, appreciating that not everything is handed to them in a simple graphical, menu-based interface.

They also form collegial bonds and help each other meet the educational challenges that the program threw at them. Berg estimates that 20% of his students don't start out liking computers; they chose his course because they couldn't find anything else they could do. And yet, with all the students working to help each other, the course achieved near-100% success. Most graduates who entered the workforce are successful there. Every single one who applied to college was accepted.

When Berg instituted his Linux-based program three years ago, an administrator contacted the Central European office of LPI and asked if they could quickly arrange for exams to be offered at the school. Aida Rosenthal in that office worked nearly 24/7 with the administrator to set up the partnership. The final paperwork went through the day before the first exams were held.

What's next? The government of the Netherlands has recently cast a new spotlight on Linux as a crucial technology. Thanks to the importance of networked, embedded devices, also known as the Internet of Things or IoT, the government recognizes that Linux is commonly used in that setting and declared last year that all computer students must learn Linux along with some other free software technologies such as Apache, PHP, and WordPress.

Source: lpi.org

Continue reading

Record your Linux command line with the script command

Linux command line FAQ: How can I record the input and output of my Linux command line (i.e., the Linux commands I type, and the output from those commands)?

Record Linux command line input/output with the script command

Today I was installing a product from the command line on a Unix/Linux system, and they asked if I could let them know how the installation went. I said "Yes, I can let you know exactly how it goes", because I knew that I could record my entire command line login session using the Unix/Linux/BSD script command.

The script command is usually a no-brainer to use. When you're ready to start recording the input/output of your login session, just type

script

at the command line. You'll get a message back from the operating system that looks like this:

Script started, file is typescript

This is the system's way of telling you that it is now recording everything you type, and everything that is displayed on your console. When you're finished recording whatever it is that you want to record, type [Ctrl]-d or exit, and your script session will end. (Make sure you only do an exit command once to exit the script environment, because if you do it twice, the second "exit" command you issue will log you out of your system.)

When you do so, you'll see a message that looks like this:

Script done, file is typescript

If you'll now look at this file named typescript, you'll see that it contains everything you typed, and everything that the system printed out, as promised. Depending on what you do at the command line it may also contain a bunch of control characters, but those are easily weeded out so you can get back to plain ASCII text. I'll show how to do that in another blog entry.

Note that the script command should be available for recording your login session on most Unix systems, including vanilla Unix, Linux, BSD/freeBSD, Solaris, AIX, and others.

Continue reading