Saturday 31 October 2020

Xen Virtualization and Cloud Computing #05: The Xen Project, Unikernels, and the Future

LPI Exam Prep, LPI Tutorials and Material, LPI Study Material, LPI Guides

Previous articles in this series introduced virtualization, described how Xen achieves it through its architecture and key features, and described other platforms with which Xen interacts. This article wraps up the series by describing some exciting advances in virtualization and how Xen supports them.

As we’ve seen, a hypervisor loads a VM that holds a complete operating system such as GNU/Linux, Windows, or BSD. The VM must be complex and large because it is essentially a whole computer system. By default, the operating system in a VM supports a wide range of hardware from many vendors with different design and drivers. These regular operating systems are also multi-user, multi-process, and multi-purpose, adding to their size.

To run an application without the bulk of extra software it doesn’t need, the concept of a unikernel was developed. A unikernel is single-purpose (usually) and is created to run in a VM or container instead of directly on hardware, so it is free of the complexity of device drivers. A unikernel is not multi-user or multi-process, but just creates a single thread of code that runs one application. It can be compiled with just the libraries it needs. Therefore, a unikernel is small, lightweight, and quick to load and run. But  it runs only in a VM or container.

LPI Exam Prep, LPI Tutorials and Material, LPI Study Material, LPI Guides
In a  very broad sense, a unikernel reproduces the simple execution environment of mainframes in the era before multi-user operating systems. It is also reminiscent of two projects from the late 1990s, MIT’s Exokernel and Nemesis, which was developed by Citrix along with several universities.

Basic architecture of a unikernel

A unikernel generates a singular runtime environment that lacks a shell, can’t spawn subprocesses, and can’t create multiple threads. Developers select the minimal set of libraries that required for their application to run. These libraries are then compiled with the application and make an executable image (the unikernel) that can be executed directly on a hypervisor without a complete operating system such as GNU/Linux or Windows OS. The unikernel is considered a VM from the point of view of the hypervisor. You can also think of a unikernel as a pure runtime for programming languages. Unikernels exist for Ocaml, Erlang, Java, Haskell, Rust, Ruby, Go, an other languages.

The Xen Project is a leader in the birth of Unikernels. This technology has perhaps become the core of a new form of cloud computing, where a hypervisor can support many VMs.

Figure 7 shows  the components required to run a unikernel.

LPI Exam Prep, LPI Tutorials and Material, LPI Study Material, LPI Guides

Figure 7. Architecture for running a unikernel

I explained earlier that the advantage of containers is their speed and small size. These features were notable reasons for using containers before the  advent of unikernels. In fact, unikernels tend to beat the containers for speed and size. They also have excellent security because they contain no extra components where vulnerabilities might lurk. Boot times are nearly instantaneous.

From the features just listed, security is the most important. The unikernels, because of the lack of functions that a standard operating system provides, have a smaller attack surface. There are no shells, no utilities, and no other programs that an intruder can exploit to gain access to the system.

Most unikernels compile into applications, each of which has a unique footprint because of its unique combination of code and library functions. Thus, even if an intruder finds an attack vector into a unikernel, it won’t work with  other unikernels.

Figure 8 shows the components of a standard Linux kernel. A unikernel removes the traditional operating system layers, leaving only the minimal necessary system libraries, the programming language runtime, and the application code itself.

LPI Exam Prep, LPI Tutorials and Material, LPI Study Material, LPI Guides

Figure 8. Typical operating system components

The Xen Project as a leader provides a number of capabilities to Unikernel lovers. I will examine some of them here.

As said before, unikernels focus on small and fast packages. In Paravirtualization (PV), Xen Project can give access to the underlying hardware layer to Unikernels. But the hypervisor still contains the actual drivers for the hardware, so they don’t have to be re-implemented in the unikernels. However, unikernels are free to use HVM mode and implement actual hardware drivers.

The security is in the Xen Project hypervisor DNA and it means the smaller attack surface. This feature is in line with Unikernels.

Performance and security are very important for both the Xen Project and unikernels. The bare-metal structure of the Xen hypervisor provides a good platform for many unikernel VMs on a single host.

Some unikernel projects include:

◉ MirageOS
◉ HaLVM
◉ ClickOS
◉ Clive
◉ Drawbridge
◉ LING
◉ UniK
◉ Osv
◉ Rumprun
◉ HermitCore

Unikraft: a new development process for unikernels


With all good features unikernels offer, they have some drawbacks. Unikernels force developers to port applications manually to the underlying minimalistic OS and custom-build the list oflibraries used. Think about a big project like Apache, MariaDB, or Suricata-IDS. As you can guess, this requires a lot of effort by experts and considerable amount of time. What is the solution? Do these developers have to sacrifice the high performance of unikernels and stick to a standard OS?

The goal of the Unikraft project is to solve this problem and provide a highly configurable unikernel code base. Another goal is to encourage developers to supply their code in libraries rather than working on independent projects. Unikraft is hoping to create a community around minimalistic OSes and provide a tool that can automatically build this kind of operating systems.

Already, a small operating system kernel called Mini-OS is included with the Xen hypervisor. It is the root of many unikernel projects because it has the kernel drivers for the Xen hypervisor. But the Unikraft project’s goals differ from those of Mini-OS.

Unikraft is at an early stage. According to the NEC Lab, the its anticipated features are:

◉ Fast boot and migration time
◉ Low memory footprint
◉ Strong security
◉ High density
◉ High performance
◉ Developer-friendly tools and processes
◉ Ability to run on all major platforms

At the time of writing this article, the Unikraft projects are:

◉ LightVM
◉ Jitsu
◉ ClickOS
◉ Elastic CDNs

LightVM

LightVM is a virtualization solution based on Xen that offers fast boot-times for VMs even when hosting a large number of them. LightVM does this by replacing the Xenstore configuration database with a new distributed solution called NoXS (no Xenstore). NoXS reduces the speed of accessing the database by providing a shared page for each device that contains all the information needed for device initialization. LightVM also using the new Chaos toolstack that does operations like instantiation, migration, saving, and restoring. Currently, LightVM relies on the Linux kernel for Dom0, but for DomU it can use both Linux and Mini-OS based guests. LightVM can boot a VM in 2.3ms, which is twice as fast as Docker. With LightVM you can say, “My VM is both lighter and safer than your Container.”

MirageOS

MirageOS is a library providing operating system functionality. It is used to create Unikernels that provide secure, high-performance network applications for cloud computing and mobile platforms. The main programming language for MirageOS is Ocaml, a high-level programming language with optional functional and object-oriented features, able to create secure and high-performance programs. You can develop your program on your PC by Ocaml and then compile it into a fully-standalone program for Unikernel. There are now more than 100 MirageOS libraries, and MirageOS offers a lot of low-level implementations of the most common protocols, such as TCP/IP, UDP, IPV6, UDPV6, ICMP, DNS, SSH, VNC, HTTP, XMPP,  and the Xen Device Drivers. MirageOS help you run your servers more cheaply, securely, and faster in any Xen based cloud.

Other aspects of the Xen Project and its future


The Xen Project is always working to support new hardware and technologies. Some of its current efforts are targettng AMD EPYC CPUs, the Hygon Dhyana 18h processor family, the Raspberry Pi4, and Intel’s AVX512.

If you believe that open source changed the world of software, you should look forward to what RISC-V, pronounced “Risk-Five,” could do for the world of microprocessors. RISC-V is based on the RISC (Reduced Instruction Set Computer) principle. It is too soon to judge the future of RISC-V. At the time of writing this article, RISC-V had some logistical problems, including a lack of a certification process, no standardization for part families or packaging, and no plug-and-play at any point of the hardware or software stacks.  Some operating systems, including GNU/Linux and FreeBSD, are being ported to RISC-V architecture. The Xen Project is working on a port too.

The embedded and automotive areas provide good news about the Xen Project too. Several companies, including EPAM, Xilinx, and GlobalLogic are working on Xen for embedded devices in order to change the world of vehicles.

All vehicles nowadays rely on computers, and they usually now control most parts of a car. Some of these parts are sensitive, and security is a big concern in this area. Components such as engine control, autonomous driving, networks, warning lights, speed display, and the fuel tank are vital and must be protected. 

Nowadays, most cars use variants of Android, but this OS is not fast and secure enough for such a purpose. More important, some of the car options need an Internet connection, which opens a door for attacks. If an attacker penetrates the car’s computer, they can play with the lives of the driver and others.

The Xen Project provides good security and performance. With a car computer running a Xen hypervisor, the car operating system can launch in the millisecond and Xen can provide a good security by isolation. In these cars, the operating system runs as a DomU under the control of the Xen hypervisor. For more information about this field, you can read “Automotive Virtualization by Xen” on the EPAM company website and “Virtual Open Systems: Automotive mixed-critical virtualization done right” on the Arm company website.

The Xen Project is active in the unikernel area, working to support new technologies and hardware.

Read the previous post


Source: lpi.org

Thursday 29 October 2020

Debug Linux shell scripts with ‘-x’


If you ever need to debug a Unix or Linux shell script, you can just add the -x option to the interpreter when you invoke it. You can add the shell's debug option on the command line like this:

sh -x misbehaving_script.sh

or at the end of the first line in your shell script (the she-bang line), like this:

#!/bin/sh -x

I was just working on a shell script related to fonts on a Mac OS X (Unix), system, and when I tried to run it normally, all it gave me was this error message:

$ sh gen_fonts.sh

gen_fonts.sh: line 41: syntax error: unexpected end of file

To help debug the script, I added the -x option, and got the following debug output:

$ sh -x gen_fonts.sh

+ DIR=/Library/Fonts

+ IFS='

'

+ FILE=fonts.html

+ FONTFILE=fonts2

+ echo '<html><head>'

gen_fonts.sh: line 41: syntax error: unexpected end of file

Read More: LPI Linux Essentials

Linux Exam Prep, LPI Tutorial and Material, Linux Prep, Linux Guides
This may not look like a tremendous help in solving the problem, but the error in my shell script was on the line just after the last one that was printed in the debug output, which was actually line eight of my shell script. So, if the debugger didn't point me directly to my syntax error, at least it pointed me in a much closer direction.

Tuesday 27 October 2020

Xen Virtualization and Cloud Computing #04: Containers, OpenStack, and Other Related Platforms

LPI Exam Prep, LPI Tutorial and Material, LPI Learning, LPI Certification, LPI Prep

Previous articles in this series introduced virtualization, discussed how Xen’s architecture provides it, and covered interesting Xen features. This article looks at some other platforms with which Xen interacts: containers and OpenStack.

Virtualization, containers—or both?

Containers are a mechanism for isolating processes, similar to virtualization. With Docker and Kubernetes, containers have became popular over the past decade and are considered by some people to be competition for virtualization. So I’ll discuss the relationship briefly.

Container technology was born in 1979 with Unix version 7 and its chroot feature. A command called chroot, which stands for “change root directory,” isolates a process and restricts its access to a specific directory. For instance, if you change a process’s root to /tmp/local23, and the process tries to run /usr/bin/python3, it will actually run /tmp/local23/usr/bin/python3.

LPI Exam Prep, LPI Tutorial and Material, LPI Learning, LPI Certification, LPI Prep
This technology was hard to use and didn’t seem to have much application, so it was mostly forgotten. But in 2000s, the idea became more popular with the introduction of FreeBSD Jails. This mechanism partitions a FreeBSD into several independent mini-systems that share the same kernel. Sun Microsystems entered the container era in 2004 with Solaris Containers (also called Solaris Zones), which combined system resource controls with boundary separation.

In 2001, this jail system was introduced into Linux with Linux Vserver, and in 2005, with OpenVZ, this technology was added to the Linux kernel.  In 2006, Linux introduced control groups (cgroups) with accounts, resource limits, and isolation of resources such as CPU and memory. Two Google engineers, Paul Menage and Rohit Seth, extended the concept with a mechanism called “process containers.” Finally, in 2008, LXC (Linux Containers) was introduced as virtualization at the operating system level. The reliability and stability of LXC persuaded developers to build other container technologies on it. The first of these was Warden in 2011, superseded by Docker in 2013.

Docker was a revolution in container technology. It offers a GUI interface and is easy to use. After Docker, another technology with the name rkt (pronounced Rocket) tried to improve on some of the gaps in Docker.

Basically, container technology is a method of packaging an application so it can run with isolated dependencies. Whereas virtual machines run their own kernels and look like complete computer systems, containers run individual processes and share the host OS through the help of container engine. Containers are lightweight and deploying them is now fast and easy. They are highly suitable for short-term applications, and when the users’ biggest priority is to run the maximum number of applications on a minimal number of servers.

Figure 5 compares containers and virtualization.

LPI Exam Prep, LPI Tutorial and Material, LPI Learning, LPI Certification, LPI Prep

Figure 5. Architecture of virtualization and containers

Despite the benefits of the containers, they also have disadvantages. I’ll mention some here:

◉ All containers on a host must be built to run on the same OS. If you want a container with a different OS, you need a different host machine. With some tricks, though, it’s possible to support applications from different OS. For example, brandZ (which stands for “branded zone”), from Oracle Solaris, added support for the Linux system call interface into the Solaris kernel so it can run native Linux applications. Such workarounds have limitations.

◉ As said earlier, the OS is shared, so any security hole in the OS kernel affects all containers on the host.

◉ If the application need the full functionality of a dedicated operating system, it can’t run in a container.

◉ Containers aren’t suitable for applications that need to be used for an extended period of time.

The sudden ascendance of Docker unfortunately led some people to think that containers are a new technology, and even worse, that containers are a superior replacement for virtualization. As time passed, most understand that these are different technologies with separate reasons for use. In some cases, a combination of these technologies is useful. We’ve seen uses for a single container inside a virtual machine, multiple containers inside a virtual machine, and even a virtual machine inside of a container.

The Xen Project and OpenStack


As we have seen, the Xen Project is the engine behind many cloud companies. Without it, some projects like AWS wouldn’t have been possible. Rackspace is one of the major companies that use Xen. In October 2010, with the help of NASA, Rackspace  started a project with the name OpenStack that became to the one of the most successful cloud platforms in the world. By now, more than 500 companies have joined this project.

The goal of the OpenStack project is to provide an open source cloud computing platform that is easy to implement, but highly scalable and can meet the needs of public and private clouds of any size. OpenStack provides a modular cloud infrastructure that runs on standard hardware, along with an enormous set of tools for management and orchestration. OpenStack is kind of virtualization management platform that sit on top of the virtualized resources and helps you to automate processes. You can use the virtual resources to run a combination of tools: for instance,  configure a network, create pooled resources, put up a user interface, etc.

The Xen Project supports OpenStack Through a toolstack called XAPI, Xen exposes an API called XenAPI to control the hypervisor.  OpenStack has a XenAPI driver to control XAPI, so all XenAPI managed servers could be used with OpenStack. XAPI is the preferred mechanism for supporting Citrix Hypervisor (XenServer).

Xen is integrated with OpenStack via the standard libvirt API for managing virtualization platforms. With Xen, the minimum level of libvirt you should use is 1.2.9.

Read the previous post


Source: lpi.org

Sunday 25 October 2020

Belonging: What Oracle LAD's D&I program is about

LPI Exam Prep, LPI Learning, LPI Tutorial and Material, LPI Certification

“The launch of Diversity Committee, where everyone can work by their selves in the thematic pillars, and where they can propose ideas and make real actions, made the transformation more significant for all employees.” (Colombian employee, Internal Survey, 2019).

Diversity and Inclusion is one of the hottest trends in HR management and we are deeply committed with that at Oracle LAD. Nevertheless, what we can see in the employee´s testimonial above is about not only D&I itself, but it is about BELONGING. Belonging is a feeling and therefore a far more powerful force than any D&I strategy could ever be. It is a fundamental human need, a word that translates across any language or culture, and a feeling that every human is wired to want.

However, to promote the best employee experience and make them feeling that they belong to Oracle, HR team in LAD has been developing a full strategy focused on Diversity and Inclusion programs, since June 2018. Collaborating closely with Traci Wade´s team, OWL Communities and OPEN chapters and, supported by EVP LAD Luiz Meisler, HR LAD has started to walk a journey to promote more equal opportunities for all at Oracle LAD.

The first stage was focused on define the five pillars of D&I to promote employee engagement, especially for minorities groups and allies. The five pillars are Women, LGBTQ+, People with Disabilities, Race/Ethnicity and Generations. Strategic and action plans were designed to get employees, leaders and external stakeholders committed to our inclusion efforts. The whole communication plan had been based on the motto “Our Differences Transform” and the main arguments to support the business case of D&I at Oracle LAD are: innovation increasing, attracting talented people and improving the understanding of our customer’s needs. All of these points tight connected to our business goals.

An important date’s calendar meaningful for employees and leaders was created for the whole region. Thus, we launched nine diversity committees in eight different counties, led by two volunteer employees, one HR partner and on Diversity Champion. The aim of these committees was offering a safe and supportive environment for sharing personal experiences between participants. Each committee composed by volunteer members, two leaders, one HR professional and one Diversity Champion, who is a senior Oracle executive who serves as ambassador/influencer about diversity and inclusion within other leaders. After 18 months since we started, the committees are counting with more than 250 voluntary members, who hold monthly meetings to raise awareness through campaigns, training and interactive actions.

The main objectives of these actions are to promote integration and share knowledge about respect, tolerance and how to work with diverse teams. We already had Soccer Championship for Female employees in Brazil, Quiz contest about LGBTQ concepts in Colombia, Generation´s Band in Argentina, Multicultural Flag Contest in Chile, National Culture Celebration in Peru, Book and Cine Clubs about D&I topics in Mexico, Foosball Championship for People with Disabilities in Brazil and more. All actions worked as an invitation for employees to join the conversation about different themes.

Every time one of these activities happened, they were followed by content exposure (a lecture, a panel, a workshop) to promote employee-learning experiences. During the actions, we try to develop a connection between employee from minorities groups and allies, willing to put all people in the same page, working together for the same goal: Inclusion. And it´s important to highlight that all projects are carried on in a collaborative way, partnering with different areas such as Recruiting, Real State and Facilities, OWL communities, OPEN chapters and more.

Regarding leaders, we developed a customized mandatory diversity training for all LAD managers and started a People Analytics´ s project to identify improvement opportunities in D&I for each LOB/country. After look at the data, we started a strongly partnership with the HR business partners to design and implement tailored action plans of diversity for each team. For example, if we have LOBs with low representativeness of women, we decide to put our efforts in recruiting fairs and events for women in tech.

One of the stories that we are proud to share is from Gabriela Marquese, a new hired Business Analyst in Brazil. Last September, Oracle sponsored Programaria Summit, where more than 250 women attended a day full of tech content. During the event, we could being volunteer in a speed mentoring activity where nine female Oracle employees from different areas, could share their professional experience and inspire the mentees. In this activity, Gabriela Marquese was a mentee and could know one of our female leaders. She got so inspired by our director´s trajectory that she decided to apply to a position at Oracle Brasil. In her first day, she met that director who inspired her and now they can inspire more women to come to Oracle.

LPI Exam Prep, LPI Learning, LPI Tutorial and Material, LPI Certification
Another project called “Challenging the statistics”, whom mainly goal is to promote inclusion of more AfroBrazilians employees in Oracle, is running in Brazil. The project has started after Oracle sponsored a conference called “Together” in the last November, which 500 AfroBrazilians composed the attendance. After the event, we decided to do more than one day of self-awareness and inclusive employer branding. Therefore, we started to invite all the participants to join Oracle XP, which is a 5-hour long event that takes place in our office in São Paulo, where the AFroBrazilian participants can know better Oracle products, technologies, executives, recruitment process and being inspired by our AfroBrazilian employees. After two editions of Oracle XP within 60 participants, we already have three AfroBrazilian new hires working at Oracle.

Our projects are being recognized even outside Oracle and many companies (including customers) are inviting us to share our D&I Best practices. We have been holding events and communications to influence our customers, marketplace, partners and external stakeholders about the importance of D & I. We are proud to be award as a Great Place to Work for All, Love4all Award and recognized by HRC (Human Rights Campaign) as one of the best companies for LGBTQ employees work in Mexico.

We truly believe that in a culture where people feel psychologically safe they will feel confident enough to put an idea on the table and it will certainly improve our business performance. Belonging creates the trust that has needed people to share their best thinking and do the best that they can for the company.

Source: lpi.org

Thursday 22 October 2020

grep command in Unix/Linux

Grep Command, Unix Command, Linux Command, LPI Study Materials

The grep filter searches a file for a particular pattern of characters, and displays all lines that contain that pattern. The pattern that is searched in the file is referred to as the regular expression (grep stands for globally search for regular expression and print out).

Syntax:


grep [options] pattern [files]

Options Description

-c : This prints only a count of the lines that match a pattern
-h : Display the matched lines, but do not display the filenames.
-i : Ignores, case for matching
-l : Displays list of a filenames only.
-n : Display the matched lines and their line numbers.
-v : This prints out all the lines that do not matches the pattern
-e exp : Specifies expression with this option. Can use multiple times.
-f file : Takes patterns from file, one per line.
-E : Treats pattern as an extended regular expression (ERE)
-w : Match whole word
-o : Print only the matched parts of a matching line,
 with each such part on a separate output line.

Sample Commands


Consider the below file as an input.

$cat > lpicentral.txt

unix is great os. unix is opensource. unix is free os.
learn operating system.
Unix linux which one you choose.
uNix is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.

1. Case insensitive search : The -i option enables to search for a string case insensitively in the give file. It matches the words like “UNIX”, “Unix”, “unix”.

$grep -i "UNix" lpicentral.txt

Output:

unix is great os. unix is opensource. unix is free os.
Unix linux which one you choose.
uNix is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.

2. Displaying the count of number of matches : We can find the number of lines that matches the given string/pattern

$grep -c "unix" lpicentral.txt

Output:

2

3. Display the file names that matches the pattern : We can just display the files that contains the given string/pattern.

$grep -l "unix" *

or

$grep -l "unix" f1.txt f2.txt f3.xt f4.txt

Output:

lpicentral.txt

4. Checking for the whole words in a file : By default, grep matches the given string/pattern even if it found as a substring in a file. The -w option to grep makes it match only the whole words.

$ grep -w "unix" lpicentral.txt

Output:

unix is great os. unix is opensource. unix is free os.
uNix is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.

5. Displaying only the matched pattern : By default, grep displays the entire line which has the matched string. We can make the grep to display only the matched string by using the -o option.

$ grep -o "unix" lpicentral.txt

Output:

unix
unix
unix
unix
unix
unix

6. Show line number while displaying the output using grep -n : To show the line number of file with the line matched.

$ grep -n "unix" lpicentral.txt

Output:

1:unix is great os. unix is opensource. unix is free os.
4:uNix is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.

7. Inverting the pattern match : You can display the lines that are not matched with the specified search sting pattern using the -v option.

$ grep -v "unix" lpicentral.txt

Output:

learn operating system.
Unix linux which one you choose.

8. Matching the lines that start with a string : The ^ regular expression pattern specifies the start of a line. This can be used in grep to match the lines which start with the given string or pattern.

$ grep "^unix" lpicentral.txt

Output:

unix is great os. unix is opensource. unix is free os.

9. Matching the lines that end with a string: The $ regular expression pattern specifies the end of a line. This can be used in grep to match the lines which end with the given string or pattern.

$ grep "os$" lpicentral.txt

10. Specifies expression with -e option. Can use multiple times :

$grep –e "Agarwal" –e "Aggarwal" –e "Agrawal" lpicentral.txt

Grep Command, Unix Command, Linux Command, LPI Study Materials

11. -f file option Takes patterns from file, one per line.

$cat pattern.txt

Agarwal
Aggarwal
Agrawal

$grep –f pattern.txt  lpicentral.txt

Tuesday 20 October 2020

Linux shell script - while loop and sleep example

Linux Shell Script, LPI Exam Prep, LPI Learning, LPI Certification, LPI Prep

Linux shell script FAQ: Can you share a Linux shell script while loop example? While you’re at it, can you show how to use the sleep command in the shell script while loop?

Sure. As a little background, I’ve written a program I call an Email Agent that periodically scans my email inbox, and does a lot of things to the inbox, including deleting the over 6,000 spams that I receive in a typical day. A recent problem with the Agent is that it runs too fast, apparently overwhelming the sendmail process on the machine that it runs on.

To help keep sendmail alive, I slow my program down in two ways. First, I try to get it to go through messages more slowly. Because the program is written in Java, I accomplish that with the Thread.sleep call in my Java code. Second, I pause the program by calling the Unix/Linux sleep command from my Bourne shell script.

Sleeping in a shell script while loop

As an example of both a while loop and sleep command, here is how my Email Agent program is now run from inside a Bourne shell script:

i=1

while [ "$i" -ne 0 ]

do

  i=./runEmailAgent

  sleep 10

done

Linux Shell Script, LPI Exam Prep, LPI Learning, LPI Certification, LPI Prep
Basically what happens is that my Email Agent program is called by my shell script, and then the Email Agent program returns a numeric value when it is finished running. A value of zero means it is finished doing all it can do, while any other number means it still has work to do, but stopped itself to give sendmail a break. After that, I give sendmail a second break by calling the Linux sleep command. I tell the sleep command to take a break for 10 seconds. When ten seconds has come and gone, the test in the while block is run again.

Linux shell script while loop and sleep example

If you ever wanted to write a while loop in the Bourne shell, I hope this serves as a simple example. Please note that it is very easy to create an infinite loop here, so be careful that the program you call will actually return a zero value at some time (assuming you decide to implement the loop exactly as shown above).

Saturday 17 October 2020

Xen Virtualization and Cloud Computing #03: Key Features of Xen

LPI Exam Prep, LPI Tutorial and Material, LPI Guides, LPI Prep, LPI Certification

The previous articles in this series introduced virtualization and showed how Xen is designed to provide it efficiently. Here we’ll delve into some interesting features and their importance. A larger list can be found at the appropriate project page on features. At time of writing this article, the most recent version of Xen Project is 4.13.

Security-related features

The Meltdown and Spectre processor vulnerabilities, which exploit complex performance-enhancing features of modern microprocessors, have presented formidable challenges to the developers of operating systems and application. Meltdown and Spectre were officially discovered in January 2018.  This section describes two enhancements to Xen to mitigate against these difficult vulnerabilities.

LPI Exam Prep, LPI Tutorial and Material, LPI Guides, LPI Prep, LPI Certification
Meltdown, which affects Intel x86, IBM Power, and some ARM microprocessors, allows a malicious process to read data from any address that is mapped to the current process's memory space. Effectively, the process can read all memory without permission. The malicious process accomplishes this by finding a timing flaw in the execution of several processor features (such as the cache and pipeline) that are individually secure. At the time of disclosure, this vulnerability affected many products, with impacts on an enormous number of servers and cloud providers. Companies began writing patches to block the Meltdown vulnerability, causing performance losses between 5 and 30 percent.

Spectre also exploits modern performance features. In modern microprocessors, a digital circuit tries to guess the outcome of a conditional operation such as an “if…else” statement, using information gathered before the execution of the program, and prepares for the most likely result. It. In other words, it tries to guess the way of an if-then-else statement will go before it knows it exactly. The name of this technique is branch prediction. It is an important component of modern CPU architectures such as the x86 and plays a critical role in achieving higher performance. Spectre exploits branch predication system to read arbitrary locations in the allocated memory of a program. This attack can be implemented in a browser using JavaScript, so it is important to keep your browser up to date.

On March 15, 2018, Intel reported that it will redesign its CPU processors to help protect against Meltdown and Spectre. On October 8, 2018, Intel added firmware to its latest processors to mitigate against these attacks.

Hypervisor changes to mitigate against Meltdown and Spectre

The Xen hypervisor, like other products, was affected by these vulnerabilities, specifically:

◉ “Rogue Data Load” (aka SP3, “Variant 3”, Meltdown, CVE-2017-5754)

◉ “Branch Target Injection” (aka SP2, “Variant 2”, Spectre CVE-2017-5715)

◉ “Bounds-check bypass” (aka SP1, “Variant 1”, Spectre CVE-2017-5753)

There is no way to completely prevent risks from these vulnerabilities, but adding execution boundaries and other checks to code can partially plug the holes. Thus, we talk about “mitigating against” the vulnerabilities.

The initial focus of the Xen Project was on fixes for Meltdown, then Spectre Variant 2, and finally Spectre Variant 1. SP1 and SP2 affect Intel and AMD processors, but ARM processors vary by model and manufacturer. SP3 affects only Intel processors. To mitigate against Meltdown, the Xen Project published three solutions with the names Vixen, Comet, and PTI. Unfortunately, the fix to mitigate against SP1 requires microcode updates from Intel and AMD. Currently, therefore, there is no mitigation for SP1. But its attack surface can be reduced through technology contributed to the Xen Project by Citrix. It works by branch hardening.

◉ SP2 can be mitigated by a combination of microcode, compiler, and hypervisor changes.

◉ SP3 can be mitigated by page-table isolation (PTI).

Core Scheduling

This technology, contributed by SuSE Linux, helps to contain the negative effects of a Meltdown or Spectre breach. Normally, every virtual CPU could be scheduled on any physical CPU, and could move between physical CPUs for efficient scheduling. This increased the risk that information could be leaked from one VM to another, just as travel between cities allows an infection to spread faster. The only way to completely mitigate against this vulnerability is to disable hyper-threading, which would cause tremendous performance hits.

The core scheduling feature allows Xen to group virtual CPUs and schedule them on a limited set of physical cores. With this technology, users can keep hyperthreading enabled. Initial benchmarks have shown lost performance for many workloads. SUSE and Citrix are working on the feature, and in upcoming releases we hope to see better trade-offs between security and performance.

Hypervisor-based Memory Introspection (HVMI)

This is technology donated by Bitdefender to the Xen project on July 30, 2020 to protect against malware in the operating systems that run on Xen. HVMI has a key advantage over malware detection systems on guest operating systems: while smart malware can take over a whole guest and disable detection or prevention mechanisms on the guest, the malware has no way to reach into the underlying hypervisor.

Malware has become extremely dangerous and hard to fight for several reasons:

◉ It can enter the system whenever a single unaware user on the system visits an infected web site or opens a file received from a trusted person.

◉ It can exploit operating system vulnerabilities to gain superuser privileges and take over the whole system. Very few operating systems divide privileges in order to limit malware to  one area.

◉ It has gotten sophisticated enough to hide its files or other traces from administrators, and to disable measures designed the thwart it.

A remarkable story showing the power of malware concerns an attack known as Carbanak, which infected more than 100 banks in thirty nations and did $1 billion worth of damage globally. In late 2013, an investigation of a bank in Kiev revealed that stealth malware injected by Carbanak monitored the internal systems of the bank for several months successfully covering its tracks. The malware recorded every employee’s activity and sent back videos and images to the intruder without drawing  any attention..

The Bitdefender name is familiar to all IT staff. It is a leading global cybersecurity company, protecting over 500 million systems worldwide. Bitdefender and Citrix collaborated on Citrix Hypervisor. As we know, the hypervisor isolates VMs from each other and provide clean, low-level information about the memory used by each virtual machine. The result of this collaboration is a new security layer that can see everything happening in your infrastructure, but which Malware cannot reach. Bitdefender’s Hypervisor Introspection (HVI) technology detects suspicious activities by working directly with raw memory. At this level, malware can’t hide.

Bitdefender HVI assumes that your systems are not clean, and you can command it to inject cleaning tools into the live virtual machines. The HVI already detects and blocks the most famous attacks, including Carbanak, Turla, APT28, NetTraveler, and Wild Neutron, without knowing the vulnerabilities used by the attackers.

When Bitdefender decided to release HVI to Xen  as open source, they called it Hypervisor-based Memory Introspection (HVMI). The HVMI technology understands and applying security logic to memory events within running Linux and Windows VMs. It examines the memory in real time for signs of memory-based attack techniques that used to exploit known and unknown vulnerabilities.

Along with this, Bitdefender open sourced its ”thin” hypervisor technology, known as Napoca, and donated it to the Xen Project. The Napoca hypervisor was used in developing HVI technology. A distinctive feature of Napoca is that it virtualizes CPU and memory, not all hardware, and therefore allows hypervisor introspection on machines that don't run a full hypervisor.

Management-related features

These features reduce the burden of managing hypervisors.

Late uCode loading

Microcode, often shorted to “uCode” (where the “u” stands for the Greek letter mu), is chip manufacturer firmware, The uCode typically contains mitigations for HW vulnerabilities and is typically updated during system initialization or kernel boot. The update formerly required a reboot and a long down-time. Xen Project 4.13 lets the Xen Hypervisor deploy a uCode update without any reboot. This feature was contributed by Intel.

Upgraded live-patching

This is a mechanism for replacing small sections of code in a running hypervisor, so that you don’t have to shut down the hypervisor and terminate all the VMs running on it. The feature is generally used to deploy critical security fixes.

Live-patching has been around for a while in several Xen-based products, and was included as a tech preview feature since Xen 4.7. Now it is a supported feature on the x86 architecture. The patching does need all activity to be paused, but this pause time should be small. Amazon is working to improve this feature further. We plan to extend it to other architectures besides x86.

Recent improvements to live-patching include the capability to patch inline assembly code, improvements to stacked modules, support for module parameters, additional hooks and replicable apply/revert actions, extended python bindings for automation, and additional validation of live patches.

Live-patching is not the final goal for  live updates, because it is limited to small, localized code changes. The Xen Project team is also working on a broader live update feature. When it’s finished, an administrator will be able to upgrade a Xen hypervisor and its tools to a new version without stopping and relaunching the guests.

Embedded and safety-critical application features

These features support particular settings that need to run the hypervisor and VMs in unusual ways.

OP-TEE support

TrustZone is a security feature of ARM processors, allowing privileged users to run a process is memory shut off from access by other processes. Because there is only one trusted zone on each chip, sharing it among multiple VMs is difficult. Therefore, Xen did not originally offer TrustZone access to guest VMs. Thanks to a feature contributed by EPAM, starting with Xen 4.13, all guests can concurrently run applications on Arm TrustZone without conflicts. More work need to be done on this feature, though.

Renesas R-CAR IPMMU-VMSA driver

Automobiles rely increasingly on software. Their multiple, concurrent software processes call for virtualization in order to protect the high-stakes security required in automobiles. Thus, many automotive systems use Xen hypervisors. Access to GPUs is valuable for the virtual processes, in order to achieve the real-time performance needed when the car is in motion, but this requires access to ARM's Virtual Memory System Architecture (VMSA). Renesas has added this VMSA support to its ARM-based chips in Xen 4.13, and a driver contributed to the Xen Project by EPAM makes that access available to automobiles’ computing systems.

Dom0-less passthrough and ImageBuilder

An earlier article in this series described the central role of the privileged domain, Dom0, in Xen. Because the presence of Dom0 adds significant time (measurable in seconds) to the loading of each VM, some embedded system developers have asked for a Dom0-less architecture. Many embedded  systems need to have several VMs up and running in less than a second after the user boots the system. The code to implement a Dom0-less architecture was contributed by Xilinx in 2018. The feature does not yet work with Paravirtualization, but works with other forms of Xen virtualization.

Because there is no privileged process and no userspace tools in a Dom0-less Xen, systems using it must load guests using U-Boot, an open-source boot loader. The guest images must contain all the required binaries, such as operating system kernels and ramdisks. Thus, a new tool named ImageBuilder, whose code is on GitLab, is provided to automate the building of Dom0-less configurations for U-Boot.

Figure 4 shows a Dom0-less architecture.

LPI Exam Prep, LPI Tutorial and Material, LPI Guides, LPI Prep, LPI Certification

Figure 4. Xen running without Dom0

Source: lpi.org

Thursday 15 October 2020

The Linux cancel command (cancel a print job)

The Linux "cancel" command lets you stop print requests from printing (cancel them).

Linux Cancel Command, Linux Study Materials, Linux Exam Prep, Linux Certification, Linux Guides

Linux printing requests can be canceled by either (a) using the printer-id or (b) by specifying the printer name. Here are examples of each approach.

cancel laser-101

This first command cancels the print request identified as "laser-101". (You would have gotten this identifier using an lpstat command.

cancel -u student1

This command cancels all of the print jobs that are queued for the user named "student1".

Read More:

Linux Cancel Command, Linux Study Materials, Linux Exam Prep, Linux Certification, Linux Guides
LPI is the global certification standard and career support organization for open source professionals. LPI has certified professionals in over 180 countries, delivers exams in 9 languages, and has over 400 training partners.

LPI Certifications

Tuesday 13 October 2020

Linux Professional Institute DevOps Tools Engineer

Linux Professional Institute, DevOps Tools Engineer, LPI Exam Prep, LPI Guides, LPI Learning, LPI Tutorial and Material

Businesses across the globe are increasingly implementing DevOps practices to optimize daily systems administration and software development tasks. As a result, businesses across industries are hiring IT professionals that can effectively apply DevOps to reduce delivery time and improve quality in the development of new software products.

To meet this growing need for qualified professionals, Linux Professional Institute (LPI) developed the Linux Professional Institute DevOps Tools Engineer certification which verifies the skills needed to use the tools that enhance collaboration in workflows throughout system administration and software development.

In developing the Linux Professional Institute DevOps Tools Engineer certification, LPI reviewed the DevOps tools landscape and defined a set of essential skills when applying DevOps. As such, the certification exam focuses on the practical skills required to work successfully in a DevOps environment – focusing on the skills needed to use the most prominent DevOps tools. The result is a certification that covers the intersection between development and operations, making it relevant for all IT professionals working in the field of DevOps.

Current version: 1.0 (Exam code 701-100)

Objectives: 701-100

Prerequisites: There are no prerequisites for this certification. However, an additional certification in the candidate’s primary area of expertise, such as LPIC-1 or a developer certification, is strongly recommended.

Requirements: Passing the DevOps Tools Engineer exam. The 90 minute exam is 60 multiple choice and fill-in-the-blank questions.

Validity period: 5 years

Cost: Click here for exam pricing in your country.

Languages for exam available in VUE test centers: English, Japanese

Linux Professional Institute, DevOps Tools Engineer, LPI Exam Prep, LPI Guides, LPI Learning, LPI Tutorial and Material
To receive the Linux Professional Institute DevOps Tools Engineer certification the candidate must:

◉ Have a working knowledge of DevOps-related domains such as Software Engineering and Architecture, Container and Machine Deployment, Configuration Management and Monitoring.

◉ Have proficiency in prominent free and open source utilities such as Docker, Vagrant, Ansible, Puppet, Git, and Jenkins.

Objectives: 

1. Software Engineering

2. Container Management

3. Machine Deployment    

4. Configuration Management

5. Service Operations

Saturday 10 October 2020

Xen Virtualization and Cloud Computing #02: How Xen Does the Job

LPI Exam Prep, LPI Tutorial and Material, LPI Guides, LPI Learning, LPI Certification, LPI Cloud Computing

This is the second article in a series. The first introduced virtualization and the Infrastructure as a Service (IaaS). This article explains how Xen uses different types of virtualization to achieve efficient isolation of virtual machines (VMs).

Hardware and operating system support

Xen runs as host on a number of Unix-like systems and GNU/Linux distributions. It is distributed with several Linux distributions by default, including, Debian, Ubuntu, openSUSE, and CentOS. Other GNU/Linux distributions can compile Xen from source code. Other Unix flavors where Xen can be installed as host are FreeBSD, NetBSD, Solaris, and OpenSolaris-based.

Although macOS is a Unix-based operating system, it is proprietary and offers licenses to a limited set of virtualization products. So Xen does not run on macOS.

Operating systems supported as guests on Xen include GNU/Linux, FreeBSD, NetBSD, OpenBSD, MINIX, and Windows OS.

The Xen Project currently supports the x86, AMD64 and ARM architectures, and a port to RISC-V has been started as of the time of writing this article. RISC-V is an open-source architecture with the potential to be an excellent platform for embedded systems, and can support 32-bit, 64-bit and 128-bit words. A RISC-V port will greatly extend the reach and use cases for Xen

Virtualization and paravirtualization

Xen offers five types of virtualization for running the guest operating system:

• PV (Paravirtualization)

• HVM (hardware virtual machine)

• HVM with PV drivers

• PVHVM

• PVH

I’ll briefly describe each type, because we’ll see the implications of their differences later in this article.

Paravirtualization (PV)

Paravirtualization is a concept introduced by Xen Project. It involves rewriting part of the guest operating system, and therefore is also called a modified guest. The new kernel code replaces nonvirtualizable instructions with hypercalls into an application binary interface (ABI). The normal goal of virtualization is to fool the guest OS kernel into thinking it’s running on the real hardware. But in paravirtualization, the guest OS knows that it is running in a virtual machine and cooperates with the hypervisor to get access to the actual hardware. In particular, the hypervisor contains a set of paravirtualized (PV) drivers that the guest loads instead of the actual hardware drivers. The guest OS asks the hypervisor to perform functions that would normally require direct access to hardware: e.g., programming the MMU, accessing certain CPU registers, etc. It is much easier for the hypervisor to translate these calls than to emulate hardware devices and network interfaces to guests.

The result of PV is a very lightweight and fast hypervisor, without the difficulty of providing native OS interfaces. Xen can offer PV even on old CPUs without any support for virtualization.

Hardware-assisted Virtualization (HVM)

In 2005 and 2006, Intel introduced hardware virtualization support for their CPUs, and was shortly followed by AMD. At first, this feature was limited and suffered from low speed. AMD developed the virtual machine capability in its AMD64 CPU family that offered VM instructions under the name AMD-V. Intel provided the same feature under the name VT-x along with another technology called VT-d that allow you to pass through instructions for devices, such as PCI devices, to the guest OS.

Both technologies were invented to solve the performance issues caused by software emulation. Handling these tasks through these CPU extensions improved performance, just as other hardware advances such as network routing and encryption have sped up critical operations. However HVM just simplifies virtualization of the processor, leaving many OS-related features that the hypervisor must virtualize.

HVM, unlike PV, allows for unmodified guests. So proprietary operating systems such as Microsoft Windows can take advantage of HVM’s fast virtualization. In particular, because Microsoft Windows is a closed source OS, it originally could not run on a paravirtualized Xen hypervisor, and had to rely on slower emulation. However, because Windows allows loading of third-party drivers, it is possible now to load drivers that can determine whether the instance of Windows is running on Xen. If so, the Windows system uses paravirtual channels to improve the performance of I/O.

With HVM, Xen uses Qemu emulation for disk and network IO. Regular Paravirtualization does not use Qemu.

HVM with PV drivers

In a fully virtualized system, the interfaces that the hypervisor must provide for the network and disks are complex. Because all modern kernels have ways to load third-party device drivers, the hypervisor can be enhanced with disk and network drivers that use these paravirtualized interfaces. The other name of this technique is full virtualized system with the PV drivers. As you can guess, it is a step toward virtualization with higher performance.

PVHVM

PVHVM (also know as “PV on HVM” or “PV-on-HVM drivers”)  is a mixture of paravirtualization and full hardware virtualization. It's a stepping stone between HVM with PV drivers and PVH. The goal of this technology is to boost performance of fully virtualized HVM guests by including optimized paravirtual device drivers. These drivers bypass Qemu emulation for disk and network IO. They also support CPU functionality like Intel EPT or AMD NPT. The result is much faster disk and network IO performance.

PVH

PVH is a new kind of guest that has been introduced on Xen 4.4 as a DomU, and on Xen 4.5 as

a Dom0. (DomU and Dom0 are described in a later article in this series.) PVH occupies a place between PV and HVM. The guest OS is aware that it is running on Xen and calls the paravirtualized drivers offered by the hypervisor. But some operations still require the Qemu emulator. Specifically the OS is still booted using the hvmloader and firmware that require emulator support.

PVH can be seen as a PV guest that runs inside of an HVM container, or as a PVHVM guest without any emulated devices. It provides the best performance currently possible and calls for fewer resources on the guest OS than pure PV.

The first version of this innovation, PVHv1, did not simplify the operating system. The second version, PVHv2 (also called HVMLite), is a lightweight HVM guest that uses hardware virtualization support for memory and privileged instructions, PV drivers for I/O, and native operating system interfaces for everything else. PVHv2 does not use QEMU for device emulation. In Xen 4.9, PVHv1 has been replaced with PVHv2, which requires guests running Linux 4.11 or a newer kernel.

Comparison of the Xen technologies

Figure 2, from the Xen Project website, shows how each aspect of virtualization is handled by these different technologies.

LPI Exam Prep, LPI Tutorial and Material, LPI Guides, LPI Learning, LPI Certification, LPI Cloud Computing

Figure 2. Differences between Xen Project technologies

Running one VM Inside Another

Some sites want to run one VM inside another, in order to test a variety of hypervisors. Running a hypervisor inside of a virtual machine is called nested virtualization. The main hypervisor that runs on the real hardware is called a level 0 or L0; the hypervisor that runs as a guest on L0 is called level 1 or L1, and finally, a guest that runs on the L1 hypervisor is called a level 2 or L2. This technology has been supported in Xen since version 3.4.

Components of the Xen hypervisor


As explained, Xen is a type-1 hypervisor that runs directly on the hardware and handles all its resources for the guest, including CPU, memory, drivers, timers, and interrupts. After the bootloader, Xen is the first program that runs. Xen then launches each guest.

Just as operating systems commonly separate the root user or superuser from other users, and give the root user special powers and privileges, Xen distinguishes the difference between the host and the guests by defining domains; each domain has access only to the resources and activities allowed to that guest. Each guest runs in a DomU domain, where the U stands for unprivileged. In contrast, a single host operating system runs in a domain called Dom0.

Dom0 is a privileged domain with direct access to the hardware. Dom0 handles all access to the hardware and I/O and manages them on behalf of the users’ VMs. This domain also contains the drivers for all the hardware devices on the system, and tools to manage the Xen hypervisor.

The separation between Dom0 and DomU allows VMs to run and use all system services without privileged access to the hardware or I/O functionality—even though they think they have that access.

Figure 3 shows the Xen hypervisor architecture.

LPI Exam Prep, LPI Tutorial and Material, LPI Guides, LPI Learning, LPI Certification, LPI Cloud Computing

Figure 3. Xen architecture

The system services shown in the Dom0 box of Figure 3 include:

◉ A Toolstack for the Xen administrator. The Toolstack provides a command-line console or a graphical interface for creating, configuring, monitoring, managing, and removing the virtual machines.
◉ Device emulators, which are exposed to the guest OS in each VM. Thus, each guest issues calls and control instructions to the emulator, thinking it is interacting directly with a device.
◉ XenBus, a software abstraction that allows VMs to share configuration information. VMs do this through access to a shared database of configuration information called XenStore,

The standard setup for Dom0 contains the following functions:

◉ Native Device Drivers
◉ System Services
◉ Virtual Device Drivers (backends)
◉ Toolstack

The Toolstack can be a command line console or a graphical interface.It allows a user to manage, create, remove, and configure the virtual machines.

Although Dom0 is normally the hub of Xen virtualization, controlling all other components, we’ll see later in this series that a Dom0-less architecture is possible for specific purposes.

The Xen hypervisor supports three technologies for I/O Virtualization: The PV split driver model, Device Emulation Based I/O, and Passthrough. Explaining these technologies lies beyond the scope of this article. I’ll just say a bit about Passthrough, which gives the VM access to the physical devices. For example, you can use PCI passthrough to give a VM direct access to a NIC, disk controller, HBA, USB controller, sound card, or other PCI device. Passthrough may cause some security problems, and comes with limitations. For example, ancillary operations like save, restore, and migration are not possible.

Thursday 8 October 2020

Open source careers: How Brittany Gates found her way to Linux

LPI Exam Prep, LPI Guides, LPI Learning, LPI Certification, LPI Prep

At the end of 2019, Linux Professional Institute (LPI) celebrated 20 years of offering training and tests. The LPI20 Birthday Contest gave us the opportunity to hear from you some amazing stories about your careers in open source and how the LPI certification program helped you. Over the next weeks, we’ll share some of these stories.

This is the story of Brittany, now Site Operations Technician 2 at Twitter.

My way to Linux

My journey into Open Source started in the mid-2000s when I flirted with different Linux distros. However, I continued to use Windows as it fit my needs ... until I used Windows 10.

From Microsoft … 

I loved Windows 7. I believe that the OS was one of the best (if not the best) Microsoft made. I loved the ease of installation, the improved plug and play capabilities, the enhanced troubleshooting / repair tools, the quick bootup time. The Windows 7 installation was more streamlined than the previous editions of Windows. And, in my opinion, it didn’t freeze up during installation as Windows XP tended to do. Microsoft improved plug and play. I didn’t have to go and find drivers to install beforehand for various devices. All I did was plug it into my computer and Windows 7 installed the right one. Third, it was easier to troubleshoot Windows 7 problems. No longer did I have to decipher cryptic error messages. And using Startup Repair resolved many of the end-users’ issues. Finally, this OS could boot up fast when compared to XP and Vista. No longer did I have to sit and stare at a loading screen for minutes.   

So I was very excited to use Windows 10. I tried the beta and wasn't too impressed. I didn't like the UI choices. The UI in Windows 7 was sleek. I didn’t need tiles which widened the menu to show me the weather or the news. However,  I waited until the OS had its proper release to fully review it. When that time came, I upgraded my Windows 7 installation to Windows 10. After using the OS for several months, I discovered I heavily disliked it. The combination of the UI changes, Windows pushing me to sync a Microsoft account, and the intrusive telemetry settings turned me off. 

Upward and onward: Fedora

Instead of going back to Windows 7, I decided to give a Linux distro another try. Out of all the distros, Fedora Linux always stuck with me. I loved its UI and the overall feel of the distro. I tried other Linux distros but I can’t remember which, meaning those didn’t impress me that much. So I installed the latest version at the time which was either Fedora 22 or 23. It took me a few days to get it configured with the software I needed. Fedora, at that time, didn’t include third-party plugins or apps so I had to read instructions on how to download a LAME MP3 encoder to listen to my music. In addition, I had to find replacements for Windows programs I used, like Office and video players. However, I found those after searching. After this setup I saw my decision to change was the right one. Since then, I have never left Fedora Linux.

FOSS

In addition to my daily use computer (which is a refurb Dell Optiplex 7010), I have an older Dell Optiplex 780 refurb desktop running CentOS 8. I use that as a backup server. I chose Centos because my employer uses it on their servers. So I get to continue building my Linux admin skills at home. And I do like the Red Hat community.

As for my apps, I use primarily Open Source. I use LibreOffice for my productivity tasks. For programming, I use Atom along with IDLE and Pycharm (although Pycharm isn't Open Source). I use GIMP almost daily now for image manipulation for my websites, Brave as my main web browser, and Calibre to create and manage ebooks.

LPI Exam Prep, LPI Guides, LPI Learning, LPI Certification, LPI Prep

Grandma’s Linux

I'm not the only one using Open Source software in the family. I installed Linux Mint onto an older computer I gave to my grandma. She was a Windows user too. Unfortunately, someone broke into her house and stole her computer one day. She’s on a fixed income so I decided to build her an inexpensive computer from parts I had and wasn’t using at the time. I didn’t buy a Windows license because my grandma uses my computer for just browsing the Internet, watching YouTube videos and checking her email.

So I searched for a Linux distro with a Windows-type UI and came across Linux Mint, mostly because its menu is like the Windows “Start Menu.” After showing her where the desktop shortcuts are, my grandma uses the computer without much issue. Plus, I’ve grown to be a fan of the Linux Mint distro. Other family members continue to use Windows though. You can't win them all. :)

The LPI Exam

Finally, let me discuss my LPI exam experience. I started my journey with the previous edition of the CompTIA Linux+ certification. If you passed that certification, you received the LPIC-1 certification too. I chose this route because I use Linux everyday at work and at home, and I would like to further my career into more Linux roles. Plus, LPI covers topics I either do daily or will need to do later. That’s why I continue through the LPIC path. Each level builds upon the foundation of Linux administration. That is very important to me. Right now I’m currently studying for the first exam of the LPIC-2, and plan to take it by the end of November 2020.

Source: lpi.org

Tuesday 6 October 2020

Xen Virtualization and Cloud Computing #01: Introduction

LPI Exam Prep, LPI Learning, LPI Certification, LPI Tutorial and Material, LPI Study Materials, LPI Cloud Computing

The Xen Project is a free and open source hypervisor that enables a computer to run multiple operating systems simultaneously on the same hardware. This article begins a series that covers the way Xen achieves this result efficiently, important features, and ways in which the Xen Project is supporting new advances in virtualization.

Xen forms the key infrastructure for many Internet hosting service companies and cloud providers, which rely on Xen’s secure isolation of users and efficient resource sharing. Private companies also use Xen to divide up resources on their servers among their internal users. Today, big companies such as Amazon, AMD, Bromium, Cisco, Citrix, Google, Intel, Oracle, Samsung, and Verizon are using Xen in products and services including Citrix Hypervisor, XCP-ng, Oracle VM Server, IBM SoftLayer, and Amazon EC2. Qubes OS, a security-focused desktop OS, enforces isolation via the Xen hypervisor.

The Xen Project was started by Ian Pratt at the University of Cambridge and is one of the most successful projects at this university. The first version of Xen was released in 2003. Soon after, Ian Pratt with the help of other Cambridge friends launched the company XenSource Inc to bring Xen to the enterprise market. The company was acquired by Citrix in October 2007, turning the page on a new era for Xen. The Citrix company produced Xen Enterprise 3.0 in August 2006, based on version 3.0.0 of the Xen hypervisor. The company open sourced Citrix XenServer (Citrix Hypervisor) in 2013. Finally, on April 15, 2013, the Linux Foundation included the Xen hypervisor in its umbrella support and named it the Xen Project.

LPI Exam Prep, LPI Learning, LPI Certification, LPI Tutorial and Material, LPI Study Materials, LPI Cloud Computing
Virtualization has gone through several stages in industry acceptance. Before the first VMware offering appeared in 1998, people used virtualization primarily on desktop systems, to do something like running Windows on their Linux box. VMware promoted a new and radically more powerful use case: server consolidation. Before VMware, every server ran on its own physical computer. A single hardware server could run multiple server applications (for instance, a database, a web server, and a mail server), but this reduced performance. VMware promoted the idea of buying one larger computer and running several virtual computers inside of it.

In 2002, there was no hardware support for virtualization (HVM) on x86. The state of the art for virtualization was called "binary translation." This was incredibly complicated, not very fast, and very difficult to get right. There were no competitive open-source implementations, so if you wanted virtualization on x86, an expensive VMWare license was your only option. Traditional virtualization, like VMWare, emulated a real computer. The effect was that you had a piece of software (the operating system) talking to a piece of software (the hypervisor) over an interface designed for hardware.

It was in this environment that Xen was conceived in 2002. The core idea was the concept of paravirtualization, described later in this series.

Virtualization really grabbed the attention of the computer field with Amazon Web Services (AWS). From the beginning, AWS ran on Xen. The business of offering virtualization in the cloud is called Infrastructure as a Service (IaaS).

What is a hypervisor?

IBM invented the hypervisor in the 1960s for its mainframe computers. A hypervisor or virtual machine monitor (VMM) is software or hardware that creates and runs virtual machines. Virtual machines act just like independent, stand-alone processors and appear to be independent processors to the user, but actually share a chip with other virtual machines. Each VM interacts with outside world in its usual way, issuing calls and control instructions to hardware and network devices, memory, and CPUs. But behind the scenes, the hypervisor intercepts all these calls and instructions. The hypervisor carries them out in a way that prevents them from interfering with other VMs, and that respects the resource needs of each VM.

Although the concept of virtual machines has suddenly become popular in the past decade, IBM invented it in the 1960s for its mainframe computers. Nowadays, some hypervisors are embedded into custom devices.

When in use, the hypervisor is called a host machine and each VM managed by this host is called a guest machine. The hypervisor shares the system resources between the VMs while keeping them isolated, so that no user can accidentally or maliciously see or change another user’s data. With the help of the hypervisor, a system can run multiple operating systems at once and use the system resources in an efficient way.

Two types of hypervisor exist, called simply type-1 and type-2. The type-1 hypervisor, also known as native or bare-metal, runs directly on the hardware and control the resources and manage guest VMs. Type-1 hypervisors needs their own drivers to interact with the particular hardware they run on. At the time of writing this article, modern and popular type-1 hypervisors include Xen Project, XCP-ng, Citrix Hypervisor (formerly know as XenServer), Microsoft Hyper-V, and VMware ESXi.

The type-2 hypervisor is a computer program that needs an operating system to work. This program acts as an interface between the operating system and guest VMs, and shares resources between them. The type-2 hypervisor represents each VM as a process to the underlying operating system. Type-2 hypervisors use the drivers supplied by the host OS. At the time of writing this article, popular type-2 hypervisors include Oracle VirtualBox, VMware Workstation Pro and Player, VMware Fusion, Parallels Desktop, FreeBSD bhyve, and KVM.

LPI Exam Prep, LPI Learning, LPI Certification, LPI Tutorial and Material, LPI Study Materials, LPI Cloud Computing

The difference between type-1 and type-2 hypervisors

Type-1 and type-2 hypervisors have different pros and cons:


◉ The pros of type-1 hypervisors lie in performance and security. It offerrs high performance because the hypervisor has direct access to the hardware. Security is also more reliable on type-1 than type-2, because there is no interface between the hypervisor and CPU.
 
◉ The main con of type-1 hypervisors is that GUI management of the VMs requires a separate machine. For example, after installing XCP-ng on a machine, it is is dedicated to XCP-ng and cannot run a shell or desktop alongside it. The result is that you need another machine to connect to XCP-ng and create and manage your VMs. In contrast, many hypervisors like Xen  and Microsoft Hyper-V let you run another operating system next to the hypervisor on the same machine. Xen even allows a parallel desktop environment, which is possible but not recommended with Microsoft Hyper-V because of potential vulnerabilities. All these hypervisors, though, can be managed from the command line.
 
◉ The pro of type-2 hypervisor is simplicity of management. First, you don’t need to install additional software to manage the virtual machines running on type-2 hypervisors. This trait makes type-2 virtualization attractive in development environments. You can run and test on multiple operating systems simultaneously without knowing a lot about virtualization. This does not mean that the type-1 hypervisors are inappropriate for the development environments, just that some users find type-2 hypervisors easier.
 
◉ The cons of type-2 hypervisor spring from its need to run on another operating system to access the hardware resources such as memory, devices, and networking. Thus, performance is inferior to type-1 hypervisors, and security is potentially weaker because an attacker who compromises the host OS can gain access to all the VMs that running on the host.

Why Virtualization?


Virtualization can bring many benefits to your organization and give it new power and capacity. The technology has become widespread and has been extensively discussed in the trade press, but I’ll highlight some of the key benefits that apply to Xen.:

Reduction in costs

Virtualization can reduce the costs of your IT infrastructure. In a non-virtualized environment, each service gets a dedicated physical server. Sharing a computer system among multiple services has a high risk. But today’s hardware is very powerful, so dedicating a server for one service or application just wastes resources. A virtualized environment lets a single physical server uses host many VMs safely. Each of these VMs can run a different operating system and offer different applications. Fewer physical servers mean lower costs, lower energy use, and less physical space.

Reduce downtime and faster recovery

For your customers, nothing is more painful than a service outage. When a disaster affects a physical server, IT staff must scramble to replace or fix it. Depending on the crisis, this could take hours or even days. In a virtualized environment, you can easiluy clone the virtual machines that have been affected in mere minutes.

Creativity

Why waste your IT team’s time on maintaining a lot of physical servers? VMs can be installed, updated, and maintained the with a few clicks. Your IT team can spend their time on other things, such as learning and implementing new technologies.

Control

Virtualization gives you more control over the development process. Consider a new update for an operating system or an application. You want to test this update to ensure that no problem happens. Clone the VM, apply the updates, and test it. If no problem appear, then apply the updates to the main environment.

Help the Earth’s environment

When you cut down on the number of physical servers in your company, it will lead to reduction the amount of power being consumed. Fewer servers allows a smaller carbon footprint and less electronic junk.