Categories
Linux Tutorial Series

Linux Tutorial Series – 157 – Shutting down your system – what goes on

Here is the video version, if you prefer it:

When you want to shut down your system, the following happens: (Ward, 2014)⁠

  1. init asks every process to shut down
  2. If some processes (or a process) don’t respond, init initiates a kill with a TERM signal
  3. If some processes (or a process) don’t respond still, init initiates a kill with a KILL signal
  4. The system makes preparations for a shutdown
  5. The system unmounts all of the other filesystems other than root
  6. The system remounts the root filesystem in read-only mode (meaning you can’t write, only read)
  7. The system writes out all of the data left over in the buffers to the filesystem (this happens during the remounting process; see (“Is the root filesystem unmounted during a Linux shutdown?,” n.d.)⁠)
  8. The kernel reboots or stops the system

My 2 cents – just know this conceptually. The only case where you will need to go deep into the details of this will be if you need to optimize something on a Linux machine (most likely you would be doing this professionally), if you need to know the details to accomplish some other task or if you are really really curious.

Hope you learned something useful!

References

Is the root filesystem unmounted during a Linux shutdown? (n.d.). Retrieved February 13, 2020, from https://unix.stackexchange.com/questions/298216/is-the-root-filesystem-unmounted-during-a-linux-shutdown

Ward, B. (2014). How Linux Works: What Every Superuser Should Know (2nd ed.). No Starch Press. Pages 144-145

Categories
Linux Tutorial Series

Linux Tutorial Series – 156.1 – init versions

There are multiple versions of init out there. I just wanted to list their names out in order for you to recognize the name and know: “Ah, I’m dealing with init.”. I never had to deal with init so far, in terms of something going wrong with it and me having to fix it. I also never modified my init (in terms of enabling / disabling what services it starts or doesn’t start), but you can do so if you want. I never had the need.

Here are a couple of popular init versions: (Ward, 2014)⁠

  • System V
  • systemd
  • Upstart

My Debian 10 system uses systemd. You can use Google to find out what init version your Linux distribution has and you can also use Google to find out how to tinker with it, if you must.

Hope you learned something new!

References

Ward, B. (2014). How Linux Works: What Every Superuser Should Know (2nd ed.). No Starch Press. Page 112

Categories
Linux Tutorial Series

Linux Tutorial Series – 156 – init

Here is the video version, if you prefer it:

As we already learned, init is important because it is a part of the booting process, as well as a part of the user space startup process. Here we talk a little bit more about init.

init is the process which starts all system services. (Shotts, 2019)⁠ init always has a PID of 1. A lot of the services which init starts are called daemon programs – programs that don’t have any user interface and do their thing in the background.

A thing to review, if you forgot – when talking about processes in general, we can say that a parent process spawns a child process. That means that the programs that init starts are its child processes and init is the parent process.

That’s it about init. Nothing too fancy. It starts a lot of essential system services (which are daemons).

References

Shotts, W. (2019). The Linux Command Line, Fifth Internet Edition. Retrieved from http://linuxcommand.org/tlcl.php. Pages 134-135

Categories
Linux Tutorial Series

Linux Tutorial Series – 155 – User space startup

Here is the video version, if you prefer it:

Let’s talk about how the user space starts up. To remind ourselves, there is the user space, which “regular” users can access and there is the kernel (system) space, which “regular” users can’t access. If they try to, the operating system doesn’t allow them to.

Here is roughly how the user space starts up: (Ward, 2014)⁠

  1. init
  2. Essential low-level services which enable managing hardware devices and system logging
  3. Network configuration
  4. Mid and high-level services (such as cron (used for scheduling tasks) and printing)
  5. Login prompts, GUIs, and other high-level applications

To remind ourselves, init is the process with the PID of 1, with which it all begins. I hope you now see why this holds – because init has the PID of 1 and it is the first step here. It is also speculated that init is how the universe was made, albeit scientist are not yet sure.

Hope you learned something useful!

References

Ward, B. (2014). How Linux Works: What Every Superuser Should Know (2nd ed.). No Starch Press. Pages 111-112

Categories
Linux Tutorial Series

Linux Tutorial Series – 154 – Viewing kernel messages

Here is the video version, if you prefer it:

The dmesg command is used to display kernel messages. (Ward, 2014)⁠ You use this command when you want to see if a kernel is giving you an error for a particular hardware device, for example.

An example:

mislav@mislavovo-racunalo:~/Linux_folder$ sudo dmesg

[2344706.157876] x86: Booting SMP configuration:

[2344706.157877] smpboot: Booting Node 0 Processor 1 APIC 0x2

[2344706.158274] cache: parent cpu1 should not be sleeping

[2344706.158419] CPU1 is up

[2344706.158436] smpboot: Booting Node 0 Processor 2 APIC 0x4

[2344706.158833] cache: parent cpu2 should not be sleeping

[2344706.158987] CPU2 is up

dmesg gives a lot of output, so be prepared. You can pipe it and use less to read through it, for example, or redirect its output in a file.

Another way to look at kernel messages (besides dmesg) is by looking at the contents of a file which is located in /var/log/kern.log. (Ward, 2014)⁠ Keep in mind that catting this file will produce a lot of output.

Thank you for reading!

References

Ward, B. (2014). How Linux Works: What Every Superuser Should Know (2nd ed.). No Starch Press. Pages 94-95

Categories
Linux Tutorial Series

Linux Tutorial Series – 153 – Boot loaders

Here is the video version, if you prefer it:

Boot loader’s job is to start the kernel – that is, to load the kernel into memory and start the kernel with a set of kernel parameters. (Ward, 2014)⁠ Kernel parameters are text-based parameters that tell the kernel about how it should start. Boot loader also selects among multiple kernels, switches between sets of kernel parameters, allows users to meddle with the kernel parameters and kernel images and provides support for booting other operating systems.

One of the most popular boot loaders is GRUB, short for Grand Unified Boot Loader. So, if you ever see some errors related to GRUB, you now know: “Oh, it’s my boot loader that’s the problem”.

To be honest, I never had to deal with boot loader problems, but I imagine they must be bad. May the force of Google help you in your quest to resolve those.

Thank you for reading!

References

Ward, B. (2014). How Linux Works: What Every Superuser Should Know (2nd ed.). No Starch Press. Pages 97-98

Categories
Linux Tutorial Series

Linux Tutorial Series – 152 – How it all begins – a simplified boot process

Here is the video version, if you prefer it:

Today we will talk about how your computer boots up (or, equivalently, how does it start up). We will look at the first steps your computer executes when you turn it on. This will be important to understand if you ever encounter a boot-time problem. Of course, you will use Google, but having context as to what is actually happening will prove really valuable.

Your computer boots up in the following sequence of steps: (Ward, 2014)⁠

  1. A boot loader is ran by machine’s BIOS or boot firmware
  2. The boot loader finds the kernel image on disk, loads it into memory, and starts it
  3. The kernel initializes the hardware devices and its drivers
  4. The kernel mounts the root filesystem
  5. The kernel starts a program called init; init has PID equal to 1
  6. init starts all of the other necessary system processes
  7. At some point at or near the end of this entire process, you get the option to log in to your system

Let’s clarify some terms here. Let’s go one by one:

  • What is BIOS? BIOS stands for Basic Input/Output System and it is used to check for hardware errors andrun the boot loader. (“BIOS,” n.d.)⁠ Think of BIOS as the starting point of your computer; when you turn your computer on, it checks if all the hardware is OK and if it is, it runs the boot loader. Instead of BIOS, you can have other boot firmware (such as UEFI – Unified Extensible Firmware Interface (“Unified Extensible Firmware Interface,” n.d.)⁠), but its job remains the same – check the hardware, if that’s OK, run the boot loader.
  • What is the boot loader? The boot loader is a small program whose task is to find the kernel image on disk, load it into memory and start it. (Ward, 2014)⁠. We will talk more about boot loaders in another post, but they are a small, albeit a critical piece of code which loads the kernel.
  • To review, the kernel is the “core” of the operating system and the root filesystem is the hierarchical filesystem starting from the root folder (/).
  • What is init? init is a special process whose task is to start all of the other necessary system processes and init has a PID of 1. We will also talk more about init in another post (or posts).

I think we clarified all of the terms present in this article.

A caveat for the those of you who are interested in the nitty-gritty details: initramfs can be used to mount the root filesystem.When the kernel image gets loaded by the boot loader, the kernel can mount initramfs, if it exists. This is an advanced Linux detail, but I just wanted to mention it here. More information can be found in (“About initramfs,” n.d.)⁠.

Hope you learned something new!

References

About initramfs. (n.d.). Retrieved March 31, 2020, from http://www.linuxfromscratch.org/blfs/view/svn/postlfs/initramfs.html

BIOS. (n.d.). Retrieved February 13, 2020, from https://en.wikipedia.org/wiki/BIOS

Unified Extensible Firmware Interface. (n.d.). Retrieved February 13, 2020, from https://en.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface

Ward, B. (2014). How Linux Works: What Every Superuser Should Know (2nd ed.). No Starch Press. Pages 93-94; 97

Categories
Linux Tutorial Series

Linux Tutorial Series – 151 – Checkpoint

Here is the video version, if you prefer it:

Moving on, we will talk about how Linux boots up, how it shuts down and about init – the process with which it all begins. Wow, I feel like I’m watching Lord of the Rings or something.

This is good to know because if you ever encounter any issues with any one of the things during boot-time, it will be useful to know where things went wrong and to have context on it.

Talk soon!

Categories
Linux Tutorial Series

Linux Tutorial Series – 150 – Review

Here is the video version, if you prefer it:

Let’s review what we learned:

  • Permanent storage device is the physical device your data is stored on
  • Partitions are subdivisions of the whole disk
  • A filesystem is a hierarchy of files and directories you are accustomed to interacting
  • Files and directories are represented as inodes
  • Link is a pointer to a file or a directory
  • Hard links point to the underlying data and symbolic links are like shortcuts
  • Devices (represented as files) are located in the /dev folder
  • Mounting is the process of attaching your device (your device’s storage) in the Linux directory structure; unmounting is the reverse process
  • Unmounting is important because it writes the data in the buffer to the device
  • Hash functions are used for checking file integrity

Talk soon!

Categories
Linux Tutorial Series

Linux Tutorial Series – 149 – A whole bunch of device related information

Here is the video version, if you prefer it:

In this post, I will devote a few sentences to device-related topics that, in my opinion, aren’t that important. Don’t get me wrong, you could write a couple of posts on each one of these topics, but they are not that important in your everyday use. Even though you will most likely never do anything with the information presented in this post, it does paint the picture of the Linux operating system and thus I decided to write up a few sentences on each of the topics I thought warranted it. My 2 cents would be: Read through the contents of those posts once, know that these things exist and if you ever need them, use Google to find exactly what you need for your particular purpose. The references are listed in the order in which the topics appear.

Partitioning disk devices

You can partition (and re-partition) your permanent storage devices from the command line. Although I never had the need to do this (except when I was installing my operating system), you maybe might, and that’s when you can call the big ol’ Google for help. Just know that you can do this.

df and free

The df command tells you how much free space you have on your disk drive. The free command displays the amount of free memory (RAM and swap). (Shotts, 2019)

Example:

mislav@mislavovo-racunalo:~/Linux_folder$ df

Filesystem 1K-blocks Used Available Use% Mounted on

udev 4001344 0 4001344 0% /dev

tmpfs 804232 87256 716976 11% /run

/dev/sda2 483076568 75734076 382733828 17% /

tmpfs 4021140 43720 3977420 2% /dev/shm

tmpfs 5120 4 5116 1% /run/lock

tmpfs 4021140 0 4021140 0% /sys/fs/cgroup

tmpfs 804228 14316 789912 2% /run/user/1000

mislav@mislavovo-racunalo:~/Linux_folder$ free

total used free shared buff/cache available

Mem: 8042284 4422848 662332 519756 2957104 2791520

Swap: 8259580 1225432 7034148

sysfs

sysfs provides a uniform view for attached devices based on their actual hardware attributes. (Ward, 2014)⁠

You can go to /sys/devices to see all the devices. It differs from the /dev directory because /dev is designed for interacting with devices, while sysfs is designed to view device information and manage the device.

lsof

The lsof command lists open files and the processes using them. By files, I mean both regular files and files that represent other non-files, such as network resources. (Ward, 2014)⁠

udev

udev is a device manager for the Linux kernel. (“udev,” n.d.)

udev manages devices nodes in the /dev directory and also handles all user space events raised when hardware devices are added or removed from the system. Term clarification: user space is the space of the user (different from the kernel space, which only the kernel can access) and you can think of events as messages that are sent when certain actions happen on the system.

UUID

UUID, short for Universally Unique Identifier, is a type of serial number used to identify filesystems. (Ward, 2014)⁠

Displaying a list of mounted filesystems

To display a list of mounted filesystems, use the mount command without any arguments. (Shotts, 2019)⁠

An example:

mislav@mislavovo-racunalo:~$ mount

sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)

proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)

udev on /dev type devtmpfs (rw,nosuid,relatime,size=4001344k,nr_inodes=1000336,mode=755)

The format of the listing is as follows: device on mount_point type filesystem_type (options) .

Repairing filesystems

In order to repair a filesystem, you will most likely use a program called fsck. fsck stands for “file system check”. If you encounter this, you are dealing with a bad problem. May the force of Google and good luck be with you in resolving it!

What the fsck?

As I said in the last post, if you ever encounter fsck, that means that you are dealing with a bad issue with your filesystem. Therefore, people sometimes utter the words: “What the fsck?” when dealing with fsck. (Shotts, 2019)⁠

/etc/fstab

/etc/fstab lists the devices to be mounted at the time the computer starts (also known as boot time). (Shotts, 2019)⁠

Here is my /etc/fstab file:

mislav@mislavovo-racunalo:~$ cat /etc/fstab

# /etc/fstab: static file system information.

#

# Use 'blkid' to print the universally unique identifier for a

# device; this may be used with UUID= as a more robust way to name devices

# that works even if disks are added and removed. See fstab(5).

#

# <file system> <mount point> <type> <options> <dump> <pass>

# / was on /dev/sda2 during installation

UUID=4b74cfa0-d49e-4b15-848b-7dd92f41b018 / ext4 errors=remount-ro 0 1

# swap was on /dev/sda3 during installation

UUID=8f39c51a-de75-466c-91a2-f3bf06b5fa89 none swap sw 0 0

The rows that begin with a # are comments and are to be ignored. The first column is the device (identified by its UUID), the second is the mount point, the third is the file system type, the fourth is the options, the fifth specifies if and when a filesystem is to be backed up with the dump command and the sixth is the order in which filesystems should be checked with the fsck command (they are checked at boot (computer start up) time).

md5sum – What is it used for?

Let’s talk about something that you may encounter sometime. It’s the md5sum command. md5sum is a hash function.

Hash functions are relevant because for different inputs they produce different outputs. Even a slight difference in the input will produce massively different output. You can, for example, use the md5sum command to verify that your file is the same file on the website available for download by calculating the md5sum of your file with the md5sum command and comparing it to the md5sum of the original file (available somewhere on the website where you downloaded the file). If those two match, you can be very confident that you have the same file. If those two do not match, then you have a different file than the original.

An example:

mislav@mislavovo-racunalo:~/Linux_folder$ md5sum aba.txt

51ecc1d1bf86853c0c5c6863f403c10c aba.txt

That’s it for this post.

Hope you learned something useful!

A convenient cheatsheet:

A reader suggested adding this Linux command cheatsheet in order to help everyone out. The cheatsheet contains more than just device-related commands. It could serve as a good reference to people using the Linux command line, so I put it here.

References

Shotts, W. (2019). The Linux Command Line, Fifth Internet Edition. Retrieved from http://linuxcommand.org/tlcl.php. Page 29; Pages 208-210; Page 220; Pages 206-208

udev. (n.d.). Retrieved February 12, 2020, from https://en.wikipedia.org/wiki/Udev

Ward, B. (2014). How Linux Works: What Every Superuser Should Know (2nd ed.). No Starch Press. Pages 47-48; Pages 172-174; Pages 76-77