In the following articles, we will talk about command types (not all commands are of the same type), some file-related commands and searching for files.
Searching for files is the thing you will use most often, but it pays to know what command types are there and how to see the difference between two files, for example. Make sure to pay attention to searching for files and do read through other content, but again, it isn’t going to be of that much importance.
What we covered in the previous post are so-called basic regular expressions. There are two things, for those of you who are interested, to look into (alongside references for each):
POSIX character classes (“POSIX Bracket Expressions,” n.d.)
Extended regular expressions (“Understanding Regular Expressions,” n.d.) (this reference covers what is covered in this article, as well as extended regular expressions)
I have personally never used regular expressions in grep because I usually want to find out if a file contains some particular word. However, I have used regular expressions in programming languages (like Python) to clean up my input, so I think you learned something useful because you will be able to use the basic regular expressions if the need arises and you have gotten an introduction to something you will probably use from time to time if you decide to write computer programs.
Regular expressions are symbolic notations used to identify patterns in text. (Shotts, 2019)
The more complex explanation of what regular expressions are goes into theoretical computer science (more specifically, automata theory) and is way out of scope for this post (and I would have to review the stuff I learned in my Introduction to the theory of computation course). But, what I will say is just this – people have found some clever mathematical ways of describing patterns in text. In order to find out more about the development of the idea of regular expressions, have a look at the Wikipedia history entry here: (“Regular expression,” n.d.)
So, regular expressions help you find patterns in text. That’s their usage.
The time has come to become acquainted with regular expressions. To repeat, regular expressions allow us to match patterns in text. (Shotts, 2019) It is important to note that regular expressions differ from shell globbing (wildcards), since shell globbing is related to the shell and regular expressions are used to match patterns in text on a much broader level (regular expressions are used in programming languages, for example, while wildcards are only used in the shell). More explanations can be found in (“Regular expressions VS Filename globbing,” n.d.) and (“Globbing and Regex: So Similar, So Different,” n.d.)
Let’s first list the special characters in regular expressions, then show them applied to a couple of examples:
Any character is denoted by .
* denotes zero or more characters; a* means zero or more characters a (“Basic Regular Expressions: Kleene Star,” n.d.)
Anchors are denoted by ^ and $; they denote beginning and the end of the string pattern we are matching, respectively
Bracket expressions are denoted with [] – if ^ is the first character in the bracket expression, we treat it as a negation (meaning match everything except the thing in the bracket expression); if ^ is not the 1st character in the bracket expansion, it is matched literally
- denotes a range in a bracket expression; if it is the first character, it is matched literally, if not, then it denotes a range; you can have multiple ranges (as in [A-Za-z] if you wanted to capture all the letters)
Let’s look at a couple of examples. We will use grep, because grep’s name is actually globally search a regular expression and print (“grep,” n.d.). So grep was actually about regular expressions all along! If this was a mafia movie, grep would now get shot and thrown in the sea by the docks. Anyway, let’s get back to our examples:
[A-Z] says find any line that “has letters A to Z in it”. The second grep call (with the regular expression [AI])says “find any line that has letters A or I in it”. [^A-Z] says find any line that “has a character which is not A to Z in it”. This prints every line, since every line contains lowercase letters. However, the regular expression ^[^A-Z] says “anything that does not begin with A to Z”. Since every line begins with an uppercase letter, I get no output.
I just wanted to note that it is vital to enclose regular expressions in quotes. Prefer single quotes over double quotes; take my word for it now, you will see the difference between those types of quotes later on. (“What’s the Difference Between Single and Double Quotes in the Bash Shell?,” n.d.)). Otherwise, expansion can occur in a place where you meant to pass a regular expression, because expansion occurs before the command is executed. Just remember this, but for an example refer to (“Globbing and Regex: So Similar, So Different,” n.d.).
So the shell performed its expansion and I have 3 filenames because of that. Note that there are no spaces between the commas and the next letter. If I put a space, I would get:
Today we are going to talk about shell globbing (sometimes referred to as wildcards). They both refer to the same thing (“Globbing vs wildcards,” n.d.), so I will use the names interchangeably, or just stick to wildcards since it is shorter.
Wildcards enable us to specify a set of file names using a shorthand. (Barrett, 2016) Let’s look at an example. Say I had these files:
mislav@mislavovo-racunalo:~/Linux_folder$ ls
aba.txt ab.txt a.txt cb.txt file.txt
Good. And let’s say I wanted to print out the contents of all the files whose filenames start with a. I could do so the tedious way as follows:
What did I just do here? Am I a magician? Well, not really, so let’s look at what happened.
As I stated above, wildcards enable us to specify a set of file names using a shorthand. With this particular wildcard (a*.txt), I am saying: “Give me all the filenames that start with a, have zero or more consecutive characters afterwards, and end with a .txt”. So in some intermediary step, my command looks like:
Now here is something important – the shell does all of this expansion (this is how it is called – turning a*.txt to all of the filenames) before it executes the cat command. So, the expansion of the wildcard is done before the command runs. (Ward, 2014)
Here is a list of wildcards and their meanings; the wildcard and its meaning is delimited with a dash (Barrett, 2016):
* – zero or more consecutive characters
? – any single character
[set] – any single character in the given set; [abcde] matches characters a, b, c, d and e, while [a-z] matches all lowercase characters from a to z
[^set] or [!set] – anything not in the set (both [^set] and [!set] have equivalent meaning); i.e. [^1] is anything but the digit 1
There are also some specifics:
If you want to include a literal dash in the set, put it first or last
To include a literal closing square bracket in the set, put it first
To include the ^ or the ! symbol literally, don’t put it first
Thank you for reading and hope you learned something useful!
References
Barrett, D. J. (2016). Linux pocket guide (3rd ed.). O’Reilly Media. Pages 28-30
To start a shell session, find Terminal on your Linux distribution (via the graphical user interface) and click on it. That should start a shell session, which means you can type in commands!
We will be writing some commands in the very next article, so stay tuned!
The shell, the shell… We all heard that. We know you can type some command in the shell in a Linux environment and get some output. But what does the shell actually mean?
“A shell is a program that runs commands”, says (Ward, 2014). That is basically it. You type some commands in, the shell executes them, then you get some output. Shell scripts, another popular term, are essentially just commands typed in a file called a shell script; so the shell has the same job – execute commands.
There are multiple kinds of shells. You can find more information here: (“5 Most Frequently Used Open Source Shells for Linux,” n.d.)
Before I end, it is important to note that programs like Terminal are not actually shells – they are graphical user interfaces running shell on your behalf. (Barrett, 2016) Figure 1 depicts this. This program with a graphical user interface, called a terminal emulator, interacts with the shell. (Shotts, 2019)
Figure 1 – Relationship between graphical user interfaces to the shell and the shell itself (modeled after figure on page 15 of (Barrett, 2016))
