Bash Mastery: The Complete Guide to Bash Shell Scripting

In this section, we are going to be focusing on two core concepts that you will use all the time when you are creating bash scripts: variables and shell expansions.

Variables and Shell expansions will prove to be absolutely core to everything you will do when creating bash scripts, and you will use the skills you will learn in this section time and time again.

By the end of this section you will be able to:

• Create user-defined variables to save useful information in your scripts under convenient names of your choosing.

• Use some common environment variables built-in to the bash shell that you can use to quickly access useful information about your system.

• Use shell parameter expansion to get data back out of your variables, and use some shell parameter expansion tricks to modify that data in a variety of useful ways.

• Use command substitution to work with the output of commands and save the output inside variables for later use.

• Use arithmetic expansion and the bc command to do simple mathematical calculations

  
  

In this video, you are going to learn what parameters and variables are, as well as how to create variables and retrieve values from them.

By the end of this video you will be able to:

• Create your own variables to store data under convenient names

• Use parameter expansion to retrieve that data and use it as the input to commands.

In‌ ‌the‌ ‌last‌ ‌few‌ ‌videos,‌ ‌we’ve‌ ‌been‌ ‌focusing‌ ‌on‌ ‌user-defined‌ ‌variables,‌ ‌which‌ ‌are‌ ‌the‌ variables‌ ‌that‌ ‌you,‌ ‌as‌ ‌the‌ ‌shell‌ ‌user,‌ ‌define‌ ‌in‌ ‌your‌ ‌scripts.‌ ‌

However,‌ ‌there‌ ‌is‌ ‌another‌ ‌type‌ ‌of‌ ‌variable‌ ‌called‌ ‌“shell‌ ‌variables”,‌ ‌that‌ ‌have‌ ‌particular‌ meanings‌ ‌to‌ ‌the‌ ‌bash‌ ‌shell,‌ ‌and‌ ‌that‌ ‌it‌ ‌sometimes‌ ‌even‌ ‌creates‌ ‌for‌ ‌you‌ ‌to‌ ‌make‌ ‌your‌ life‌ ‌easier.‌ ‌

In‌ ‌this‌ ‌video,‌ ‌you’re‌ ‌going‌ ‌to‌ ‌learn‌ ‌about‌ ‌some‌ ‌commonly‌ ‌used‌ ‌shell‌ ‌variables,‌ ‌what‌ ‌they‌ ‌do,‌ and‌ ‌how‌ ‌to‌ ‌use‌ ‌them‌ ‌for‌ ‌yourself.‌ ‌

By‌ ‌the‌ ‌end‌ ‌of‌ ‌this‌ ‌video,‌ ‌you‌ ‌will‌ ‌be‌ ‌able‌ ‌to:‌

• Explain‌ ‌the‌ ‌difference‌ ‌between‌ ‌bourne‌ ‌shell‌ ‌variables‌ ‌and‌ ‌bash‌ ‌shell‌ ‌variables.‌ ‌

• Explain‌ ‌the‌ ‌purpose‌ ‌of‌ ‌the‌ ‌PATH,‌ ‌HOME,‌ ‌USER,‌ ‌HOSTNAME,‌ ‌HOSTTYPE,‌ ‌and‌ PS1‌ ‌shell‌ ‌variables‌ ‌ ‌

• Access‌ ‌the‌ ‌data‌ ‌held‌ ‌in‌ ‌these‌ ‌variables‌ ‌for‌ ‌yourself.‌ ‌‌

In this video you’re going to learn some cool and highly convenient tricks you can use to manipulate the values of your shell parameters.

By the end of this video, you will be able to:

• Modify the case of your parameters

• Determine the length of a parameters

• Slice parameters to only output the specific parts you want to work with

In this video you’re going to learn what command substitution is, how to use command substitution in your scripts

By the end of this video, you will be able to:

• Use command substitution to save the output of commands in variables

• Use command substitution to use the output of one command inside another command

I hope that you had loads of fun trying out the “Improved backup script” project!

In this video, I am going to walk you through the model solution step and can also grab a copy of the official solution script from this video’s resource section.

In this video you’re going to learn how to perform calculations within a script using arithmetic expansion.

By the end of this video, you will be able to:

• Add, subtract, divide and multiply numbers within your scripts

• Perform exponentiation and modulus operations within your scripts

• Explain the impact of the order of precedence of arithmetic operators

In this video you’re going to learn how to use the bc command to handle decimal numbers in your calculations

By the end of this video, you will be able to:

• Use the bc command to execute mathematical expressions that require decimal values in your scripts

• Set the scale variable within the bc command to determine how many decimal places are provided in the output of the bc command

In this video, you’re going to learn what tilde expansion is, and how to use it and some of its more advanced features.

By the end of this video, you will be able to:

• Use tilde expansion to easily access your home directory and the directory of other users

• Use tilde expansion to easily switch between your current and previous working directories.

In this video, you’re going to learn about the different syntaxes to use brace expansion in your scripts

And by the end of this video, you will be able to:

• Use brace expansion generate text that follow particular patterns of your choosing

• Use brace expansions in commands to massively scale up their effectiveness.

Well done for completing your second section of the course all about variables and shell expansions.

The tools and techniques that you have learned in this section will come up time and time again when making bash scripts, so a massive congratulations on working through this section and getting to grips with them. You are doing great!

In this video, we are going to have a quick recap of the points that we covered in this section to make sure that you are comfortable with everything.

And also, as a reminder, you will find the cheat sheet for this section in the resources section of this video, which you can download and refer to any time to assist you with everything we have learned.

In this section I am going to be breaking down, in detail, each step that bash is doing behind the scenes to interpret the commands you give it so you can become a more intentional and more sophisticated user of the bash shell.

Now, because scripts are nothing more than multiple command lines one after the other, this section will be a game changer to your ability to write bash scripts.

After this section, everything else in the course will pretty much be a walk in the park, so I hope you’re excited!

You will learn lots of really useful skills in this section, but, as a high level goal, by the end of this section, you will be able to:

• Explain, in detail, the 5 steps that the shell goes through when processing command lines

In this video, you’re going to get an overview of the 5-step process bash uses to interpret a command line.

By the end of this video, you will be able to:

• Describe the general process that bash goes through to interpret a command line.

In this video, you’re going to learn about the three types of quoting in bash, and the different effects that these types have on how the shell processes characters on the command line.

By the end of this video, you will be able to:

• Describe the differences in functionality between bash’s three quoting methods: backslashes, single quotes, and double quotes

In this video, you’re going to learn what metacharacters are, how the shell uses them to perform tokenisation, and what the different types of tokens are.

By the end of this video, you will be able to:

• Explain the purpose of the tokenisation step

• Identify the two types of tokens and their subtypes

• Explain the purpose of metacharacters in the tokenisation step

• Explain the effect of quotes on tokenisation

In this video, you’re going to learn how the shell identifies the difference between simple and compound commands, what those commands are, and you’re also going to learn how the shell determines where commands start and end.

By the end of this video,

• You will be able to explain the difference between simple and compound commands

• You will be able to explain how the shell identifies where simple commands start and end

• And you will also be able to explain how the shell identifies where compound commands start and end

In this video, we are going to take a closer look at how the shell processes expansions

By the end of this video, you will be able to:

• List the 4 stages that the shell goes through when performing shell expansions.

• Explain the effects of precedence of the expansions on the order in which shell expansions are performed

In this video, you’re going to learn:

1. What word splitting is

2. How bash performs word splitting on the result of certain expansions

3. And how you can set the IFS shell variable to control how word splitting is performed.

By the end of this video, you will be able to:

1. Describe what word splitting is and when it is performed

2. Explain what the IFS variable is and what its default value is

3. Use word splitting and the IFS variable to manipulate the result of various expansions

In this video, we will be giving you an introduction to globbing and its core features

By the end of this video

• You will be able to explain what globbing is

• And You’ll be able to use asterisks, question marks, and square brackets to create basic globbing patterns

In this video, you’re going to learn how the shell performs quote removal

By the end of this video, you will be able to:

• Explain how quote removal works

• Identify what quotes the shell removes during quote removal

In this video, you’re going to learn about the various ways to perform redirections within your bash scripts.

By the end of this video, you will be able to:

• Redirect standard input, standard output and standard error to a file

• Append standard input, standard output and standard error to a file

• How to redirect standard input to choose input sources for your commands

We have covered a lot of different things in this section of the course, so now we will be visually working through the process together on some example commands to make sure that you are able to see how it all fits together.

By the end of the next few videos, you will be able to:

• Mentally walk through how the shell will process a given command line

• Explain how each of the steps of shell operation are handled for a given command line

In this video, we are going to be working through an example that should allows us to get a bit more an insight into how word splitting works in detail and how understanding the quoting rules and the bash processing flow can help you avoid frustrating syntax mistakes

Hello there you beautiful people, now in our final worked example, we are going to be throwing in some globbing and command substitution for a bit of a twist.

In this segment you will have a problem set that you can use to practice working through what you have learned about the Bash’s 5-step process for interpreting command lines.

Worked solutions are also attached.

In this video, we are going to have a quick recap of the key points in this section on How Bash Processes Command Lines so that you can make sure that you’re comfortable with everything before moving on.

I have also put a cheat sheet summarising everything that we have covered in this section of the course in the resources section for this video, which you can download and refer to whenever you need to.

This video provides an overview of this section of the course on Requesting User Input

You have probably used countless commands where you type a command name and then had to give some extra inputs on the command line to make the command work properly.

Some common examples are the “cd” command, where you have to give the command the folder you want to move to, or the “cp” command, where you have to type the file you want to copy and where you want to copy it to.

Well, what makes this possible is a feature called “positional parameters”, and in this video you are going to learn how to make your scripts use positional parameters too!

By the end of this video, you will be able to:

• Use positional parameters to work with command line arguments in your scripts

In this project you will be applying what you have learned about positional parameters to make a simple calculator!

In this video, we are going to walk you through our solution to the positional parameters project so you can see exactly how we would have gone about it.

You can also grab a copy of the official solution script from this video’s resource section.

In this video, we are going to look at the # and 0 special parameters. You are going to learn what these special parameters do, and how you can use them in your scripts

By the end of this video, you will be able to:

• Explain what special parameters are

• Explain the role of the $# and $0 special parameters

• Use the $# and $0 special parameters in your scripts

In this video, you’re going to learn how to use the $* and $@ special parameters, and how their meaning changes when you wrap them in double quotes.

By the end of this video, you will be able to:

• Use the $* and $@ special parameters to conveniently represent all positional parameters

• Explain the different meanings that these special parameters have when they are wrapped inside double quotes.

In this project, you are going to use special parameters to improve your calculator script so that it can take an unlimited amount of numbers, and more than one operator.

In this video, I will walk you through the solution for the special parameters project.

You can download the model solution from the resources section of this video.

In this video, you’re going to learn how to use the read command to get input from the user

And by the end of this video, you will be able to:

• Use the read command to ask the user questions to receive input

• Save that input in variables which can be retrieved in your script

• Use various options of the read command to expand and adjust the functionality of the command

In this project, you are going to use the read command to write a bash script that will collect information about employees at your company, and save that information to a CSV file.

In this video, I will walk you through the solution for the Read Command project.

You can download the model solution from the resources section of this video.

In this video, you’re going to learn how to create menus using the select command, and how to have the shell carry out commands using what is selected.

By the end of this video, you will be able to:

• Interpret the syntax for the select command

• Use the select command to present menus to the user to select various options

• Use these options to set variables and use these variables within your script

In this project, you will use your new knowledge of the select command to improve your switchboard script.

In this video, I will walk you through the solution for the Select Command project.

You can download the model solution from the resources section of this video.

In this video, we are going to have a quick recap of the key points in this section on User Input so that you can make sure that you’re comfortable with everything before moving on.

I have also put a cheat sheet summarising everything that we have covered in this section of the course in the resources section for this video, which you can download and refer to whenever you need to.

This video provides an overview of this section of the course on Logic

In this video, the major topic we will be covering are list operators.

You’re going to learn how to create sequential lists of commands and how you can use list operators to chain commands together

And by the end of this video, you will be able to:

• Use the “;”, “&”, “&&”, and “||” list operators to create chained lists of commands

• Explain the roles of these operators and how they differ

In this video, however, you’re going to learn how to use test constructs and the different conditional operators you can use to perform different logical checks.

By the end of this video, you will be able to:

• Use the test constructs to compare different information

• Use integer test operators to perform logical checks on numbers

• Use string test operators to perform logical checks on text

• Use file test operators to perform logical checks on files

You’ve come a long way, and now it is time to learn how to create programs that can think and decide.

In this video, you are going to learn how to construct if statements in bash.

By the end of this video, you will be able to:

• Understand the syntax for Bash’s if statements

• Use if statements to determine if a command should be run or not

• Use the elif and else keywords within your if statements to create sophisticated work flows in your scripts

In this video, you are going to learn how to create more sophisticated conditions by combining the “and” and “or” operators.

By the end of this video, you will be able to:

• Use the && operator to combine test conditions

• Use the || operator to combine test conditions

• Explain the roles and differences of each.

It’s one of those days. In this project, you will be using if statements to help you log performance information related to a recent server performance fault.

While no-one likes a server fault, on the plus side, at least it gives you a great chance to practice your new skills!

In this video, I will walk you through the solution for the if statements project.

You can download the model solution from the resources section of this video.

In this video, you’re going to learn how to create case statements to implement branching logic

And by the end of this video, you will be able to:

• Interpret the syntax of case statements

• Create specific “cases” and write commands that should be run in each of those cases.

• Create “default” cases that act as fall-backs none of the other cases match

In this project, you will be using case statements to build a script that matches city names to countries.

In this video, I will walk you through the solution for the Case Statements project.

You can download the model solution from the resources section of this video.

In this video, we are going to have a quick recap of the key points in this section on Logic so that you can make sure that you’re comfortable with everything before moving on.

I have also put a cheat sheet summarising everything that we have covered in this section of the course in the resources section for this video, which you can download and refer to whenever you need to.

This video provides an overview of this section of the course on Processing Options & Reading Files

In this video, you’re going to learn how to write a while loop in bash and how to use while loops to continue to run commands whilst certain conditions are met.

By the end of this video, you will be able to:

• Interpret while loops in scripts

• Use while loops to repeat sequences of commands whilst a certain condition remains met.

In Bash, options are the little letters that come after a hyphen that you can add to a command to affect the way a command runs.

Options are a core part of what makes the command line so powerful, and in this video, you’re going to learn how to use getopts command to allow your scripts to accept command line options too!

By the end of this video, you will be able to:

• Explain the structure of the getopts command

• Use getopts command to allow your script to run differently according to what options are provided

In this project, you are going to use a while loop to create a command-line countdown timer!

In this video, I will walk you through the solution for the While Loops project.

You can download the model solution from the resources section of this video.

In this video, you’re going to learn how to use read-while loops to iterate over the contents of a file line by line.

By the end of this video, you will be able to:

• Use read-while loops to iterate over the contents of files

• Use read-while loops + process substitution to iterate over the output of commands

In this project, you are going to use a read-while loop to process a file containing folders to be created, and create a folder for each line of the file.

In this video, I will walk you through the solution for the read-while loops project.

You can download the model solution from the resources section of this video.

In this video, we are going to have a quick recap of the key points in this section on Processing Options & Reading Files so that you can make sure that you’re comfortable with everything before moving on.

I have also put a cheat sheet summarising everything that we have covered in this section of the course in the resources section for this video, which you can download and refer to whenever you need to.

This video provides an overview of this section of the course on Arrays & For Loops

In this video, you’re going to learn how to create and work with indexed arrays.

By the end of this video, you will be able to:

• Explain the difference between arrays and regular variables.

• Create literal indexed arrays

• Add, remove, update and extract elements from indexed arrays

In this video, we are going to be discussing the readarray command, which is a great tool for automatically generating arrays.

In this video, you’re going to learn:

• How to use the readarray command to generate indexed arrays from the outputs of commands

• How to use the read array command to generate indexed arrays from the contents of files

By the end of this video, you will be able to:

• Use the readarray command to easily generate indexed arrays, without having to type the values out manually for yourself.

In this video, you’re going to learn how to write a for loop and how to use them to iterate over arrays.

By the end of this video, you will be able to:

• Use for loops to iterate over lists you write manually

• Use for loops to iterate over arrays

• Use for loops with logic to perform different actions for each element of an array

In this project, you are going to use a for loop to build a script that will grab the html headers received from attempting to access some well-known websites, and save the headers into custom-named text files.

In‌ ‌this‌ ‌video,‌ ‌I‌ ‌will‌ ‌walk‌ ‌you‌ ‌through‌ ‌the‌ ‌solution‌ ‌for‌ ‌the‌ ‌Arrays‌ ‌and‌ ‌For‌ ‌Loops‌ ‌project.‌ ‌

‌

You‌ ‌can‌ ‌download‌ ‌the‌ ‌model‌ ‌solution‌ ‌from‌ ‌the‌ ‌resources‌ ‌section‌ ‌of‌ ‌this‌ ‌video.‌ ‌

In this video, we are going to have a quick recap of the key points in this section on Arrays & For Loops so that you can make sure that you’re comfortable with everything before moving on.

I have also put a cheat sheet summarising everything that we have covered in this section of the course in the resources section for this video, which you can download and refer to whenever you need to.

This video provides an overview of this section of the course on Debugging

In this video, you’re going to learn how to use shellcheck to check scripts that you have written for syntax errors

By the end of this video, you will be able to:

• Use shellcheck to check your scripts for syntax errors

• Interpret the feedback that shellcheck provides

• Understand the limitations of shellcheck, so you remain in control and don’t trust this great tool blindly.

In this project, you are going to use shellcheck for yourself to debug and fix some broken scripts.

In this video, I am going to walk you through the shellcheck project solution step by step. You can download the fixed scripts from the resources section of this video

In this video, you’re going to learn how to interpret the structure of error messages and learn how to interpret and fix common types of errors you might face when writing bash scripts

By the end of this video, you will be able to:

• Interpret the structure of error messages

• Understand what common error messages mean

• Fix common errors

In this video, you’re going to learn:

• How to use the man, help, and info commands to find information about commands on your system

By the end of this video, you will be able to:

• Determine the difference between internal and external commands

• Determine the appropriate use cases for the “help”, “man” and “info” commands

• Be able to use each of these commands appropriately to find the information you need to write your scripts

In‌ ‌this‌ ‌video,‌ ‌we‌ ‌are‌ ‌going‌ ‌to‌ ‌have‌ ‌a‌ ‌quick‌ ‌recap‌ ‌of‌ ‌the‌ ‌key‌ ‌points‌ ‌in‌ ‌this‌ ‌section‌ ‌on‌ ‌

Debugging‌ ‌so‌ ‌that‌ ‌you‌ ‌can‌ ‌make‌ ‌sure‌ ‌that‌ ‌you’re‌ ‌comfortable‌ ‌with‌ ‌everything‌ ‌before‌ ‌moving‌ ‌

on.‌ ‌

‌

I‌ ‌have‌ ‌also‌ ‌put‌ ‌a‌ ‌cheat‌ ‌sheet‌ ‌summarising‌ ‌everything‌ ‌that‌ ‌we‌ ‌have‌ ‌covered‌ ‌in‌ ‌this‌ ‌section‌ ‌of‌ ‌

the‌ ‌course‌ ‌in‌ ‌the‌ ‌resources‌ ‌section‌ ‌for‌ ‌this‌ ‌video,‌ ‌which‌ ‌you‌ ‌can‌ ‌download‌ ‌and‌ ‌refer‌ ‌to‌ ‌

whenever‌ ‌you‌ ‌need‌ ‌to.‌ ‌

This video provides an overview of this section of the course on Scheduling And Automation

The “at” command

In this video, you’re going to learn how to use the cron service to schedule your scripts to run on a repeated schedule

By the end of this video, you will be able to:

• Understand how the cron service works

• Explain what a crontab is, and be able to use crontabs to schedule your scripts to run automatically at regular scheduled times

In this video, you’re going to learn how to use crontab.guru to create cron expressions for your crontab.

By the end of this video, you will be able to:

• Navigate the crontab.guru website

• Use crontab.guru to create cron expressions that will schedule jobs whenever you want.

In this video, you’re going to learn what cron directories are, where the cron directories are located, and how to use them.

By the end of this video, you will be able to:

• Understand the different cron directories that your system makes available

• Use cron directories to run your scripts at common times

• Create your own cron directories to bundle groups of scripts to run whenever you want

In this project, you are going to practice your new automation skills by creating a bash script that will automatically update your system, and then create a crontab entry that will schedule the script to run once per day.

In this video, I will walk you through the solution for the Cron project.

You can download the model solution from the resources section of this video.

Hello there you beautiful people and welcome to this video where we are going to be taking a look at the super awesome anacron program.

Anacron has a secret weapon that gives it a big benefit over the normal cron service, and in this video, you’re going to learn what the anarcon program is and how to use it

By the end of this video, you will be able to:

• Explain how anacron works and how it is structured

• Run your scripts using anacron

In this video, we are going to have a quick recap of the key points in this section on Scheduling And Automation so that you can make sure that you’re comfortable with everything before moving on.

I have also put a cheat sheet summarising everything that we have covered in this section of the course in the resources section for this video, which you can download and refer to whenever you need to.