Ubuntu Linux Fundamentals Linux Server Administration Basics

Ubuntu Installation Options

Ubuntu has many, many installation options.

We’ll have a quick look through them, then understand the ones that will be covered in the course.

If you go to https://help.ubuntu.com/community/Installation, you’ll be able to have a look at the possible options as we go through them.

We’ll discuss system requirements in the next lesson.

A Standard Installation involves downloading a LiveCD image from Ubuntu’s site. You then burn it to a DVD then boot to it.

A live CD lets you try Ubuntu with out installing it, then install it later if you choose to.

Alternate installations cover various methods of installing without having to boot to a DVD.

• You can install (Quick or Normal) from USB, 

• Use iso2usb, 

• Use a Smart Boot Manager, 

• Install within Windows using Wubi, 

• Install with Floppies (anyone remember or know what those are?), 

• Install from Existing Linux, 

• Install a Virtual Machine, or 

• Install a Portable installed system booting from Unified Extensible Firmware Interface (UEFI) and Basic Input Output System (BIOS)

Server and Network Installations are possible.

• Local Network

• Installation/Netboot

• Netboot Install From Internet

• Network Console

• On Network File System (NFS) Drive

• On NFS Drive with Local Boot

• Quick Install over Secure Shell (SSH)

• Install with Floppies - Again with the Floppies!

Installing on External or Redundant Array of Independant Disks (RAID) Hard Disks

Various options are supported, but will not be covered in this course.

Logical Volume Manager (LVM) Installation

Various methods are supported, but will not be covered in this course.

Minimal Installations

Ubuntu runs on a wide range of hardware, including some very old systems that may be collecting dust in your basement.

If you want to run on one of these, a minimal install may be a good option for you.

Once the installation process is started for any of the interactive methods, the process is very similar.

We’ll do an Ubuntu Server 16.04 install and an Ubuntu Desktop 16.04 install to VirtualBox. This will create a VirtualBox VM which will be used throughout the course.

I chose VirtualBox because it is free, easy to use virtualization software.

It lets you run other operating systems in an application running on your computer. 

AMD64 images should work if you’re using VirtualBox. If you have trouble with that, try an i386 based image.

You can browse to https://www.ubuntu.com/download to find the appropriate version for your needs.

Please download an LTS version for server and desktop as we’ll be using both.

Minimum System Requirements

Ubuntu runs on a very wide range of hardware, old and new.

You’ll still want a certain minimum level to make sure your system performs well.

For Desktop Installations

If you want to run Ubuntu with a full installation including the Graphical User Interface or GUI, you should have at least:

• A 2 GHz dual core processor
• 2 GiB of RAM (system memory)
• 25 GiB of free hard drive space
• A VGA or better video card with a resolution of at least 1024x768 pixels
• A CD/DVD drive or USB port for the installation image
• [Note] A CD/DVD drive or USB drive is not required for installation of Virtual Machines 
• Internet Access

[Note] Running the Unity or GNOME GUI requires more capable hardware:

• 4 GiB of RAM
• A 3D Accelerator capable video card with at least 256 MB of Video RAM 

These are the requirements as of 3 October, 2017. Please double check them to see if they have changed prior to your installation. They can be found at https://help.ubuntu.com/community/installation/SystemRequirements.

For Ubuntu Server, as it is usually not installed with a GUI, the requirements are much lower.

• A 300MHz x86 processor
• 256 MB of system memory (RAM)
• 1.5 GB of disk space
• A graphics card and monitor capable of 640x480 pixels resolution
• A CD drive

If your system won’t support the full version and you want to use Ubuntu Desktop, you could opt for a lightweight GUI alternative, Xubuntu, or Lubuntu

These only require:

512 MB of RAM

5 GB of disk space

A Graphics card and monitor capable of 800x600 resolution

If you have a system made in the last several years, you should be fine, and should be able to load the full Desktop operating system.

For practice throughout the course, you should have Ubuntu Server and Ubuntu Desktop installed in a virtualization platform. The tutorials cover installation on VirtualBox.

You should be able to accept the default system specifications during the VirtualBox install and won’t have to worry about them meeting the minimum requirements.

For installation on a local hard drive, where you want to boot to and use Ubuntu right on the system, it is good to have more than the recommended minimum if possible. This is called a “bare metal” install.

The minimum will function but may not provide the most pleasant experience.

If you easily meet the minimum requirements, please download the appropriate image for your system.

If not, you can check out the Low Memory System instructions available in the downloadable document for this lesson. 

Installing VirtualBox

You want to run Ubuntu, but you don’t have any extra hardware, and you don’t want to overwrite or dual-boot your Windows or MAC computer. What to do?

Use VirtualBox! It’s free software that lets you run multiple operating systems within an application on your existing computer.

It’s available for Windows, MAC OS X, Linux, and Solaris. That covers any operating system you’re likely to be using.

Once you’ve tried a virtual environment, installing on hardware seems slow and painful.

Many server infrastructures are virtualized, running on VMware ESXi, Microsoft HyperV, or are in the cloud on Amazon Web Services, DigitalOcean, or a similar provider.

Taking a moment now to learn VirtualBox increases your options for trying new operating systems and practicing new things yourself, and can make using virtualization software at work more understandable.

The Operating System you install the downloaded VirtualBox software to is called the Host. The systems you install are virtual machines, or Guests.

Virtual machines are often abbreviated as VM’s.

So, let’s get VirtualBox installed.

Download the appropriate package for your operating system from https://virtualbox.org/wiki/downloads.

Once it’s downloaded, browse to the file. It will probably be in your Downloads folder, unless you typically download to another location or moved it.

I’m showing the steps for MAC OS X here, but it will be similar for other operating systems.

For MAC OS X, double click on the VirtualBox…dmg file to start the installation process.

Your MAC will verify the integrity of the downloaded file.

The VirtualBox installation window will open. Double click on VirtualBox.pkg. 

Your system will verify the integrity of the VirtualBox.pkg file.

A window will open asking you to run a program to determine if the software can be installed. Click on Continue to continue the installation.

In the Install Oracle VM VirtualBox window, click on Continue.

Enter your username and password if prompted and click Install Software.

If the installation was successful, click Close to close the installer window.

Under Number 2 in the VirtualBox window, double click the Applications folder to open it and browse to VirtualBox.

Double click on VirtualBox to launch it.

You can also launch it from Launchpad.

For Windows, double click the executable you downloaded and follow the default installation prompts. Then open it as you would any other application.

On a Windows install, consider right clicking the downloaded executable and installing it as Administrator. I haven't had it happen myself, but I had one student report grayed out permissions when he was trying to select a certain network setting. Installing as Administrator will minimize your chances of encountering this error.

You’ll see how to install new virtual machines and work with them in upcoming lessons

Installation - 32-bit vs. 64-bit

In this lesson, we’ll discuss 32-bit and 64-bit images, why they’re different, and which is best for you.

Ubuntu .iso images are available in 32-bit and 64-bit versions.

What They’re Talking About

When discussing the number of bits, we’re talking about the amount of data a processor can grab at once. 

There are some considerations addressed under the relevant processors below.

32-bit

Grabs data in 32 bit “chunks”

Can support up to 4 Gigabytes of memory unless you have a Physical Address Extension (PAE) kernel

64-bit

Grabs data in 64 bit “chunks”

Ubuntu documentation says Ubuntu supports 16.8 million Terabytes (16 exabytes) but constraints are in place that limit this to around 1 Terabyte.

Performance

All things being equal, which they almost never are, performance of a 64 bit system should be about twice as fast at a given task as a 32 bit system.

There are significant caveats though.

To take advantage of the extra available processing power, applications have to be written to do so.

Virtualization Settings and BIOS

In order to run VirtualBox, or other virtualization software such as VMware Workstation or Player for Windows, or VMware Fusion for MAC OS X, you may have to change some BIOS settings to make virtualization, or 64-bit VM’s work properly, or at all.

I wish I could walk you through an example, but the steps depend very much on what BIOS you’re running. 

Please have a look in your BIOS for Virtualization related settings. They will often be visible under the Advanced tab.

You’ll want to enable virtualization support and save your settings.

If you don’t find these settings easily, Google for virtualization support for your computer make and model.

For This Course

For the lessons contained in this course, you can do all the activities with either 32-bit or 64-bit versions of Ubuntu. 

If you have difficulty getting 64-bit installed as a VirtualBox VM, please post in the Q&A for the course or try downloading and installing 32-bit and seeing if that works.

Bottom Line

You should always opt for 64-bit operating systems for servers that may expect a high amount of traffic over the Internet in their normal operation. 

For lighter weight applications, or when making use of old, otherwise obsolete hardware, 32-bit is still an option.

A Domain Name System (DNS) Server for a small network, for example, may not require robust hardware and could be run on an old 32-bit system, but you may want to upgrade if for no other reason than reliability. Your users will be unable to do much of anything network related without a functioning DNS Server.

Desktops vs. Servers

Let’s discuss Ubuntu Desktop and Ubuntu Server.

Why are there two versions? What’s the difference between a Desktop and a Server?

Time to learn.

The Desktop operating system is geared toward individuals, end users, or family members. It has a full Graphical User Interface or GUI, and all of the features you’d expect of a home or work computer. It has word processors, spread sheet software, web browsers, ways to play music and videos, and much more.

The Server operating system is for providing services to many people. It could be a web server that presents web pages to visitors. Or, a database server that stores bank account information, or a file server that stores your files in a networked, multi-user environment.

The hardware for a desktop is designed for a person to use. When we say Desktop Operating System here, we also include laptops.

You’ll have a monitor with great resolution, a keyboard, a mouse or touchpad, speakers, a camera, a microphone, and all of the things you’d expect to find on a system you use every day.

There isn’t, however, much redundancy, in the form of redundant hard drives or power supplies. If your hard drive dies on your laptop or desktop, it could be catastrophic, unless you regularly back up to the cloud or an external drive.

Servers don’t provide any bells and whistles you’d be interested in as an end user. The do, however, have redundancy built in. You can get Servers with multiple hard drives configured so that, if one drive dies, the others will pick up the slack, until you can get a replacement installed. They also come with the option of multiple power supplies. If one power supply dies, it will run on the other, until it is replaced.

Desktops are designed for individuals, or at least one person at a time, and servers provide for the needs of many.

The primary difference between Ubuntu Desktop and Ubuntu Server, is that Desktop comes with the features end users want, and Server is stripped of those, but provides features Server Administrators want.

We’ll be covering both, but understand that anything that can be done at the command line on a Server, can generally can also be done on a Desktop, once you reach the Terminal window. 

The converse is not true. I don’t know anyone who would recommend installing a GUI on an Ubuntu Server, unless you have a very good reason to do so.

Let’s install some Operating Systems!

Ubuntu Linux Fundamentals

Installation - Downloading the ISO

The files you’ll need to install Ubuntu are commonly assembled into a package called an iso image. Iso images are packaged in a way that they would be if they were put onto an optical disk such as a CD or DVD.

ISO stands for the International Standards Organization. It’s the body that comes up with all kinds of standards including the file system format for optical disk images. That’s ISO 9660 in case you are remotely interested. It may also contain other files commonly used by DVDs and Blu-ray Discs.

The nice thing about an operating system that has been packaged in such a way is that you can boot to it and install an operating system from it.

Another nice thing about iso images is that they do not have to be put onto optical disks to be usable. You can fool your computer into believing it’s loading from an optical disk when it’s actually loading from the downloaded image file directly or from a USB device.

To download the image you’ll need, go to http://www.ubuntu.com/download/server.

Choose the latest Long Term Support (LTS) image. As I write this that is 20.04.1. For you it may be 20.04.something. If you’re downloading after 2022, it will be 22.04.something.

Keep track of where the file is downloaded and its name as you’ll need these in the installation lesson for your installation method of choice.

I strongly encourage you to try Ubuntu out as a virtual machine using VirtualBox which will be explained in a later lesson.

Clicking on Download started downloading the file ubuntu-20.04.1-live-server-amd64.iso. The image is fairly large, about 812 megabytes, so it may take quite a while to download depending on your Internet speed. Over an hour for my horrible Internet connection!

Be patient and let the download complete.

You’ll then have the iso file you’ll need for upcoming lessons.

More Information

ISO Image Files

https://en.wikipedia.org/wiki/ISO_image

Ubuntu Server Download

https://www.ubuntu.com/download/server

Ubuntu Linux Fundamentals

Ubuntu Server Installation

Ubuntu 20.04 LTS Server on VirtualBox

In this lesson, we’ll do an Ubuntu Server 20.04 installation on VirtualBox.

Ubuntu 20.04 is the latest version as of this lessons creation. It was released in April of 2020.

In the previous lesson, you downloaded the latest Ubuntu Server 20.04 Long Term Support (LTS) .iso file. In this lesson, you’ll use that to create the server virtual machine (vm) you’ll use throughout the course.

Open VirtualBox then click on New.

Create a name for your server, and select Linux for the Type and Ubuntu (64-bit) for the Version.

Click on Continue.

I usually set my vm's to 2048 MB (2 Gigabytes), but you can leave the Memory size at default (1024 MB here) if you like or change it to 2048 and click Continue.

Leave Create a virtual hard disk now selected and click Create.

Leave VDI (VirtualBox Disk Image) selected and click on Continue.

For Storage on physical hard disk, leave Dynamically allocated selected and click on Continue.

Click on Continue.

Leave File location and size at default, unless you have to store in another location, and click on Create.

Your Server VM is created and waiting for Ubuntu Server to be installed.

Right click on your new server and select Settings.

Click on Storage.

Under Controller: IDE click on the Empty CD/DVD icon.

To the right, next to IDE Secondary Master, click on the disc icon and select your disk image. If it doesn’t show up there, you’ll have to browse to it by clicking Choose Optical Disc file and finding your downloaded image.

If you didn’t already have it downloaded, you can pause here, download it from ubuntu.com/download, then continue.

Click on OK.

Double click on your Server VM to start it.

Keep English selected, or select your language by scrolling up or down using the arrow keys, then hit Enter.

Under Keyboard configuration, leave English selected, or select your language, then hit Enter.

Leave Install Ubuntu selected and hit Enter to continue.

Leave the network connections at default and hit Enter on Done.

Leave Proxy address blank and hit Enter on Done.

Leave Use An Entire Disk selected and hit Enter.

Leave the default value in Choose the disk to install to: and hit Enter to continue.

Leave Done selected for Filesystem setup and hit Enter to continue.

Hit the down arrow under Confirm destructive action to select Continue and hit Enter.

Enter your name, hit Tab, enter your server’s name, hit Tab, and continue down the form.

Be sure you enter your password, then enter the same password to confirm it during that step.

When you get to Done at the bottom of the form, hit Enter to continue.

Software installation will commence based on the settings you specified earlier.

Once installation completes, hit Enter with Reboot Now selected to reboot into your newly installed OS.

When the text finishes scrolling, hit Enter after Please remove the installation medium.

Login with the username and password you supplied earlier.

My first step upon installing a server is to update the server. Here’s a one-liner that will do that for you:

sudo apt update && sudo apt upgrade -y && sudo apt full-upgrade -y

You will be prompted for your password to run the command with sudo privileges which will be covered in greater detail later on the course.

You’ll usually have to remove old kernel versions after an initial update, so I run sudo apt autoremove if needed after updates.

Now you’re ready to learn how to configure and maintain your server!

Additional Resources

Official Ubuntu Server Guide

https://help.ubuntu.com/lts/serverguide/introduction-chap.html

CD/DVD Installation Methods

In this lesson, we’ll discuss what you need to do a hardware install of Ubuntu.

When you download the files you need to install Ubuntu from the web site, they come in the form of an image. An iso image.

An iso image is basically an archive file of an optical disk (CD or DVD). In order to use the image, you have to burn it to a CD or DVD, or copy it to a USB in a certain way. You can’t simply copy the image like you would any other file, and have it work.

For burning to a CD or DVD, you’ll have to use a utility specifically for image burning.

For Windows:

Windows Disk Image Burner - Windows 7 has one built in. You can just right click your .iso file and select Burn Image.

For other versions of Windows, you’ll have to install your own. 

Please go to https://ninite.com and download an .iso burner from their Utilities section. You could use ImgBurn, CDBurnerXP, or InfraRecorder. I’ll use ImgBurn in this tutorial.

Be sure you DON’T download any of those from other sources. If you’re not very, very careful, you’ll be getting malware packaged in with the file you download.

Files hosted on Ninite are malware free.

Select ImgBurn (and any other software you find interesting) on Ninite’s main site.

Click on Get Your Ninite.

The files you chose will be bundled into one clean installer file for you and downloaded.

After that’s complete, double click on the file to install your application(s).

If prompted, enter your credentials to complete the installation.

Once you’ve installed ImgBurn, open it. 

In a Web Browser, browse to ubuntu.com/download and select Server or Desktop (latest Long Term Support (LTS) release) and download your .iso image if you have not already done so.

Bring ImgBurn up again and select Write image file to disc.

Click on the icon with the folder and magnifying glass under Source and browse to your downloaded file. I’m using an Ubuntu Server image in this example.

Insert a writeable CD or DVD into your burner. 

Click on the image to CD icon near the bottom of the ImgBurn window.

I was prompted to format the DVD. If you are prompted to format, use caution, as you may be overwriting previous data if you have a writeable CD or DVD.

After the burn, ImgBurn will attempt to close the drive after it was ejected, so it can verify that the burn was successful. In my case, since this was done on a laptop, this will not work. I have to manually close the drive so the verification can happen.

Close the drive and click on OK.

ImgBurn will make sure there are no errors on the burned CD or DVD.

Click on OK for Operation Successfully Completed and label your CD or DVD with a permanent marker if you want to keep it.

Close ImgBurn.

If you browse to the newly created CD or DVD, you’ll see the file structure created in the burn.

It includes a boot folder, which will be used during boot, and other folders the system will look for as it installs Ubuntu on your computer.

MAC OS X

On MAC OS X, the process is similar, but you’ll use MAC specific software to burn the image.

I don’t have a burner, so I can’t test this, but there’s a link to an OSXDaily.com article below that says it can be done natively in MAC OS X El Capitan, and I assume newer versions of OS X.

Please find the link below and let me know if you have any issues.

Before you can use the image you created to install Ubuntu from, you’ll likely have to make a change to your BIOS which will be covered in the lesson after USB Drives.

Next, we’ll learn how to do the same process with a USB Drive.

USB Installation Method

In this lesson, we’ll discuss how to install Ubuntu from a USB Drive.

If you recall, from the CD/DVD installation lesson, you can’t just copy an .iso file to a storage medium, boot to it, and install Ubuntu.

You have to use software designed to make the .iso usable.

The software for creating a bootable USB from an .iso is different from what was used for the CD/DVD lesson, but other than that, the process is quite similar.

Windows

For Windows, you’ll be downloading an application called Rufus to create a bootable USB from the Ubuntu .iso file you downloaded earlier.

Open a web browser, and go to rufus.akeo.ie.

Scroll down to the Download section, and click on the link to the latest version of Rufus (2.17 at the time I’m making this).

Double click the downloaded file to run Rufus.

You’ll need a 2 GB or larger USB drive to install your .iso onto.

It should be free, or you should be OK with formatting it, which will remove all data stored there.

Again, be sure you’re OK with losing all data on the USB drive!

Plug the USB device you intend to use into your Windows computer.

Rufus will detect your USB drive and fill in information about it.

Under Create a bootable disk using, click on the folder and locate your downloaded .iso file. 

Click on it, then click on Open.

Rufus will fill in a new volume name for the device based on the image name provided.

You may, or may not get the additional warning I received stating that some files would have to be downloaded from the Internet. I clicked on Yes to download.

Leave Write in ISO Image mode selected and click on OK.

Be sure you’re OK with deleting all the data on the USB drive, then click on OK to continue.

The creation of your bootable USB will commence, and the USB will be made into a bootable device you can use to install Ubuntu from it.

I find it a little confusing for such an app, but Start will still be highlighted when the image is done. The status bar just above it will say READY. Just click on Close to close Rufus.

Browse to your newly created bootable USB device, and you’ll see the file structure looks very similar to that on the CD/DVD you burned in the last lesson, if you decided to complete that.

In the next lesson, we’ll see how to change BIOS settings if needed, to boot to CD/DVD or USB for a hardware install.

MAC OS X

The MAC OS X process is similar, but will use Etcher instead of Rufus.

Browse to etcher.io and download the installer.

After the download completes, double click on the installer to open it.

Follow the instructions, adding the installation file to your Applications folder.

Open it by going to Launchpad and finding it, or browsing to it in your applications folder.

You’ll get a warning saying Etcher.app was downloaded from the Internet. Are you sure you want to open it? Click Open.

Click on Select Image, then browse to your downloaded .iso file.

If you don’t already have it, you can get it by going to ubuntu.com/download.

If you have a USB Drive plugged in, Etcher will detect it. Use caution here! If you have multiple USB Drives plugged in, whichever one you choose will be overwritten! 

I strongly recommend having only one USB Drive plugged in at this time, and that is the one you want to format for use as your bootable device.

If you don’t have one plugged in, you’ll be prompted as shown here.

Plug in the device you want to use, remembering that your device will be erased and any information on it will be lost.

The device will be detected, and you can click on Continue.

Click on Flash, and the drive will be made bootable and have the Ubuntu .iso installed on it.

Enter your password if prompted.

The image will be installed, then validated.

You’ll get a message saying that the disk is not readable by your mack. Click Eject to safely remove the USB drive.

You’ll likely have to make BIOS changes to the target system before it will boot to the USB device.

Making the required BIOS changes is covered in the next lesson

Installation - Making BIOS Changes

In this lesson, we’ll discuss how to make the required changes to BIOS to boot to CD/DVD or USB Drives.

While some systems may be set to check for bootable CD/DVD’s or USB’s on boot up, many aren’t.

If you had your CD/DVD or USB device plugged in, but your system booted as normal, ignoring the new image you created, you’ll have to change your BIOS settings to proceed.

BIOS stands for Basic Input/Output System. It’s a set of configuration settings that are read to boot your computer.

It tells the computer how to do things like find and load hard drives, load system devices, and where to check for the bootable operating system.

The specific steps you’ll take (what key to press on boot up to access the BIOS settings, for example), and the way you’ll navigate your BIOS’s menu to get to the settings will vary with manufacturer and motherboard maker.

The basic idea of finding the boot order and modifying it to have your new image recognized, though, carries across all systems.

I’m doing this procedure on a Dell laptop, as you’ll see from the boot screen.

I hit F12 to enter BIOS settings during boot.

If you miss it (it can go fast), just shut down and try again.

After hitting F12, I’m taken to a screen where I could, if I chose to, just select a one time option to boot from CD/DVD. 

This option may not be available on your system and doesn’t show you how to boot to USB, so I’ll scroll down to BIOS Setup and enter the more advanced settings available there.

You’ll see that I have an OK looking Graphical User Interface for my BIOS, but that may not be the case for you. You may have to navigate a text based menu with arrow keys and select with your space bar.

After clicking on BIOS Setup, I’m taken into the BIOS Setup screen. I apologize for the resolution. I’m capturing an actual BIOS screen with a video camera.

Under General, I use the arrow key to get down to the Boot Sequence menu item.

I can add a check mark to USB, or CD/DVD here.

Be sure you leave Internal HDD checked along with whichever other item you want selected.

If you uncheck Internal HDD, your system will always want to boot to the USB or CD/DVD.

Here, I click on CD/DVD, and Apply, then Exit. My changes will be saved and the system will reboot, booting to the media I specified first.

You can see, the system boots to the CD/DVD image burned in the previous lesson, and from there, the installation process is the same as it was on VirtualBox.

Next, I show the process for changing BIOS to boot from USB.

It’s the same, but select the USB checkbox.

That’s it! Enjoy your Ubuntu installation.

Server - Installation - Partitioning

We accepted all the defaults when installing our Ubuntu server regarding the layout of the hard drive. We accepted the sizes and mount points recommended by the installer.

For many installations, letting the installer choose the size, based on the available drive space, works fine.

However, there are times when it is best to define the sizes and locations of some directories yourself. We’ll cover some of those in this lesson.

When There’s Risk

When the default partition scheme provided by the installer is used, the entire hard disk is made part of the root file system.

As any directory grows in size, it takes away from the free space available to all other partitions, including those needed for the system to function.

Once the file system in the root partition is out of space, the system can no longer perform critical functions, like starting new processes. 

It will eventually fail, hanging or rebooting your system in an unstable state. 

This can happen intentionally to create a Denial of Service (DoS) for the services your server provides, or accidentally, when a process performs in an unexpected way, or a user simply adds too much data.

The risk of this happening is relatively low for most server implementations, but we’ll cover some instances where you may want to set certain directories on separate partitions.

You can check out your partition layout anytime with the command

sudo parted -l

Recommended Minimums

When deciding on a custom partition scheme, be sure you don’t take so much of the available space away from other partitions that they’ll dip below their minimum recommended size.

Other partitions may be sized according to the function(s) of the server and the space available.

Server Types

Some of the types of servers you may want to protect with separate partitions are, web servers, file servers, and shared access servers. We’ll cover each below.

Web Servers

Some directories are used more heavily by web servers than others. /var/www and /var/log are two examples. 

/var/www is fairly static and should not be a cause for concern.

/var/log can grow quickly as web traffic increases.

Properly setting up log rotation for your log servers averts most of the risk here, but if you want added safety, you can mount /var/log on its own partition.

If /var/log fills up and it’s on a separate partition, the web services may crash, but the operating system will still be up so you can log in and remedy the problem.

File Servers

File servers, as the name suggests, are set up to allow a number of users to store and share files with others, or for their own centralized repositories of their work.

You may mount the place the data is stored under any name, but I suggest using one that makes sense, and has its own partition.

The size will depend on the amount of space available on the drives the system has access to.

Shared Access Servers

One example of a shared access server is one that serves students doing thinks like taking Linux courses at a University.

The administrator there will want to make sure student directories are on a separate partition. The administrator should also implement quotas on how big users home directories can get.

You’ll want a big /home directory mounted separately.

Security Recommendations

The Center for Internet Security (CIS) has security benchmarks available for free for many systems, including Ubuntu 16.04 LTS. The link is provided in the More Information section below.

These are excellent tools in helping you harden your servers.

One recommendation is setting up separate partitions for several directories. Those are:

/tmp

/var

/var/log

/var/log/audit

/home

The benchmark also recommends setting certain options like nosuid, nodev, and noexec on some of those partitions.

Please see the CIS Benchmark for your operating system for detailed information.

Understand Your Server

We’ve only covered a few of the possibilities here.

If you’re spinning up a server, understand the type of work it will be doing, and what areas need protection by learning ahead of time, or learning as you go.

You can modify your partition scheme at any time if you installed using Logical Volume Manager, as I recommended in the previous lesson, and always recommend.

Great work!

Server - Shutting Down

So far, so good, but how do you power down or reboot from the command line?

There are, as with many things Linux, several options including halt, shutdown, and with systemctl commands.

My preference is the shutdown command.

Shutdown and Power Off Now

To use the shutdown command, you have to specify what you want the server to do when the operating system is halted, and a time when you want the action to occur.

sudo shutdown -h now

will halt the operating system gracefully, then power the system off, and run the command immediately.

The -h option is needed to power the system off after the operating system stops.

now means the command should be run immediately. You can specify a wait time here if you want instead.

Shutdown and Power Off Later

To shutdown at a time in the future, use the + sign immediately followed by the number of minutes option.

sudo shutdown -h +<time in minutes>

So, to shutdown in 10 minutes from now,

sudo shutdown -h +10

Since Linux is a multi-user operating system, you can send a message to other logged in users letting them know when the server will shutdown so they can save their work.

sudo shutdown -h +10 <message>

For example:

sudo shutdown -h +10 "Shutting down in 10 minutes!"

You can cancel a scheduled shutdown by using the -c option.

sudo shutdown -c

Reboot

Rebooting is similar.

sudo shutdown -r now

You just specify the -r option for reboot.

When To Shut Down

As with Windows, you should not simply power off a running system by hitting the power button, whether you’re running a physical system or running on a virtual platform like VirtualBox or VMware.

Telling the operating system you want it to shutdown gives it a chance to do house keeping and gracefully shutdown processes, and to shutdown only when that is complete.

If you just power off, you may find it in an unstable state when you reboot.

Always do a shutdown whenever possible before powering off.

The rare exception being when the system will not respond to any input, and you have to power cycle it to recover.

Ubuntu Server - Welcome to Ubuntu Server!

In this lesson, we’ll explore navigating the Ubuntu Server command line and using the help system.

I chose to start with a few commands and the help system, because you need that understanding to do much of anything in Linux Server.

That little blinking cursor waiting for input, with no visible help of any kind can be intimidating!

Have no fear.

You’ll be hopping around and entering commands in no time.

Most Used Commands

There are a few commands you’ll use over and over in a given day of Ubuntu Server administration. We’ll cover the top ones, as well as have a look at the help system that’s always available to you.

pwd

How do you know where you are in Ubuntu?

If you have your prompt set at default, the working directory is part of your prompt.

That’s changeable though. Also, if you’re several levels deep in the directory structure, it can become truncated.

To know where you are at any given time in Linux, you can type pwd for print working directory.

ls

If you want to see what’s in a directory, type ls, for list directory contents.

ls and some common options will be covered more shortly.

cd

To move from the directory you’re in to another, you use the cd command, for change directory.

Practice

For practice with the three commands you’ve learned so far, do the following:

• Change directory to /etc

  • cd /etc

• List the contents of the /etc directory

  • ls (ls with nothing entered after it will list the content of the present directory)

• Print your working directory

  • pwd

Help

Well, look at that! Just three commands, and you can see where you are on the system, change to different directories, and list files.

What if, though, you wanted to see something other than what’s available by just typing ls in a directory?

For example, if you have many files listed, and aren’t sure what one you want, but you know the date it was created, how could you see that?

There are two types of help in Ubuntu that can assist you.

Linux Help

One is the —help system. (that’s two dashes or minus signs together).

Type ls —help and hit Enter.

From here on, I won’t say to hit Enter unless it’s unclear when you should do so. It will be implied.

Wow, the help scrolls by pretty quickly.

We’ll cover redirecting output with the pipe | in a later lesson. For now though, type ls —help | less.

This sends the output to a command called less, letting you scroll up and down to read the content.

When you enter a command in Linux, it can often be followed by an additional option that makes it behave differently.

These can be referred to as options, switches, or arguments.

Scrolling up a ways, you can see the -l (lower case l) option. It says it will display a long list format.

Hit q to quit using the less command.

Back at the prompt, type ls -l.

Now, you can see many fields, including the date the file was created.

Manual, or man pages

Linux’s —help system is unique to Linux, and may not be available on all Unix-like operating systems, and may not be available for all commands.

The more universally accepted way to find help in Linux is with the man pages.

Think of it as a huge stack of documentation, organized and made available to you by typing a quick command.

Type man ls.

The output is also a help system, but it’s formatted a little differently.

There’s the Name and what it does

A Synopsis telling you you could type ls options and a file name. The items in square brackets are optional.

and a Description with a sentence then a list of the options.

With some commands, like ls, there can be a huge number of options, so you may have to scroll around a bit to find the one you’re interested in.

You may also come across some you hadn’t though of that can be very useful.

When you’re done, you can type q to quit the man page.

So, now you know some commands to get you started, pwd, cd, and ls, and two ways to get help in Linux.

More Practice

Check out the man pages and help systems for the commands used in this lesson. What did you find when you tried to bring up the man page for the cd command? How about when you used —help?

Next, we’ll gain some understanding of what a shell is and what some more abut the BASH shell.

Ubuntu Server - Shells and the BASH Shell

In this lesson, we’ll look at what Shells are, what the BASH shell is, and why they’re important.

The Linux Shell

You have to interact with the system somehow, and the Shell in Linux is the most common way to do that.

The Shell is a special program that is you interface to the kernel. It lets you type commands in, and have them interpreted and executed by the kernel or the operating system.

There have been many shells developed in the years since Unix was invented.

We’ll look at some of the main ones now.

shell

Ken Thompson developed the first Shell, called the V5 shell in 1971.

It had functionality we would recognize, but lacked some features we consider basic today, like command completion.

It was intentionally minimalistic to keep it small. Resources, like memory and drive space, were scarce in the beginning.

Bourne shell

The next major Shell developed was the Bourne shell, released in 1977, by Stephen Bourne.

The Bourne shell let programmers and administrators use many of the standard programming constructs in use, including control flows, loops, and variables. This means scripts could become much more powerful and could act like, and call actual programs.

There are many other shells including the Korn Shell (ksh), by David Korn, the tcsh for TENEX C shell, by Ken Greer, developed in 1975, and merged into the C shell in 1981, and the  Bourne Again Shell or BASH, which will be the focus of the rest of this lesson.

One of the goals of the BASH shell was to grab much of the useful functionality of the other shells and wrap them in to one “super shell”.

Unless you’re using a system that doesn’t have BASH, I know of no reason not to use it.

Behind the Scenes

So, what happens when you type a command, like ls into the shell?

When you hit Enter, your the Operating System searches for it in several places, and if it is found, executes the command.

It will start with the functionality the command invokes, then look for any options and arguments, as applicable for the command.

When you type ls, the system looks up the command, finding it in the /bin folder, and runs it.

You can see the places the system finds a command using the which BASH command.

If you type which ls, you’ll see that ls is found in /bin.

The same process is used for any command you type.

We’ll go much more into the possibilities in the following lessons.

Server - Introduction - Man Pages

Since the man pages are on almost every Linux system, and they’re a well established, authoritative source of help and information, we’ll be having a closer look at them in this lesson.

man’s man page

A logical place to learn more about the manual system is the man page for man.

man man

We see the standard breakdown of a man page, with NAME, SYNOPSIS, DESCRIPTION, and OPTIONS.

There is also an OVERVIEW, and DEFAULTS section for man’s man page.

Manual Sections

Although it’s normally transparent to you, because you’ll likely just type man and the command you’re interested in, behind the scenes, there are sections to the manual.

Sometimes, a command is found in multiple sections, and you may want the man page entry for a particular section.

The sections are broken down as follows:

1   Executable programs or shell commands

2   System calls (functions provided by the kernel)

3   Library calls (functions within program libraries)

4   Special files (usually found in /dev)

5   File formats and conventions eg /etc/passwd

6   Games

7   Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7)

8   System administration commands (usually only for root)

9   Kernel routines [Non standard]

Most often, just typing man and the command you’re interested in will provide what you want. 

If you’re a developer creating some lower level interactions with the system, you may want the man page entry from section 3, Library calls, or section 9, Kernel routines.

Most, if not all of what we’re going to be doing in this course will come from section 1, Executable programs or shell commands.

If you’re curious to see what sections a command may be found in, you can type

man -a <command>

and you’ll be taken to each of the sections it is in.

man -a passwd

--Man-- next: passwd(1ssl) [ view (return) | skip (Ctrl-D) | quit (Ctrl-C) ]

--Man-- next: passwd(5) [ view (return) | skip (Ctrl-D) | quit (Ctrl-C) ]

Before each of the lines above was shown, you would have been taken to the respective man page entry for that section.

If you do want the man page entry from a particular section, when it is in multiple sections, you can just put the section number in parenthesis after the command.

Sections Within man Pages

Inside each man page, there are common sections defined:

NAME,  SYNOPSIS,  CONFIGURATION, DESCRIPTION, OPTIONS, EXIT STATUS, RETURN VALUE, ERRORS, ENVIRONMENT, FILES,  VERSIONS, CONFORMING TO, NOTES, BUGS, EXAMPLE, AUTHORS, and SEE ALSO.

Searching For a man Page

If you know part of the command you’re looking for, or what it relates to, you can search the man pages by using the -k option.

man -k group

Produces anything with the word group in the description.

man -k pass

Returns anything with the term pass in the description including passwd, if you had been looking for that but couldn’t remember if it was passwd or password you were hunting for.

man page Navigation

When you open a man page, it is displayed by a “pager” which is a program that shows you text one screen or “page” at a time, and lets you scroll forward or back, typically with the up and down arrows, and the page up, and page down keys to scroll more quickly.

You can also search within a man page by typing /<search term> where <search term> is the information you’re looking for.

Within the man page for ls, you can search for the term recursive to see what options you have for telling ls to search the directory you’re in plus subdirectories.

/recursive

Takes you to

-R, --recursive

       list subdirectories recursively

so, to do a recursive search, you can use the -R option or type —recursive. I favor the option that involves less typing, but you may want to spell things out sometimes for clarity.

apropos - An Alternative to -k

You can also search the man pages for an entry by using the apropos command (the results will be the same as what man -k <command> would return)

apropos passwd

Returns

chgpasswd (8)        - update group passwords in batch mode

chpasswd (8)         - update passwords in batch mode

gpasswd (1)          - administer /etc/group and /etc/gshadow

grub-mkpasswd-pbkdf2 (1) - generate hashed password for GRUB

mkpasswd (1)         - Overfeatured front end to crypt(3)

pam_localuser (8)    - require users to be listed in /etc/passwd

passwd (1)           - change user password

passwd (1ssl)        - compute password hashes

passwd (5)           - the password file

update-passwd (8)    - safely update /etc/passwd, /etc/shadow and /etc/group

Great work! Now you know how to understand and navigate the built in man pages.

Case Sensitivity

In the English language, capital letters are called upper case, and small letters are lower case.

If you’re a Windows user just getting started with Linux, you may be confused by the fact that Linux is case sensitive, while Windows is not.

Windows

In Windows, the file named “ThisFile”, a file named “thisfile” would be considered the same. In fact, if they are in the same directory, you could not even create a file called “thisfile” if there were already an existing one named “ThisFile” in that directory.

When you’re searching for “ThisFile” in Windows, you can look for “ThisFile”, “thisfile”, “ThIsFiLe”, or anything variation and get the file you’re after.

The same is not true in Linux.

Linux

In Linux, you have to specify case when naming or searching for files. 

“ThisFile” and “thisfile” are seen as different, and can reside within the same directory.

This is convenient and intuitive once you’re used to it.

You may find yourself forgetting about case sensitivity and searching for “thisfile” when you want to find “ThisFile” at first though.

We’ll experiment with this now.

We’ll start by moving into the home directory.

cd ~

We’ll then create three files.

touch ThisFile thisfile ThIsFiLe

Now we’ll do some searching.

find ThisFile

Which one is returned?

Yes, just the one named ThisFile.

Recall that, in Windows, you would not even have been able to save those files to the same directory or folder.

Making Search Case Insensitive

You have to add an option when searching to ignore case (find a file of a given name regardless of case).

We’ll address this more in a later lesson, but you can have commands used to find files ignore case with the -iname option.

find -type f -iname “thisfile.txt"

means find the object of type file, and conduct a case-insensitive search.

Returning:

./ThisFile.txt

./thisfile.txt

./ThIsFiLe.txt

Commands

Commands are also case sensitive in Linux.

ls and Ls are not the same. Ls will not be recognized as a BASH command.

Please keep this in mind when you wonder why a search isn’t returning the results you expect or a command isn’t running when you think it should in Linux. 

Check spelling and case.

Ubuntu Server - The Super Special root User

In this lesson, we’ll look at what the root user is, why the root user is so special, and how to work with root permissions.

The root user

All Linux systems, and for that matter, Unix, and BSD systems, have a root user. The root has god-like power over your system. It can do ANYTHING!

If you’re doing good things, and not making any mistakes, that’s fine.

If, however, you make a critical error when you’re working as root, the consequences can be devastating to your server.

We’ll look at a simple command to illustrate the potential problems that can ensue if you make a mistake as root.

The rm command removes a file, or set of files, or set of files, and directories you specify.

rm removes files. With the -r option, it will remove the present directory and all sub-directories. With the -f option, it will force the deletion, even if it would otherwise have resulted in an error.

If you wanted to remove something from your home directory, and you were logged in as a regular user, you could have a situation like the following:

rm -rf /home/theo/somefile

due to a typo becomes

rm -rf / home/theo/somefile

You were in a hurry, and you somehow added a space after the / or root directory.

The shell understands rm -rf but it then sees / which it also understands and home/theo/somefile, which it sees as an argument it can’t interpret.

bash does exactly what it’s supposed to do, and starts deleting everything it can from /.

Since my privileges aren’t elevated, it will only be able to delete files I have the access to remove.

Not pleasant, but not catastrophic for the system as a whole.

What do you think happens if this is done as root?

Please watch the lesson to see.

The moral of the story is to make sure you work as a non-root user unless you have to elevate your privileges for some reason.

When you do have to, use the sudo command if it will get the job done.

Only on rare occaisions work as root, and do it by typing sudo su -, not by enabling login for the root account.

Also, be sure any critical systems are backed up!

Ubuntu Server - Getting Started - Common Commands

In order to unleash the true power of Linux, you have to get comfortable working at the command line. 

In this lesson, we’ll learn some of the most common commands, so you’ll understand when you see them in future lessons, and you’ll be comfortable working with them on your own.

cat

cat stands for concatenate. To concatenate something is to link or chain parts together. cat streams or prints the output of whatever file you ask for to the Terminal.

To use cat, type cat followed by the file you want to look at:

head

head - Show’s the first several lines of the file you specify (10 lines by default).

tail

tail - Show’s the first several lines of the file you specify (10 lines by default). 

pwd

pwd - Print working directory. Shows your present location within the file system.

cd

cd - Change directory. Moves you from the directory you’re in to the one you specify.

ls

ls - List the files in the present directory.

find

find - Searches the file system for files or directories you specify.

locate

locate - Searches a database of the files on your system for a file you specify.

grep

grep - grep stands for global regular expression print. You tell grep what to search for and the directory or file to search, and it prints all of the lines that have a match.

more

more - Lets you pipe output from another command (like cat) into it so you can scroll around in it and look at things.

less

less - It was supposed to be an improved version of more, but now they’re quite similar. However, more does let you use your arrow keys to scroll, which I really like.

touch

touch - Although its intended purpose is to change the modification date of a file, it can also be used to quickly and easily create an empty file of the name you specify.

That’s a lot to take in in one lesson! Please practice each command, navigating around your Ubuntu system and looking at things.

If you don’t know where to start, /var/log is a good place. You’ll have to use sudo privileges to look at some of the files there.

Bear in mind, there are entire lessons coming up on some of these commands, like grep.

You don’t have to be expert with these yet. Just familiar.

Ubuntu Server - Getting Started - BASH Shell Navigation

There are some small, but very helpful tools built into the BASH Shell. 

Whether you find yourself managing hundreds of Linux Servers or just running your personal blog on one, these few tools will save you quite a bit of time.

command history

When you type a command and hit Enter in BASH, the command you use is stored in the .bash_history file in your home directory.

This can be useful information, that can keep you from having to type a previous command you want to use again, or to help you jog your memory about how you did something.

To access this information, though, you don’t have to work with that file itself.

You can just hit the up arrow to see previous commands and the down arrow to scroll down in the command history.

This may not sound super useful, but trust me, it’s great. As you start using Linux more, you’ll understand the benefit.

tab completion

BASH will also try to figure out what you want to do and will complete commands, file names, and directory names for you.

If you type cat /var/log/au and hit tab, BASH will auto-complete auth.log for you.

If you have your computer sound enabled, you’ll hear a bonk kind of sound which indicates there are multiple possible completions, and it has shown you the first.

You can either hit Enter, to run with the file you have, or you can hit Tab again, to see the options available.

In my case, I hit Tab again and see that I have the following possibilities:

auth.log    auth.log.1

I can either hit Enter to see auth.log, or I can type a period, then hit Tab again to autocomplete to auth.log.1, which is an older log file created by log rotation.

Tab completion also works for changing directory.

If you type cd /var and hit tab, you’ll get the bonk sound. Hitting Tab again displays all of the sub directories in the /var directory.

If you go to a directory with a huge number of sub directories in it, you’ll get a warning asking you something like “256 results found. Do you want to display all 256?” You can type Y to display them all or N to refine your search further.

BASH asks because it is unwieldy to have a large number of options shown at once at the command line.

cd ../..

We already learned how to change directory. Here’s a short cut for changing directory upward from your current location.

Let’s cd into /etc/skel.

To go up one directory, we can just type cd ..

If you type ls -al in any directory, you’ll see, at the very top of the list “.”, then “..”. The . represents the current directory, and the .. represents the parent directory, or the directory that’s one level above your current location.

You can also string ../../.. together to go up several directories at once (you’ would go up three levels here).

Ubuntu Linux Server

Relative Vs. Absolute Path

When you are navigating the file system, there are two ways to specify the path to a file or directory you’re interested in.

Although this topic may be quickly covered in other lessons, I feel it deserves a quick lesson of its own.

Relative path is the path relative to where you are now. You specify a relative path by not preceding the file or directory you’re interested in looking at with a leading /.

Absolute path is the full path to a file or folder. Absolute paths start with a leading /.

Let’s look at some examples.

If you type cd and hit Enter, or cd ~ and Enter, you’ll be in your home directory.

If you want to look at what’s in the /var directory, and you’re currently in your home directory, and you type ls var, what happens?

You’ll get an error saying:

ls: cannot access 'var': No such file or directory

That’s because there is no var directory inside your home directory. You were looking for the directory in the root of the file system called var, or /var, but bash was searching within your present working directory, or your home directory.

Now, if you type ls /var, what happens?

Yes, you get the content of the var directory listed:

theo@ubuntu-18-04:~$ ls /var

backups  cache  crash  lib  local  lock  log  mail  opt  run  snap  spool  tmp  www

Let’s make a var directory in our home and see how to distinguish between the two.

cd

mkdir var

Now, checking our location with pwd, we see that we’re in our home directory.

We can then type ls var and we get an empty return because there’s nothing in /home/theo/var, or my home directory. But, we don’t get an error as before.

Now that /home/theo/var exists, how can we look at the var folder in the root of the file system or /var?

To see what’s in /var instead of /home/theo/var, specify the absolute or full path with the ls command.

ls /var reveals the contents of the var folder in root as before.

Likewise, if you’re in the root of the file system and you want to see what’s in /home/theo/var, you just type ls /home/theo/var, or navigate to /home/theo by typing cd /home/theo, then typing ls var.

This is important to understand when you’re navigating the file system, but will be more important in your scripting. Often, you’ll assume a script is running from a particular directory, and everything will work fine in your testing. Then, you’ll publish your script and someone will run it from a different directory, and something will break.

One of the first things to look at is whether you’ve specified a relative path where you should have used a full path.

You can get more information by looking at any error messages you script generated, but it is a common mistake.

Also, this is a reminder that it is important to look for error messages as you work. If you’re trying to manipulate a file or folder and you get a file or directory not found error, check where you are with pwd and make sure you’re using an absolute or relative path as applicable.

Now, you understand that a relative path is a path that is relative to your present working directory, and absolute path is the full path from the root of the file system.

References

Ubuntu Forum post - Absolute vs Relative Path

https://ubuntuforums.org/showthread.php?t=1444892

Ubuntu Server - useradd

In this lesson, we’ll look at useradd and how to add a new user with it.

useradd

useradd is a command that will work across Linux and BSD versions, so it’s more universal than the adduser command we’ll look at in the next lesson.

To create a user with useradd, you have to specify several arguments upon creating the account.

You’ll almost always want the user to have a home directory, based on his or her username, commonly found in the /home directory.

The command to add a user here, is:

sudo useradd -d /home/dtrump -m dtrump

Looking in the /home directory, we can see that the sub-directory of dtrump was added.

Now, before Mr. Trump can log in, we have to add a password to the account.

As root, or with sudo, you can set the password for other peoples accounts.

The command to do this is passwd.

To do this, type:

sudo passwd dtrump

You’ll be prompted to enter your password, then confirm it.

Nothing will appear on the screen when you’re typing the password. This is normal.

If they match, you’re all set.

Ubuntu Server - adduser

In this lesson, we’ll look at adduser and how to add a new user with it.

adduser

adduser is not in all distro’s. It is in Ubuntu and Debian. I’m not sure what other dirstro’s have it, but in some distro’s it is simply a link to useradd.

In Ubuntu, it is a Perl script that, in the process of setting up a new user, calls on the useradd binary.

It is a human readable file.

It does several steps for us in logical order, saving some typing, which is always a good thing.

Ubuntu Server - usermod

In this lesson, we’ll learn how to modify users with the usermod command.

usermod

What if you have to modify a user after creation? usermod is made for that.

You can assign a user to an additional group, lock the account, unlock the account, move the home directory, or set an expiration date for the account, to name just a few of the possibilities.

In this lesson, we look at how to add a user to the sudo group, giving the user sudo permissions and how to lock and unlock a user account using usermod.

Ubuntu Server - /etc/passwd

In this lesson, we’ll learn what the /etc/passwd file is, how it relates to users, and how it’s set up.

The /etc/passwd File

A key file for managing users and logins on your Ubuntu Server is the /etc/passwd file. 

It’s Ubuntu’s go-to source of information for user accounts.

If the file becomes corrupt or unreadable, you’ll have a very difficult time logging into the system.

You may be able to reboot into single-user mode and repair your /etc/passwd file, but let’s not go there unless you have to.

Here’s one line from the /etc/passwd file on my test server:

lskywalker:x:1002:1002:Luke Skywalker,1,555-1212,555-1213,Jedi Knight:/home/lskywalker:/bin/bash

The line is a set of fields separated by colons “:”

1. Username. 

2. Where the encrypted password used to be (it is now stored in /etc/shadow)

3, User ID - generally ID’s for users start at 1000, and can go up to 65535.

4. Group ID

5. Full name of the user, room number, work phone, home phone, other information

6. Home directory 

7. Shell to be loaded for the user

I specifically filled in those requested items when making this practice user.

Most often, you’ll only assign the username, hitting Enter for the rest when using the adduser command.

A more typical entry would look like this:

theo:x:1000:1000:theo,,,:/home/theo:/bin/bash

Given the issues you could have if you edit by hand and them up, editing yourself is not recommended. Use the following programs to help avoid corruption of the files:

vipw for editing the /etc/passwd file

vipw -s for editing the /etc/shadow file

vigr for editing the /etc/group file

vigr -s for editing the /etc/shadow file

Ubuntu Server - /etc/shadow

The /etc/password file, but itself, cannot perform the function of permitting user login on modern Linux systems. It has to be coupled with the /etc/shadow file.

The /etc/shadow File

The /etc/shadow file, paired with the /etc/passwd file, permits users to log in. The system checks the entered password against the value stored in /etc/shadow, and if it’s right, the user is permitted to log in. If not, you can try again. Only a few more times, though if an account lockout is set. 

Here’s a line from the /etc/shadow file:

lskywalker:$6$7AGLK73G$wCV11kWNLz2a/zWUZH5coRvTKP48VQOluVJo0MHN7SdmQW7JFibGfnYQxP89V3PWXHWDQR5qOmNDnpoIvCnv./:17473:0:99999:7:::

As with the /etc/passwd file, the line is a set of fields separated by colons “:”

1. Username (lskywalker). 

2. The encrypted password.

The encrypted password consists of the following fields:

a. $6 - This value could be a number from 1 to 6, and it signifies the encryption level used.

$1 = MD5

$2a = Blowfish

$2y = Blowfish - With correct handling of 8 bit characters

$4 = sha-256

$6 = sha-512

b. $7AGLK73 - This is the salt (after the $) used to create the encrypted password.

c. $wCV11kWNLz2azWUZH5coRvTKP48VQOluVJo0MHN7SdmQW7JFibGfnYQxP89V3PWXHWDQR5qOmNDnpoIvCnv./ - The encrypted password

3. Last password change date (days since 1 January 1970). Weird way to calculate it. (17473 here)

4. Minimum password age (0)

4. Maximum password age (99999) ~274 years! In effect, it never expires.

5. Number of days before password expires to warn the user. (7)

6. Normally blank, but if filled in, it will indicate the number of days after the password expires until the account is disabled.

7. Expiration - Days from 1 January 1970 that the account will be disabled on. An expiration.

If you look at the file, you’ll notice many users with an * in the password field, as in the entry below:

games:*:17379:0:99999:7:::

For those accounts, the password is not set, so that account cannot be used to log into the system.

Remember, to edit the /etc/shadow file, which you probably shouldn’t do manually anyway, give yourself a little protection by using the vipw -s command.

Ubuntu Server - Removing Users With deluser

Deleting users is easy with the deluser command, but there are some other considerations, like do you want to delete the home directory, the associated group, and all files the user owned on the system too?

We’ll cover the use and some of the possibilities in this lesson.

deluser

There is another program that can be used to remove users, called userdel. 

Here’s the top of the man page for it:

USERDEL(8)                                 System Management Commands                                USERDEL(8)

NAME

       userdel - delete a user account and related files

SYNOPSIS

       userdel [options] LOGIN

DESCRIPTION

       userdel is a low level utility for removing users. On Debian, administrators should usually use deluser(8) instead.

Like adduser and useradd from earlier lessons, the more admin friendly option is deluser, which we’ll cover here. You’re welcome to try userdel if you want.

We’ll do as the man page recommends and go with deluser.

Some of the options we may want are:

—remove-home

—remove-all-files

—backup

—backup-to DIR

If you regularly add and remove users and want to make sure their accounts are backed up on removal, use the —backup and —backup-to DIR options, and backup to a directory you specify.

Ubuntu Server - Groups - Overview

In this section, we’ll be working with groups.

Groups in Linux are used as they are in other operating systems, to control access to resources like files, directories, and permissions to run applications or make system changes.

We saw a brief example with the sudo command in an earlier lesson. 

To give a non-root user sudo permissions, the easy way is to add the user to the sudo group.

In coming lessons, we’ll look at the primary group assigned to a user and what that means, secondary groups, and adding users to and removing them from groups.

You’ll learn to change group ownership of files and folders with the chgrp command.

Ubuntu Server - Groups - Primary and Secondary Groups

In this lesson we’ll have a look at what a primary group is and what secondary groups are and how to manage which group is primary.

When a user creates a file or folder, it is given a set of permissions.

There is rwx for read, write, and execute for the user who created the file, for the group the user is in, and for all other people who may interact with the file.

Doing ll, or ls -l on a file in the user’s home directory will generate output like the following:

-rw-rw-r-- 1 theo theo    0 Nov 10 17:22 GroupPermissions.txt

The first field, permissions, can be broken down as follows:

-|rw-|rw-|r--

1| 2 | 3 | 4

1 - Indicates type - Could be Directory, regular file, as in this case, symbolic link, etc.,

2 - The next three characters, rw- are the permissions the user who created or owns the file has on it. In this case, r and w for read and write. The third character, the -, would be replaced by an x if this were a file that could execute, like a binary file or a shell script. It’s a text file, so that is not set for this file.

3 - The next three characters, rw- again, are for the group that is assigned to the file. This means the group has read and write access, like the file owner.

4 - The third three characters, r-- are for all other users, so they can read, but not write to the file.

Continuing on in the output, the theo after the number 1 is the username of the user who owns the file.

After that is the group that owns the file. theo again here.

These are the properties we’re interested in for this lesson.

Primary Group

You can see your primary group by typing the groups command.

The first one listed is your primary group.

When theo created the file above, the group that was used to assign group permissions was theo’s primary group, theo in this case.

When you create a user with adduser, the user is assigned to a primary group during account creation. 

adduser creates a group with the same name as the user and makes that the primary group for the newly created user.

Anytime theo creates a file, unless he changes the group afterward, it will be assigned to the theo group.

This is fine for a user who works alone, mostly in his or her home directory, but may not be ideal for someone who often works with others and shares files with them for work.

We’ll discuss some reasons for changing this behavior and how to do it later in the lesson.

Secondary Groups

Along with the primary group, users can have any number of secondary groups.

We’ve already discussed the sudo group.

On Ubuntu, using its default configuration, any user added to the sudo group, has sudo permissions and can run any command as root.

Much more granularity is possible. 

We could make a Help Desk Group, and give them permission to look at /var/log/auth.log and /var/log/syslog with read only permissions, for example, but that’s beyond the scope of this lesson. 

For permitting access to files and folders, the three permissions for the group that has been granted access are read, write, and execute. rwx.

Changing Primary Group

You can change a user’s primary group by using the usermod command. You would type usermod -g <group name> <user name>

The command to add a user to an additional group, while not changing the primary is

usermod -aG <group name> <user name>

Most of the time, you will want to add a user to a group without changing their primary group.

Practical Application

How can you actually use group membership to control access to to resources?

An often used example is that of a Finance group.

You could have a bunch of people in Finance. You make them a Finance folder, create a Finance group, and put them all into it.

They can then go into the Finance folder, create documents and sub-folders, and share and edit them.

Please see the next lesson for a walkthrough of this process.

Ubuntu Server - Groups - Controlling Access

In this we’ll have a learn how to control access to folders or files based on group membership.

Sharing a Folder

You’ll find any number of answers when you ask the question “Where should I keep files or folders I want to share between users on a Linux system?”

Some possibilities are:

/var/share

/var/local/share

/home/share

/share

/srv -for files that may be shared using a service externally.

For this lesson, we’ll go with /home/rebel-alliance

mkdir /home/rebel-alliance

Creating a New Group

We have our new folder, /home/rebel-alliance.

Now, let’s add a group so we can add it to the folder, then assign users to it.

Use the command

addgroup rebel-alliance

To add the new group to your system.

Giving the Group Permissions on the Folder

We have a folder, and we have a group. Time to marry the two.

Use the chgrp command for this.

First, look at what the owning group is:

ls -l /home/rebel-alliance

and it’s root for the group owner.

chgrp rebel-alliance /home/rebel-alliance

ls -l to confirm the change.

Adding Users to the New Group

We’ll add users to the group with usermod.

Type usermod -aG rebel-alliance lskywalker

This adds Luke to the group rebel-alliance.

Do this for any users you want to be able to work in that folder and leave out ones you don’t.

Adding the Group to Files

If Luke creates a file, say, plans-for-death-star.txt and wants someone else to see them, but no one outside the group, he does this with the chgrp command.

Make the plans.

cd /home/rebel-alliance

touch plans-for-death-star.txt

This creates the file, but it still owned by user lskywalker and group lskywalker.

To change the group, type chgrp rebel-alliance plans-for-death-star.txt

 Confirm that it worked as expected.

ls -l plans-for-death-star.txt

Now, you can add other users to the group, using sudo, of course, like Leia Organa, but leave out ones you don’t, like Darth Vader. Hopefully, this is a Rebel Alliance controlled server, and no Federation people have sudo permissions!

Ubuntu Server - Editing Text Files - Overview

As you learn to manage your server, you’ll find that you’ll often be editing text files.

Text files are the most common way to manage settings in Linux.

So, how do you edit a text file at the command line?

We’ll look at two methods.

vim - Vi IMproved

vi is a venerable text editor that has been around since the earliest versions of Linux, and before.

vim is a pure text editor, with no menus. This sounds like a disadvantage, but I hope you’ll come to see it as a great feature once you give vim a try.

I use vim exclusively in maintaining Linux systems.

You’ll learn the different modes of operating in vim, how to find, and edit text, and how to save your file.

nano

nano is a quasi-text editor, with a GUI-like menu available. I’m personally not a big fan, but I know many Linux admins I respect greatly like and use it.

Please give both a try, and get comfortable using whichever one you prefer.

There are also many people who use GNU Emacs, but I have never tried it.

Ubuntu Server - Editing Text Files - vim

The first editor we’ll work with is vim.

There are literally whole books written on using vi and vim. I have the book “Learning the vi Editor, Sixth Edition” by Linda Lamb and Arnold Robbins, O’Reilly Press which, sadly, I’ve never completed.

We’ll hit on enough here to let you quickly and easily edit files with vim, and this is likely all you’ll need!

As mentioned in the previous lesson, vim stands for Vi IMproved. vi is an editor that you’ll find on almost any Linux, BSD, or Unix system. That makes it pretty universal.

If you want to edit a text file, you’re almost guaranteed to be able to use vi to do it.

The thing is, vi does not behave the way you would think it should. The up arrow may not work at all, or may not take you up a line in the file you’re editing. You use h, j, k, and l instead of arrow keys to move around.

vim is more user friendly in that regard.

vim is available on all Linux systems, although you may have to install it as a package on some. It’s in the default installation for Ubuntu Server.

vim - Vi IMproved

To use vim, you just type vim, then the file name of the file you want to open for editing. 

vim my-first-text-file.txt

If the file exists, you’ll be in it and ready to edit. If it doesn’t, it will be created and you’ll be able to edit it in vim.

Two Modes

vim has two modes of operation, Insert mode, and Command mode. 

When in Insert mode, you can write text into the file as you would in any text editor.

When in Command mode, you can efficiently navigate your document and manipulate text.

You enter Insert mode by typing the letter “i”. You leave Insert mode, going back to Command mode by hitting the “esc” key.

Insert mode

Once you open a file, you’ll be in Command mode. To be able to edit the file in the way you would in any text editor, you type “i” to enter Insert mode.

In Insert mode, you can use the arrow keys to go to the area you want to edit. You then hit the Del key to delete something that’s there now, or just type what you want inserted into the file.

To save your work, you hit the esc key to leave Insert mode, then type a colon “:” and type w then q to write your work to the file and quit.

Try editing a file using just what you learned above. 99% of the time, that’s all I need to edit a file.

Command mode

Command mode is where you do things like search within a file, or save and exit.

To search for text, hit esc to make sure you’re in Command mode, then type a forward slash “/“ followed by the text you want to search for.

If there are several instances of the word you’re searching for, you hit “n” for the next instance or “p” for the previous instance (searching forward with “n” and backward with “p”).

To save your work, as mentioned above, you hit esc to make sure you’re in Command mode, then type “:wq” to write and quit.

Typing “:q!” quits without saving, in case you change your mind about saving your edits.

Try editing some files with just these commands. 99% of the time, they’re enough.

If you want to get a bit more proficient, try the things below.

Command mode Advanced Usage

Here are some more advanced navigation techniques for those interested.

Using Letters Instead of Arrow Keys

In Command mode, you can navigate more quickly with some keyboard shortcuts.

Moving the Cursor

h, j, k, and l can be used instead of the arrow keys.

• h moves the cursor one character to the left.
• j moves the cursor down one line.
• k moves the cursor up one line.
• l moves the cursor one character to the right.

Remember though, the arrow keys function as you would expect, with right arrow moving to the right left to the left, up moving up one line, and down moving down one line. Holding a key moves multiple steps, like normal with arrow keys.

Moving to Different Words

Use w, b, and e to navigate to the start of the next word, the end of the word, or beginning of the word.

• w moves you to the next word.
• b moves you to the beginning of the previous word
• e moves you to the end of the current word

Beginning and End of Line

• o moves to the beginning of the line.
• $ moves to the end of the line.

Beginning and End of File

• G moves to the end of the file.
• gg moves to the beginning of the file.

Finding Your Last Edit

• `. moves to your last edit. That first character is a back-tick, usually located under the tilde “~”, above the Tab key on North American keyboards.

Putting a number before a command, will make that command repeat that number of times. To go ahead five words, you would type 5w.

Command Mode Editing

Learning a few editing commands for use in command mode can speed your editing up greatly.

Deleting, Undoing, and Redoing

• dd deletes the current line.
• dw deletes a word.
• do deletes to the beginning of a line.
• d$ deletes to the end of a line.
• dG deletes to the end of the file.
• u undoes the last operation (don’t we all need that from time to time!)
• Ctrl-r redoes the last thing you undid with u.

Searching and Replacing

/<text> searches for the thing specified in <text>.

n finds the next instance of the word searched for.

N moves the cursor to the previous instance.

?<text> searches upward in the file for the thing specified in <text>. 

:%s/<text>/<replacement text>/gc does a search for <text> through the entire document, replacing it with <replacement text>. It will ask you to confirm before replacing.

Those few commands are more than enough to do most tasks quickly, easily, and efficiently at the command line.

Please practice. 1/2 hour of playing and practicing can yield incredible results, and start you on your path to becoming a command line guru.

Ubuntu Server - Editing Text Files - nano

Now, we’ll learn to work with nano.

Like vim, volumes could be and have been written about nano.

We’ll cover some basics to get you started.

nano - Nano’s ANOther editor

nano is also a text based editor, but it has a menu system to walk you through its commands.

It also makes use of hot keys, keys paired with the Ctrl key. Ctrl-G (pressing and holding Ctrl, then pressing G), for example, brings up the Help system for Nano.

Where you see M referenced in the menu and help files, it is referring to the Alt key.

Editing Text

Editing text in nano is super simple. Just open your file by typing nano <filename> and start moving around with the arrow keys, using delete to delete text, and typing in what you want to add.

Saving Changes

Ctrl-O writes your changes to the hard drive, and Ctrl-X exits.

nano Help

Ctrl-G or F1 accesses nano’s help system.

Searching For Text

Ctrl-W, or F6 for Where Is, will let you find text you specify.

Alt-W, or F16 for find next occurrence. (My F keys only go up to F12, so I’m not sure how I would use that).

Cutting And Moving A Line

Ctrl-K or F9 Cuts a line.

Ctrl-U or F10 Pastes a Line.

Wrapping Text

Alt-Shift-$ Enables word wrap. For lines that are very long, they won’t wrap by default.

Moving To Beginning Or End Of File

Alt-\or Alt-| Go to the beginning of the file.

Alt-/ or Alt-? Go to the end of the file.

Finding And Replacing Text

Ctrl-\

Type your search term          [press Enter]

Type your replacement term [press Enter]

A    - replace all instances, or be prompted at each instance found.

Ubuntu Server - Editing System Files

Since you now know how to edit text files with vim and nano, I want to help you avoid some pain and suffering by showing you how you can keep the original of any system file you may edit, and subsequent versions as you modify it going forward.

The Pain of Experience

I wish I could tell you I read this somewhere, or learned it in a course as you are, but no. I modified a configuration file on an nginx (web) server and didn’t have the original or a previous version to fall back on.

If it were a simple edit or two, no big deal, but the editing was extensive, and having saved and exited, I couldn’t use the undo feature.

To get an original, you may be able to just copy from another server, or a clean build, provided it is running the same version of software, but this takes precious time.

You should also have backups of any production server, but again, restoring from backup or finding a file in backups takes time.

Simple Versioning

Short of using a full repository for your files, like git, how can you keep the original and versioned changes to your system files?

The solution is simple. Before you edit, make a copy!

Keeping Versions

If I wanted to edit /etc/ssh/sshd_config, for example, I’d first make a copy of it by typing sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.0. That would make a copy of the file in its original location with a .0 at the end. That tells me it is revision 0, or the original file.

If I want to make changes to the present copy of /etc/ssh/sshd_config, I don’t overwrite sshd_config.0, I create a new file, sshd_config.1, and so on for further revisions. 

If you find your directory getting cluttered, you could delete some of your older versions.

I always keep the original, and the latest change though.

Why The Extra Work?

If you edit a file, which you will do on your Ubuntu Server, you may later find that your edit creates some unforeseen issue and want to look back at what you changed.

If you edited the file, and don’t have a record of what you did, this can be a challenge.

If you don’t have the original, you can’t restore to that and start fresh.

If you do, you can use a program called diff to see what changed from the original, or if you have them, later versions, and the working copy.

How Does diff Work?

diff tells you what’s different between two (or more) files. I find the output confusing if used on more than two files, but it can be done.

You type diff <file1> <file 2>.

diff /etc/ssh/sshd_config.0 /etc/ssh/sshd_config

will show you the differences between the original copy of /etc/ssh/sshd_config (sshd_config.0 in this examplel) and the working or current version, /etc/ssh/sshd_config.

Armed with that information, you can troubleshoot more quickly by undoing your changes one at a time, or restoring to a previous version, and implementing the changes one at a time.

How To Restore?

To restore to a previous version, you just use the mv command.

mv moves a file, but if you move a file to an existing one, the existing one will be overwritten.

If you want to keep the present version, with the possible errors in it, for troubleshooting, just save it as the latest revision first.

Then, type mv /etc/ssh/sshd_config.0 /etc/ssh/sshd_config. Substitute .0 for the revision you want to restore to.

Please practice these steps as you go through the course, whenever we edit system files.

You’ll likely see me do it, and remind you, when applicable, both because it’s a habit for me, and because it’s a habit I want you to get into.

Ubuntu Server - Linux Filesystem - Overview

Now we’re going to have a look at the Linux Filesystem.

There are literally thousands of directories, sub directories, and files on a Linux installation.

How do you know where your stuff should go?

We’ll cover the basics, which should get you through typical use cases, and also where to find more information if you need it.

There are two high level sources we’ll discuss in the following lessons.

hier Man Page

Ubuntu is one of the Linux distributions that has a hier (for hierarchy) man page.

You can type man hier at any time and search for the directory you’re interested in knowing more about.

To search for information on the /opt directory, for example, you would open the hier man page, and type the / then the text you’re interested in.

/opt

There are many instances of opt (as in optional), so hit n until we get to the one we’re interested in, /opt.

If you want to go only entries with “/opt”, you can type

//opt

/opt is the directory you would use to install a program that is not part of the managed package system.

When we install the Lynis security auditing tool, in a later lesson, for example, it will go here.

The Filesystem Hierarchy Standard

You can also turn to version 2.3 of the Filesystem Hierarchy Standard (FHS). 

Version 2.3 was released in 2004, so this isn’t a very active project, but it is a resource you can use for looking up how things should be done.

Bear in mind, Distro’s are not required to put things in any specific directories, so you will see some variation between them.

That said, the general layout should be intelligible, once you understand where things belong.

Ubuntu Server - Linux Filesystem - Manual Installs

Now that we have an understanding of the filesystem, we’ll learn where to install custom software.

When you install an application, if you’re using APT, the location of the files to run the application will be chosen automatically by the package management tool, as dictated in the package file.

When you install an application, hundreds or thousands of files and many directories may be added to your system.

For applications that aren’t handled by your package manager, I usually put them in /opt, especially if they are small and self-contained.

We’ll cover one example of why you may want to install an application manually instead of using the packaged version that’s included with Ubuntu.

Lynis Open Source Auditor

There’s an excellent free tool to help you audit your system for security best practices called Lynis. 

There is an APT package for it, which we’ll install by running:

apt update  (to make sure your database of available software is up to date) then:

apt install lynis

We’ll cover using Lynis to secure your system in a later lesson. For now, please just have a look to follow the lesson.

After installing, run Lynis by typing:

lynis audit system

Notice the warning saying our installation of Lynis is out of date.

This is important for this program because newer versions have more or different security checks they do.

Why is it out of date? We updated our database and downloaded the latest version available from the standard repositories.

There’s a lag between when most software is released or updated, and when that latest version is available directly from Ubuntu’s repositories.

People have to package it, and test it prior to release.

To take advantage of the checks available in the latest version, we’ll install our own copy manually.

Downloading to /opt and Installing Manually

An alternative method is to install Lynis manually.

You can just download Lynis to /opt, extract it, and run it from the /opt directory with the command:

./lynis audit system

First, cd to /opt

cd /opt

Next, download the file you’re after from cisofy using a free, built in tool, called wget.

wget https://cisofy.com/files/lynis-2.5.7.tar.gz

This downloads the lynis-2.5.7.tar.gz file to the /opt directory.

Please use the latest version of Lynis available to you at the time you’re doing this.

Go to https://cisofy.com/downloads for the current version.

Next, extract the lynis program files with tar:

tar -zxvf lynis-2.5.7.tar.gz

You’ll see a new directory in /opt named lynis. cd into lynis:

cd lynis

Run your newly downloaded, up to date version of Lynis by typing:

./lynis audit system

The reason I did it this way, instead of downloading the packaged version is to keep it clear to me where I have to go to run or update Lynis.

I don’t want to be confused by what is managed by APT, and what is managed by me.

Now you know one use for the /opt directory and one way to handle installing apps yourself.

Ubuntu Server - Files - Permissions

In this lesson, we’ll have a closer look at file permissions. We worked with them briefly in some previous lessons, but there is more you need to know about them.

File Permissions - review

We’ll start with a review of how to see file permissions on a file or folder. We’ll build on this for the rest of the lesson.

When a user creates a file or folder, it is given a set of permissions.

There is rwx for read, write, and execute for the user who created the file, for the group the user is in, and for all other people who may interact with the file.

Doing ll, or ls -l on a file in the user’s home directory will generate output like the following:

-rw-rw-r-- 1 theo theo    0 Nov 10 17:22 GroupPermissions.txt

The first field, permissions, can be broken down as follows:

-|rw-|rw-|r--

1| 2 | 3 | 4

1 - Indicates type - Could be Directory, regular file, as in this case, symbolic link, etc.,

2 - The next three characters, rw- are the permissions the user who created or owns the file has on it. In this case, r and w for read and write. The third character, the -, would be replaced by an x if this were a file that could execute, like a binary file or a shell script. It’s a text file, so that is not set for this file.

3 - The next three characters, rw- again, are for the group that is assigned to the file. This means the group has read and write access, like the file owner.

4 - The third three characters, r-- are for all other users, so they can read, but not write to the file.

Continuing on in the output, the theo after the number 1 is the username of the user who owns the file.

After that is the group that owns the file. theo again here.

These are the properties we’re interested in for this lesson.

Managing Permissions Using Letters

There are two ways you can represent the permissions on a file or folder. 

The first is with letters, which seems pretty straight forward. Again, it’s r for read, w for write, and x for execute.

Here’s a table representing the possible permissions.

In chmod, you use the leading u to set user permissions, g for group, and o for other.

If you want to change permissions for the user, the owner, to read, write, and execute and take away all permissions for others, you could use the chmod command and type out:

chmod ug=rwx o=rwx 

You can set permissions for user, group, and others all at once by specifying a for all before the permissions:

chmod a=rwx

would set user, group. and others to read, write, and execute being permitted, overwriting any existing permissions.

From a security perspective, it is not recommended to give the other group full permissions on a file or directory. This is just an example.

Operators For Changing Permissions with Letters

The three operators for changing permissions with letters are =, +, and -.

= Overwrites any existing permissions with the ones specified.

+ Adds the permission specified to existing permissions.

- Removes the permission specified from existing permissions.

If you have a file with rwx for the group, and you want to remove x only, you would type:

chmod g-x <filename>

Similar for adding permission. To add write and execute for the group to a file, you would type:

chmod g+wx <filename>

would add write and execute to the existing permissions for the file or folder.

I hope that is clear. Please review and work with permissions a bit to become familiar with them.

But, there’s an easier way.

Numeric Permission Changes

There’s another way to represent the permissions on a file or folder.

The three possible permissions on a file can be represented numerically as well.

4 = Read

2 = Write

1 = Execute

Why would you want to represent them with numbers?

Because, to change permissions on a file using the numeric method, you can represent read, write, and execute with one number, which is the total of adding the three together.

For read, write, and execute, it would be 4 + 2 + 1, or 7.

We just represented three permissions, r, w, and x, with one number, 7.

Pretty cool!

For read, execute, it would be 4 + 0 + 1, or 5.

755 is a fairly common set of permissions for directories. That would be rwx for the user, and r-x for the group and others.

You have to permit execution (x) on a directory in order for users to be able to access it.

So, the command:

chmod 755 <filename>

Could replace the two commands:

chmod u=rwx <filename>

chmod go=rx <filename>

Perhaps this doesn’t sound like much of a savings, but if you’re working with permissions often, it can mean a lot less typing.

It’s good to understand both methods, but you’ll likely go with numeric after a little practice.

It’s less typing, and, once you’re used to it, less thinking, and that’s good!

Ubuntu Server - Working With Files - Hidden Files

In this lesson, you’ll learn about hidden files and directories in Linux.

All you have to do to set a file as hidden in Linux is add a period to the beginning of the file or folder name.

System files and folders are sometimes hidden to reduce the chances of users altering them.

Home Directory

There are several files and directories in user’s home directories that are hidden by default. Here are some:

Directories:

.cache

Files:

.bash_history

.bash_logout

.bashrc

.profile

.selected_editor

.sudo_as_admin_successful

.bash_history

Doing tail .bash_history will show you that this is where the history of the commands you have run is stored.

Remember that this file is not written to until you log off normally though.

.selected_editor

Some commands, like visudo, for editing the sudoers file, and vipw for editing /etc/passwd, will ask you what editor you would like to use, the first time they are run.

They will store your answer here, so they don’t have to ask you again.

.sudo_as_admin_successful

This file will show up for users who have successfully used their sudo privileges.

Those are just a few examples of the kinds of things you’ll find in hidden files.

Remember, though, all you have to do is add the -l option to ls to see them, so it’s not an effective way to hide files from a security perspective.

Ubuntu Server - Files - Copy-Delete-Move

In this lesson, we’ll have a closer look at file copying, deleting, and moving files.

Copying

We used the cp command in the lesson titled TAKE PRECAUTIONS When Editing System Files.

At its most basic, you simply type:

cp file1 file2 

to copy a file named file1 to a new file named file2 in the directory you’re in.

There are times when you may want to do more though.

What if you want to copy all files in a directory, or if you want to copy files and subdirectories? We’ll cover those possibilities and more now.

Deleting

rm

The rm command is used to remove or delete files. 

You’ll remember from the Super Special root User lesson how dangerous rm can be, especially when used with -r for recursive (deletes everything in the current and lower directories), and -f for force.

We ran the command rm -rf / and bypassed the warning about rm commands on the root directory to quickly destroy a system.

With sufficient tools, knowledge, and time, it may be possible to recover deleted files. For secure deletion, use shred.

shred

Shred lets you do secure deletions of files.

wipe

Wipe lets you do secure deletions of files, and directories.

Moving and Renaming

mv is the command used to move files. It also works to rename files. If I move a file within the same directory, it has the effect of renaming it.

To rename, just mv within the same directory. The old file will be gone and only the renamed version will exist.

Ubuntu Server - Groups - Hard Links and Soft Links

In this lesson, we’ll discuss something we haven’t touched on yet, linking files.

Instead of making copies of files all over your system, it is sometimes desirable to just link to one version of the file.

This is done with links.

A link is similar in concept to a shortcut in Windows.

Links are pointers to the actual file.

There are two types. Soft links, often referred to as symbolic links or sym links, and hard links. We’ll discuss both in this lesson.

Before discussing that, though, we’ll take a look at inode numbers, and what they are. You’ll understand why shortly.

inodes

Along with a name, every file is assigned an index or inode number. The inode number is how the system actually keeps track of the file.

Typing ls -li (the i is to print inode numbers) will yield a line similar to the following:

22514 -rw-rw-r-- 1 theo theo 1167 Nov 10 20:39 file-permissions.txt

The leading number, 22514, is the inode number for the file file-permissions.txt.

The number of inodes available on a system is limited, but is very large, depending on the file system in use.

Typing df -i will give you the status of inode use on your file system.

theo@ubuntu-server:~$ df -i

Filesystem                          Inodes  IUsed  IFree IUse% Mounted on

udev                                121959    456 121503    1% /dev

tmpfs                               127009    589 126420    1% /run

/dev/mapper/ubuntu--server--vg-root 557056 130883 426173   24% /

tmpfs                               127009      1 127008    1% /dev/shm

tmpfs                               127009      3 127006    1% /run/lock

tmpfs                               127009     16 126993    1% /sys/fs/cgroup

/dev/sda1                           124928    313 124615    1% /boot

tmpfs                               127009      4 127005    1% /run/user/1000

You can see that inode use is very low on my vm except for /dev/mapper/ubuntu—server—vg-root.

You can get more information about a particular mount point with tune2fs: 

sudo tune2fs -l /dev/sda1 | grep -i inode

Filesystem features:      ext_attr resize_inode dir_index filetype sparse_super large_file

Inode count:              124928

Free inodes:              124615

Inodes per group:         2048

Inode blocks per group:   256

First inode:              11

Inode size:           128

Now, lets see how inodes relate to linking to files.

Hard Links

To create a hard link to a file, you just type 

ln <file-name> <link-name>

If you have a file called locs.txt and you want to make a hard link to it called hard-link-to-locs.txt, you would type

ln locs.txt hard-link-to-locs.txt

Running ls -li on the directory those files are in shows they point to the same inode number.

It is essentially two names pointing to the same underlying file.

Soft Links

To create a soft link, the command is the same, but you add the -s option.

If you have a file called locs2.txt and you want to make a hard link to it called soft-link-to-locs.txt, you would type

ln locs2.txt soft-link-to-locs.txt

Running ls -li on the directory those files are in shows they point to different inode numbers.

A soft link points a new file name to the name of the original file.

ls -li will show an arrow pointing from the soft link to the file name it is linked to:

soft-link-to-locs.txt -> locs2.txt

When listing files, working soft links will show up light blue, and broken ones will show up red by default on a color capable system.

Behavior

Changing the name of a file with a hard link will not break the link, as it is pointing to the inode number, not the file name.

Deleting the file a hard link points to will not break the link either. The hard linked file will just be the only instance of the file left on the system.

Changing the name of a file with a soft link to it will break the link. The soft link will no longer be able to find the file.

Deleting a file with a soft link to it will also break the link. 

Practical Use

I most often see links used for system files in web servers like Apache and Nginx. When you want links, potentially in several locations, to point to one “master” file. You make the changes in the original file, and all links are automatically up to date.

Ubuntu Server - Managing Files - find and locate

You’ll often want to find files from time to time on your Ubuntu server. In these instances, find and locate are the the most common commands used to find files at the Linux command line.

Both are very powerful, but they work a little differently.

find 

find is a super powerful tool for finding a file, or files using many possible properties of the file. Some common properties you may use to find files are, of course name, or parts of the name, type, owner, permissions,  

find has loads and loads of options. We’ll cover some of the most common, and one not so common option to get you familiar with finding your files with find.

Common Options

The most common use of the find command is to search for a file by name.

You type find, then the starting point for the search, then the -name flag, then the name of the file.

find / -name “my_file.txt”

searches from the root of the file system for a file called my_file.txt.

Remember, searching from root will recursively search all directories on the file system. This could result in a long search!

Recursive searches start at the top level directory you’re in or the one you specify, and search all directories under it.

Putting quotes around the name can help you avoid issues with wild card characters like * and is not a bad habit.

Also remember, if you start a search from root, searching all directories, and you don’t run the search with sudo permissions, you’ll get a lot of errors due to not having read privileges on some directories.

You’ll either want to run it with 2>devnull, or run with sudo privileges.

Well, that’s pretty straight forward.

Search by User (owner)

What about searching for files by a particular user?

find / -user lskywalker 2> devnull 

finds all files on the system owned by lskywalker, sending errors to dev null.  Remember, dev null is a special device that quietly discards anything sent to it.  It’s often referred to as a bit bucket.

Search by File Type

File types for the find command are specified after the -type flag.

Types are:

f = files

l = links

s = sockets

d = directories

find / -type d -name mydir

will find all directories from system root downward that are named mydir.

Wild Cards

An * is a wild card character in searches.

find / -name “*y_file*” 

searches for all files with y_file in them.

Ignoring Case

Always remember, unlike Windows, Linux is case sensitive.

Since that is true, my_file.txt is different from My_File.txt.

To find a name, whether upper or lower case, use the -iname option.

find / -iname “my_file.txt”

Would return both my_file.txt and My_File.txt. 

Permissions

You can find files with certain permissions set by using the -perm option.

find / type d -perm 777

returns all files that have full permissions on user, group and others for a file.

The not Operator

If you want to find all files except ones with a particular pattern, you can precede the ones you don’t want with an !.

find . -iname "my_file*" ! -name "*.txt"

Will return upper and lower case files with my_file in the name, but exclude .txt files. You can do the same by replacing the ! with -not.

find . -iname "my_file*" -not -name “*.txt"

Performing Operations

You can perform operations on files that have been found.

When you remove a user, if you use deluser and you don’t specify the —remove-all-files option, some files may be left behind that have no valid owner.

These are called orphaned files.

You can find them as follows:

sudo find / -nouser

You can use the -exec option to remove them as follows:

sudo find / -nouser 2> devnull | xargs sudo rm

Use caution with such operations, especially when running as root!

Remember, you’re automating a process and running as root. A mistake here could leave your system in bad shape.

Just verify that it will do what you want before hitting Enter.

locate

locate is another powerful search tool. It operates a bit differently though. It searches a database of information about the files on your system, not the actual file system itself.

The database is periodically updated by the system when it runs the updatedb command. 

Let’s make a new file in our home directory, then find it with find.

What gives? Why doesn’t locate know where our file is? We can see it right there?

Remember, locate uses a database to find files, not the actual file system. By default, the updatedb command runs at midnight each day.

If you don’t want to wait for midnight, you’ll have to run the updtedb command manually with sudo privileges to cause the system to update its database.

sudo updatedb

Now, running locate again, we find our file!

Given the potential inaccuracies in using locate, when would you want to use it instead of find?

The syntax for locate is pretty quick and straight forward to type. If I’ve been working along, and I know I’ve recently created a file, but don’t remember exactly where it is, or don’t want to browse to it, I can run sudo updatedb, then find it with locate.

Searching a large number of directories, like all from the system root / and all sub-directories, because locate is just searching a database, it returns results much more quickly than find.

OK, if locate is so much quicker, why not just always use it? locate isn’t as capable as find. 

My experience is that you’ll want to be most familiar and comfortable with find, but also be comfortable with running updtedb and locate for quick tasks.

find is more capable, and searches the actual file system.

Ubuntu Server - Configuring Remote Access - Overview

One thing you’ll want to understand and be able to set up securely is remote access to your Ubuntu Servers.

In modern computing, it is quite rare that you will have to, or want to manage a server by a directly connected keyboard and mouse, known as a console connection.

You’ll want to be able to sit at your desk, or at home, or wherever you choose and manage your server using Secure Shell (SSH).

There are many pieces involved in setting up SSH, but we’ll break it down, so you’ll be able to handle it like a pro!

SSH 

SSH is a network protocol specifically designed to let you securely access servers over a network connection.

It uses Transmission Control Protocol on port 22 to listen for connections by default.

Traffic, including your credentials, is kept encrypted, so eaves droppers can’t see or tamper with your data in transit.

Once you authenticate to an SSH service listening on your server, you’ll be able to manage it using a terminal emulation program from anywhere.

There are several steps in configuring remote access securely.

• Ensure openssh server is installed on your server
• Configure networking on your VirtualBox VM so you can connect
• Configure your ufw firewall
• Download a terminal emulator
• PuTTY for Windows
• iTerm for MAC OS X
• OpenSSH Client for Linux
• Configure Public Key Infrastructure (PKI) encryption
• Generate your keys
• PuTTYgen for Windows
• ssh-keygen for MAC OS X and Linux
• Copy your public keys to your account on the server
• Test public key access
• Disable remote access for the root account
• Disable password authentication for remote access
• Consider installing Fail2ban and possibly changing the listening port number
• Fail2ban will be covered in the Security section

Wow, that sounds like a lot of stuff!

It is quite a bit, but it’s broken down for you, step-by-step. 

You’ll understand SSH and be comfortable installing, configuring, and testing it by the end of this section.

Ubuntu Server - Configuring Remote Access - Preparing Your Server

To use ssh to connect to your server, you’ll have to prepare it with three things.

• Make sure OpenSSH Server is installed
• Make sure you can connect over the network
• Make sure you have a firewall exception for ssh

We’ll cover each in this lesson.

Install OpenSSH Server

You can see if OpenSSH Server is installed by typing man sshd_config.

If you have OpenSSH Server installed normally, this will bring up the man page.

If not, you’ll get an error

No manual entry for sshd_config

If it’s already installed, go to the next step.

If not, type sudo apt update, then sudo apt install sshd-server, hitting y when prompted to install it.

Configure Networking

If you’re using VirtualBox to run your Ubuntu Server, you will probably have to make a small configuration change to be able to connect from the network.

I’m assuming you’re on a home network with some kind of wireless router/firewall protecting your computer from the Internet. 

It shouldn’t matter much either way, though, as long as your root account is disabled for login, and you have a strong password on your user account. Also, be sure your system is up to date by running sudo apt update, then sudo apt upgrade -y, and sudo apt full-upgrade -y.

Once your system is up to date, power it down. Right click on it and select Settings, then click on the Network tab. In the dropdown for Attached to:, select Bridged Adapter.

This will give your Virtual Machine its own IP address on your local network, which is the most straight forward way of being able to connect to it over your network.

Configure The ufw Firewall

Once both of those are set up, configure the ufw firewall. 

More is covered on ufw, and how it works in the Securing Ubuntu Server section, in the Enabling Your Firewall (ufw) lesson.

Here, we’ll do the bare minimum, preparing your server to accept connections on over SSH on TCP port 22.

To see the status of ufw, type sudo ufw status.

If this is a default installation, you’ll probably get the reply Status: inactive.

Allow incoming connections for SSH by typing

sudo ufw allow ssh

Allow outgoing traffic and deny incoming traffic that wasn’t requested by your server by typing

sudo ufw default deny incoming

sudo ufw default allow outgoing

Enable the firewall by typing sudo ufw enable.

Ensure that it’s running by typing sudo ufw status again. This time, it should tell you the firewall is enabled, and that port 22 is allowed.

You can get more information by typing sudo ufw status verbose.

Start OpenSSH Server

Now that you’ve installed OpenSSH Server on your Ubuntu Server, configured networking, and configured your firewall, you can start it with the following command:

sudo service ssh start

You can check the status by either typing:

sudo service ssh status

or

sudo netstat -tanup | grep ssh

Ubuntu Server - Remote Access - Connecting With SSH

Now that we’ve set up SSH on the server, we can connect to it from any computer that can reach it over the network it is connected to.

If you’re running a virtual machine on VirtualBox, you can connect from your host computer (the one running VirtualBox).

I’ll cover connecting with iTerm2 for MAC OS X in this lesson, and we’ll cover connecting with PuTTY for Windows in an upcoming lesson.

iTerm2 

iTerm2 for MAC OS X is free to install, but isn’t available in the App Store. Just browse to iterm2.com, click on Download, and double click the downloaded app once the download is complete.

Finding Your IP Address

You’ll have to know your server’s IP address to be able to connect to it.

The command to learn the IP address is

ip addr | grep inet

You can see several IP addresses listed.

127.0.0.1 is a reserved IP address for a special internal network adapter called a loopback adapter. It is used to allow network communication internally on your server.

The lines that say inet6 refer to IP version 6 addresses, which we’re not interested in right now.

The one that says inet and starts with a 192., a 172., or a 10., is likely the one you’re after. If you’re on a home or work network, it will likely be one of those. If your computer is directly connected to the Internet, it could be different.

Mine is 192.168.254.105.

Connecting With SSH

Open iTerm2, and type ssh <user>@<ip-address>

For me, the command is

ssh theo@192.168.254.105

You’ll be prompted for your password, so enter it.

And you’re in!

Now, you can do anything you could do at the command line on the console.

Well, almost anything.

Bear in mind, when you’re making changes to your networking configuration, your SSH configuration, or your firewall, they can affect you, since you’re connected over the network.

This is no big deal, if you’re working on a system where you have direct and easy access to a console, but can be a bit of a pain if the server is across the country or in the cloud somewhere.

When making changes to any of those items, keep your existing terminal session up, if possible, and test with a new connection, before you disconnect your original session.

Some changes, such as a mistake in your firewall configuration, could kill your remote session.

In that case, you’ll have to have console access, or think of another way in.

Also, this lesson only covered connecting with username and password, which isn't nearly as secure as key based authentication, which we'll cover in upcoming lessons.

Ubuntu Server - Remote Access - PuTTY for Windows

In order to connect to your server over SSH, you’ll have to use a terminal emulation program. It’s a program that’s designed to act like you’re sitting at a terminal typing commands.

For Windows, I recommend PuTTY. It comes with another utility called PuTTYgen we’ll use in a later lesson, so please download both, or just download the entire PuTTY software suite.

PuTTY Download 

Browse to putty.org and click on the link that says “You can download PuTTY here.”

Under Package files, click on the link for 64-bit MSI(‘Windows Installer’).

Once putty has downloaded, double-click on the .msi file to install the PuTTY Suite.

Connecting

 Double-click on PuTTY to start the program.

Obtain the IP Address from your Ubuntu Server.

The command to do that, again is

ip addr | grep inet

Type the IP address into the Host Name or IP address dialogue box.

If you’re going to be connecting to your server often, and you likely will, put a name in under Saved Sessions, and click on Save to the right.

Then, when you want to connect in the future, you can just double click on the saved session to connect.

I usually just make the name of the session the same as the name of the server.

Ubuntu Server - Remote Access - Key Based Authentication

Password authentication is OK, provided you create a strong, difficult to guess or crack password.

Here’s a weak, well known password, that provides zero protection:

P@ssw0rd

This is in the word list of many programs that try common passwords on systems they’re trying to break into.

Here’s a much stronger password:

R0deMicrophonesR#0ne!!!

That password is long enough and complex enough that it would be a real challenge for a password cracking program, if the proper encryption and hashing are used.

If you configure a maximum number of retries for SSH login, the default is 3 attempts for Ubuntu Server, the chances of someone correctly guessing the strong password above are near zero. The chance of it guessing the weak password are relatively good, as it is often used both by users, and by bad guys.

If you ever re-use your password(s) though, they can become compromised through breaches.

Compare both, of the passwords above to the following public key generated for key based authentication:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+D+LxaxTxMIY25V5IX/gbKEEWsYqrcSVZOv9Rr/xoHatuljwfsi0shQGHxKpfr2LOwBX5q4dt3/Bq18oXOeJmhVIAcnL/gKsMAw1GsnMNFPmwe1G3QrwV/ofxIrLGWlb6/vbBjZBmRmv/KLxyUcBbpsg8rGFobfLq15o3s4h/CAOYlr9OD8BO58DzW0nyEJvfNS/RAQ2Q6e8KH0+E91sxIU3bauDkSzEB6jh1u7drQEvB7OSuPZVA2jE3yGFi2U344gtq+BXlNc0kEQWAdLqi7L4MnAHLXWYWMNYylrzrMZlpR1VO3Kv9y6NxEYIOphGWiZxCd+44HxC+V1lVvpxH theo@Ubuntu-Remote

You can see, this is a practically uncrackable public key.

[NOTE] The public key I provided above was a disposable one for a temporary server. Even though it’s “public” you should not expose it unless you have to. 

It is OK to provide it to someone who wants to give you access to their server. It will not provide them access to your server.

To make things even more difficult for an attacker, that key is paired with a private key, and a mathematical operation is used that makes use of both keys for encrypting and decrypting traffic.

It’s a really, really good system for controlling access to your server.

It is worth the effort, and will become easy once you’re done it a few times.

Generating Your Keys

In order to use public key authentication, you’ll have to generate your keys. Linux provides a tool that makes the process easy called ssh-keygen. It works for Linux, BSD, and MAC OS X.

ssh-keygen will generate your public and private keys, store them in the default location, and let you set a password on your key, all in a few simple to follow steps.

I recommend setting a reasonable password (12 characters, using upper and lower case letters, numbers, and special characters), or a passphrase with those elements on your key.

That way, if someone is able to access your keys, they still can’t get on your server.

Protect your keys, especially your private key! If someone gets it, they can try to log in as you. The only thing that will save you is the password on your key at that point.

Here, I’m going to be on ubuntu-server, the server I’m using throughout most of the course, creating a key for use on Ubuntu-Remote, the server I’ve just configured for this lesson.

I’ll generate the keys on ubuntu-server, then copy them to Ubuntu-Remote.

The username will be theo on both servers.

Type ssh-keygen.

Accept the default location for the keys generated: /home/theo/.ssh/id_rsa by hitting Enter.

You’ll see that the directory /home/<username>/.ssh was created. That is the hidden directory where your keys will be stored.

You’ll get a key fingerprint, which you will not need, and an ascii art graphic showing you the randomness of the generated key.

This process will generate two files, id_rsa, your private key, do not ever give this out, and id_rsa.pub, your public key.

If you cat your public key, you’ll see a bunch of gibberish looking text like the output copied before.

Adding Your Public Key To The Remote Server

Now that you have your public key, you can add it to your new server by connecting over SSH. I recommend doing this over iTerm for MAC OS X, at the command line in Linux, or using PuTTY in Windows.

You can use the command ssh-copy-id to copy your ID to the new server.

ssh-copy-id <username>@ip_address

For my Virtual Machines, that will be:

ssh-copy-id theo@192.168.254.105

This creates the proper files and directories, if necessary, on the remote server, and puts your public key into them.

The .ssh folder will be created, if it doesn’t exist with 700 permissions, and the authorized_keys file will be created with 600 permissions.

SSH keys will not work if the permissions are not set up correctly on the file or the directory.

If you cat authorized_keys, in the .ssh folder, you’ll see your public key from the server you connected from.

If the authorized_keys file already exists, the new public key will be concatenated to the end of the file.

Why The Extra Security?

You may wonder, “Why would anyone attack my server? I have nothing valuable on it.”

Unfortunately, attackers likely don’t care who you are or what’s on your server. There are people and programs constantly scanning the Internet for vulnerabilities and automatically or through manual intervention, leveraging them to compromise systems.

I spun up a disposable server on DigitalOcean to illustrate this.

The server was up for 5 days. In that time, 1044 login attempts with invalid usernames were made.

This is a throw-away server with nothing anyone could want on it.

If nothing else, if it is compromised, it could be used to scan for and/or attack other resources.

Protect yourself with public key encryption.

In a future lesson, you’ll learn how to disable password authentication, so only key based authentication will be used. Until that is done, the system will fall back to password authentication if key based authentication fails. That’s not what we want.

Ubuntu Server - Remote Access - Windows Key Based Authentication

In the last lesson, you learned how to set up key based authentication for Linux, and MAC OS X. In this lesson, we’ll learn how to configure it for Windows. 

We’ll also learn how to set key based authentication up manually, without the aid of ssh-copy-id.

We’ll start by adding a new user to our server and giving it sudo privileges.

adduser <username>

<password>

<password>

sudo usermod -aG <username> sudo

Test your remote access.

In PuTTY, enter <username>@<ip-address> in the Host Name (or IP address) text box. Save the session if you will be using it often.

Make sure you can use sudo permission with the new user.

sudo ls /root

Windows Keys

The things we have to do will be the same as we did with Linux, but the tools we use will be a bit different for Windows.

In a previous lesson, we downloaded a component of PuTTY called PuTTYGen.

PuTTYGen is a utility to generate public and private key pairs in Windows.

Either browse to it or just click on the start menu and type PuTTUGen. Click on it to open PuTTYGen.

Click on Generate to create your public and private key pair.

You’ll be asked to generate some randomness by moving your mouse around, so do that.

Click on Save public key, and accept the default name and location, then Save private key and do the same.

When prompted, enter and confirm the password your keys.

This password or passphrase will be used to encrypt the keys, so even if they’re stolen, they won’t be of use until the password is entered.

The public key will appear in a text box. Copy the key so you can paste it into the correct location on your Linux server.

If you forget to do that, you can do it using the Conversions tab later.

Adding The Key To Your Server

Now, SSH to your server by opening PuTTY and typing <username>@<ip-address> in the Host Name (or IP address) text box and clicking Open.

We’ll have to create the .ssh folder, add an authorized_keys file, and paste your public key into that file.

mkdir .ssh

chmod 700 .ssh

cd .ssh

touch authorized_keys

chmod 600 authorized_keys

You may be tempted to create the authorized_keys file using vim and paste your key into it. I recommend using the following method instead.

Being in the .ssh directory, type

echo “<public-key>” >> authorized_keys

Replacing <public-key> with the actual public key you copied earlier.

The reason I recommend this method instead of using vim or nano is that a single typo, even one you can’t see, in your authorized_keys file will break key based authentication. A text editor may insert something you can’t see easily, that will make your key not work.

Testing

Before you disconnect your remote session, test by opening a new PuTTY session. You’ll have to tell PuTTY to use your newly created key. 

It is best if you create a saved session at this point, if you haven’t already done so.

Click once on the session to select it, then click on SSH under Connection, then Auth.

You’ll see a section at the bottom of the window that says Private key file for authentication. Click on Browse and locate the private key you generated earlier.

Scroll back up in the Category window, click on Session, then click Save to save your changes.

Now, double-click on your saved session to open it.

You’ll be prompted for the password you assigned to your key.

Once you enter it, you should be in!

Be sure your sudo privileges work.

sudo ls /root

Troubleshooting

Because of the extra steps involved, you’re more likely to run into issues here than with the Linux process.

You can find helpful clues about why authentication may not be working in /var/log/auth.log on the server you’re trying to connect to.

You can either tail the log file, or grep for the username that’s having issues, and see what the log file says.

We’ll start with the basics, then work to more specific likely causes.

SSH Service

Type service ssh status. If the status says it is loaded, you’re good. If not, type sudo service ssh start.

Network

Type netstat -tanup | grep ssh.

You should get a line like the following:

tcp   0   0 0.0.0.0:22      0.0.0.0:*    LISTEN      1088/sshd

This means it is listening on all IP Address and it’s good.

ufw

Check the ufw status.

sudo ufw status

You should get output similar to the following:

Status: active

To                         Action      From

--                         ------      ----

22                         ALLOW       Anywhere

If not, type sudo ufw allow ssh and try again.

Permissions And File Content

If the service is started and listening, and ufw is configured correctly, there is likely a problem with permissions or the content of the authorized_keys file.

1. Permissions are not correct on the .ssh folder or the authorized_keys file
1. Permissions on the .ssh folder must be 700
2. Permissions on the authorized_keys file must be 600
2. There’s a typo in the name of the authorized_keys file. Make sure it is authorized_keys.
3. There’s a typo, possibly hidden, in the content of the authorized_keys file.
1. Remove the contents by deleting everything in the file.
1. echo “” > authorized_keys
2. Re-do the command to paste your key into the file, ensuring that you’ve pasted it exactly as it was in the PuTTYGen output screen. If you no longer have that, you can use Import key under Conversions to open the public key.

If you’ve done all of the above, and you’re still having issues, please post a question in the course Question and Answer section, and one of your fellow students or I will try to help. You can also Google for help.

Saving Your Keys in Pageant

If you’re managing a lot of Linux servers from a Windows computer, you may get tired of entering the password for your SSH key each time you connect to a new server or reconnect to one you were working on.

Wouldn’t it be nice if there were a way to just save your keys once, and have them used every time you log in to a server that has those keys?

There is!

It’s one of the programs bundled with PuTTY called Pageant.

Adding Keys to Pageant

Storing your keys in Pageant is pretty straight forward.

Go to the Start menu and type in Pageant to search for it, then click on it to open it.

When it opens, you won’t see anything happen unless you’re observant and notice the new app appearing in your system tray.

I do find it a little frustrating that it doesn’t just open a window and ask you to add your key like most applications, but other than that, I like it.

Right click on Pageant and select Add Key to add your key.

Browse to the location where your key is stored, and click on your private key file. It should have a .ppk extension.

Enter your pass phrase for the key in the window that pops up.

Click OK.

Connecting to Your Server

Now that your key is stored in Pageant with the password already entered, you can open Putty and connect.

Notice that you didn’t have to enter your password. Sweet!

This does store the unencrypted private key in memory making it vulnerable to an attacker having access to your system’s memory, but I know of no exploits available in the wild that take advantage of this to steal your unencrypted key.

That’s it for this lesson.

Configuring sshd_config on Ubuntu 20.04

I was updating my Ubuntu course from 18.04 to version 20.04 and found an intriguing new line in /etc/ssh/sshd_config.

Include /etc/ssh/sshd_config.d/*.conf

Looking at the man page for sshd_config, the SSH daemon, on starting, will now look for configuration settings in /etc/ssh/sshd_config.d/. The files should have any name ending in .conf. A possible name would be my_sshd_config.conf.

Why would I want to create a new, external file, instead of just editing the sshd_config file as I’ve always done?

From doing a quick search on the Internet regarding securing SSH, it looks like that’s exactly what most people are doing.

This new system introduces an opportunity though.

It follows the format for systemd unit files. I can now put my settings in a file in /etc/ssh/ sshd_config.d/ and they’ll persist even if /etc/ssh/sshd_config is overwritten as part of an upgrade to the OpenSSH version running on my server.

It will also supersede settings in the /etc/ssh/sshd_config file. For environments where a lot of people may be managing systems, it’s nice to know that someone tweaking a file in the wrong way will not compromise critical security settings for your SSH service.

So, how does it work?

In the /etc/ssh/sshd_config.d/ directory, create a new file. I’m naming mine 10-my-sshd- stuff.conf. You’ll have to do this with sudo privileges.

sudo vim /etc/ssh/sshd_config.d/10-my-sshd-stuff.conf

Inside that file, enter the keywords and arguments you want to change from default, or ensure are explicitly addressed. Note that keywords are case-insensitive and arguments are case-sensitive.

There are some configuration changes I pretty much always make for any publicly exposed server I’ll put inside the file.

• DebianBanner no

• DisableForwarding yes

• PermitRootLogin no

• IgnoreRhosts yes

• PasswordAuthentication no

• PermitEmptyPasswords no

  Caution! Be sure you have key based authentication set up and working before you set PasswordAuthentication to no!

  Content of my-sshd-stuff.conf file

  Be sure you understand the implications of the settings before applying them. Most are fairly innocuous, but setting PasswordAuthentication to no will lock you out of a remote system if you restart SSH and log out before testing. You’ll have to get back in through a console, then set up key based authentication.

  You can find my tutorial on setting up key based authentication here.

  Once you’re sure key based authentication is working, and your configuration file is the way you want it, restart SSH to make the settings take effect. There are a few ways you could accomplish this, but here’s one:

  sudo service ssh reload

  Reloading should not kill your active setting in case you’re in remotely and want to test with another connection before logging off.

  Thats it! Now you can manage all your SSH settings in one file and they won’t be clobbered when OpenSSH is updated.

In this lesson, we'll have a look at each line of our configuration file. We'll look at what the line sets, what the default is, and review why we set it this way in the file.

Further details on the settings that can be configured in this file can be found in the man pages by typing man sshd_config.

Here are the settings again:

DebianBanner no
DisableForwarding yes
PermitRootLogin no
IgnoreRhosts yes
PasswordAuthentication no
PermitEmptyPasswords no

Now we'll look at each line.

DebianBanner: Specifies whether the distribution-specified extra version suffix is included during initial protocol handshake.  The default is yes.

Information on what operating system is in use can be very helpful to attackers. This command will display slightly less information for people grabbing banners from SSH. Banner grabbing is looking at what information or what "banner" is returned from any connection attempt, successful or not.

We'll use nmap to see what is displayed when DebianBanner is set to yes, or default, then when it is set to no.

Here's what is displayed when DebianBanner is set to yes:

Theodores-MBP:~ tedl$ nmap -sV -p22 192.168.254.147
Starting Nmap 7.80 ( https://nmap.org ) at 2021-01-11 22:03 EST
Nmap scan report for ubuntu-20-40-server (192.168.254.147)
Host is up (0.0023s latency).

PORT   STATE SERVICE VERSION
22/tcp open  ssh     OpenSSH 8.2p1 Ubuntu 4ubuntu0.1 (Ubuntu Linux; protocol 2.0)
Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 0.39 seconds

Note that an attacker now knows we're running Ubuntu Linux.

Here's what it looks like when this setting is changed to no.

Nmap done: 1 IP address (1 host up) scanned in 0.39 seconds
Theodores-MBP:~ tedl$ nmap -sV -p22 192.168.254.147
Starting Nmap 7.80 ( https://nmap.org ) at 2021-01-11 22:07 EST
Nmap scan report for ubuntu-20-40-server (192.168.254.147)
Host is up (0.0019s latency).

PORT   STATE SERVICE VERSION
22/tcp open  ssh     OpenSSH 8.2p1 (protocol 2.0)

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 0.38 seconds

Note that only the information on the OpenSSH service is displayed now. This is what we're after when setting DebianBanner to no.

DisableForwarding: Disables all forwarding features, including X11, ssh-agent(1), TCP and StreamLocal.  This option overrides all other forwarding-related options and may simplify restricted configurations.

Forwarding can legitimately be used for the following:

• To tunnel sessions transfer files through jump servers

• Connect to a service on an internal network from the outside

• Connect to a remote file share over the Internet

Please be sure your developers are prepared for you to disable this if they typically use it.

So, what's wrong with forwarding from a security perspective?

Forwarding can be used to pivot or to gain access to a network an attacker didn't have access to via a device that does have access to it. Many types of attacks can be carried out once access is gained including scanning for vulnerabilities and sending payloads and exploits. Also, since it's SSH, the traffic is encrypted, often letting the bad guys traffic remain hidden from view.

There are many types of forwarding including X11Forwarding and AgentForwarding. You can disable all forwarding with the single DisableForwarding directive.

If it's not needed, forwarding can be disabled using the global DisableForwarding directive.

DisableForwarding yes

PermitRootLogin: You will not typically connect to an Ubuntu server over SSH as the root user. For that reason, setting PermitRootLogin to no will disallow the root user from logging in over SSH. You'll connect as your regular user with sudo privileges and elevate when needed with the sudo command.

PermitRootLogin no

IgnoreRHosts: Specifies that .rhosts and .shosts files will not be used in HostbasedAuthentication.

Default is yes. RHosts is ignored. You may skip this and achieve the desired behavior with the default configuration. You may get complaints from scanning tools like Lynis and Nessus may complain if it isn't explicitly denied though.

IgnoreRHosts yes

PasswordAuthentication: Specifies whether password authentication is allowed. 

Default is yes, or allowed.

Once you have key based authentication enabled and tested, there's no longer a need to have password authentication enabled. If key based authentication is enabled but doesn't work for a user, and password authentication is still enabled, the default behavior is to fall back to password and let the user try her password for the maximum number of attempts.

This is not desired behavior for a system that uses key based authentication. We do not want any chance of a misconfiguration letting a bad guy guess a password and get access.

PasswordAuthentication no

PermitEmptyPasswords: When password authentication is allowed, it specifies whether the server allows login to accounts with empty password strings. 

The default is no.

This setting can also safely be ignored and you'll get the default behavior. Again, some scanning tools may complain if this isn't explicitly set to no.

That's it! Remember, you can find all of this and more in your systems man pages by typing man sshd_config. It's worth a quick look at that man page.

NOTE: If you're using Ubuntu 20.04 Server or newer, I strongly recommend using a custom file in /etc/ssh/sshd_config.d/ as taught in the previous lesson. This lesson is being left in place only for legacy support (support of installations older than 20.04).

If you did as instructed in the previous lesson titled "Managing sshd in Ubuntu 20.04", please skip this lesson.

Ubuntu Server - Remote Access - sshd_config - (Deprecated Lesson)

Now that we have key based authentication configured for SSH, we can harden our SSH configuration. This is done in the /etc/ssh/sshd_config file.

sshd_config

There are several settings we’ll want to verify or modify in the sshd_config file to make it more secure against attack.

The sshd_config file configures the ssh daemon. The ssh daemon reads this file on starting and uses the configuration kept there to run.

The ssh daemon is the server service that listens for incoming SSH connections on port 22, by default.

There are several items that need our attention. We’ll want to:

• Consider changing the default port from 22 to something else, like 2222 

• Ensure Protocol is set to 2 

• Disable root remote access

• Disable ChallengeResponse Authentication 

• Disable password authentication

Editing sshd_config

As usual, when editing a system file, we’ll save the original or the latest version.

If you’re doing this on a clean Virtual Machine, you probably don’t have a copy of the original yet, so create one

cd /etc/ssh

sudo cp sshd_config sshd_config.0

If you do already have a copy of the sshd_config file, increment appropriately.

Now, we can edit the sshd_config file.

sudo vim /etc/ssh/sshd_config

The file isn’t that big, and we’re only looking for a few lines to confirm or edit settings on. You can just scroll through, or type Esc, then / followed by the term you’re looking for if you want.

Port Number

The first thing we come across, going from top down in the sshd_config file is the TCP Port number that the SSH daemon will listen on. It is Port 22 by default.

I’m not a big fan of security through obscurity. I’d rater just leave SSH running on the default port of 22, and secure the SSH configuration.

However, you will significantly cut down on the “noise” of random login attempts if you migrate to a different port.

Bear in mind, that if you shift to a different port, you’ll have to specify the new port number in the terminal emulator you use (iTerm for MAC, SSH Client for Linux, or PuTTY for Windows).

You’ll also have to permit inbound connections to that new port on your firewall.

sudo ufw allow 222

for example, will allow connections on port 222. You would just open port 222 on ufw, change the Port in sshd_config from 22 to 222, and restart the ssh server with service ssh restart.

Test your connection on port 222 before disconnecting your existing SSH session.

That way, if something’s wrong, you’ll be able to fix it with your logged in session.

To use a different port when connecting from a port other than 22, you specify it with the -p option.

ssh <username>@<ip-address> -p <port>

For the user theo to connect to 192.168.254.105 on port 222, the command would be:

ssh theo@192.168.254.105 -p 222

For Windows users, just specify the different port in the Port text box next to the IP Address. You can save this with your profile.

To re-cap, TCP Port 22 is designed to allow secure communications over SSH, but it is well known by attackers, and they’re constantly scanning for and attempting to log in on it. You may change the default port, if you choose, by changing the Port specification in sshd_config. Just remember, you’ll also have to reconfigure your firewall and connect on the new port always.

You make the actual change in the file by editing the following line:

# What ports, IPs and protocols we listen for

Port 22  < Change this number to the port you want and save your changes.

Remember to restart SSH, with sudo service ssh restart, and to test connectivity before closing your existing session.

Specify Protocol 2 (SSH Version 2)

There are older versions of SSH that have security flaws that have been addressed in SSH Version 2. 

Be sure only version 2 is used by making sure Protocol 2 is explicitly spelled out (this is the default for any recent versions of Ubuntu).

Protocol 2

Disable Root Login Over SSH

We do not want to allow root login over SSH. The root user is not able to log in by default on Ubuntu server. We’d like to keep it that way, but if it is enabled for some other reason, intentionally or through a configuration error, we do not want people to be able to try logging in with it.

root is the name of  the super user on all Linux systems, and bad guys know this. Every system has a root user. 

You may be thinking it might be OK, maybe even a good security practice to change the name of the root user. It isn’t, and please don’t. Many, many programs may break if this user name is changed.

Even if you change the name though, its User ID number will still be 0. You will break still more stuff if you tamper with that.

The thought was reasonable, but please keep the root user, but don’t log in with it.

To make sure remote login isn’t enabled for root, find the following line in the # Authentication section and set it to no:

PermitRootLogin no

Disable Challenge Response Authentication

This is a default, but please ensure it is set to no.

ChallengeResponseAuthentication no

Challenge response authentication could prompt for authentication credentials even if PasswordAuthentication is set to no through PAM (covered later in this lesson). Please just know where this setting is and confirm that it is set to its default value of no.

Disable Password Authentication

Make sure the line, PasswordAuthentication is explicitly set to no. 

Locate the following lines:

# Change to no to disable tunnelled clear text passwords

#PasswordAuthentication yes

Remove the # sign before PasswordAuthentication, and replace ‘yes’ with ‘no’:

The line should look as follows:

PasswordAuthentication no

Maximum Login Attempts?

You may be wondering how the system blocks login attempts after a certain number of failures for SSH users.

Please scroll all the way down to the bottom of the sshd_config file, and you’ll see an entry that says UsePAM yes.

PAM stands for Pluggable Authentication Modules, and it is how many aspects of logging in are handled, and not just for SSH.

We aren’t allowing remote access over SSH, because we explicitly stated that only key based authentication is permitted, but I want you to understand why the setting isn’t in the sshd_config file.

Even More Security

In a future lesson, in the Securing Your Server section, you’ll learn how to install Fail2ban. Fail2ban is a free, open source application that watches for malicious activity, like failed login attempts, and blocks the IP Address sending such requests.

That’s enough for now though.

Great work!

Two-Factor Authentication (2FA) for SSH on Ubuntu Server

In the following lessons, we’ll configure Two-Factor Authentication or (2FA) on your Ubuntu Server.

If you want to add an extra layer of protection to authentication on your server, 2FA can help.

We’ve already enabled key-based authentication, and you should have a password on your SSH key. We’ve disabled root authentication over SSH and password based authentication.

This makes authentication to our server quite secure.

What if someone gets your SSH keys and cracks or obtains the password for the key?

The chances of this should be pretty slim, but if it happened, the attacker would be able to easily log into your server.

Enter 2FA. It’s free, and easier than you may think, to enable 2FA on your Ubuntu server.

We’ll cover the following:

An overview of 2FA and how works.
Apps you can use on your phone to authenticate
Adding 2FA to your server
How to recover from a lost, stolen, or broken phone

Ubuntu Server - SSH - 2FA - What 2FA Is And How It Works

In this lesson, we'll have a high-level look at what 2FA is and how it works.

For a normal login to a system, you'll supply a username, asserting who you are, and an additional piece of information or “factor” such as a password.

In the world of authentication, factors include:

Something you know, like a PIN or password
Something you have, like a security badge or USB Key
Something you are, like a fingerprint, retinal scan, or facial recognition. I think this is a bit of a misnomer because I’m not my fingerprint, but it’s typically based on some biological feature that links to you. It can include voice recognition.

For our solution, we’ll install an app on our server that will generate a random, time-based one-time password (TOTP). There’s an algorithm that generates the TOTP and the TOTP is relatively short, but only good for a short time.

It is typically a 6 digit numeric or number/letter combination.

This is incredibly short for a password and could be easily cracked by exhausting all possible combinations from 000000 to 999999 by a computer.

The thing is, it’s only good for a short time, typically 30 to 90 seconds, and the number of attempts is limited.

The app will synchronize with an external app on a phone or similar device. Once the two devices are synchronized, they independently generate the same number in the same timeframe based on the shared settings for the algorithm.

When you go to login, you’ll enter your name and password, then be prompted for the TOTP generated by your app. If they match, you’re in! If not, you’ll have to try again.

In our case, we’ll be authenticating using a certificate and its password. We’ll then be prompted for our TOTP.

You’ll see exactly how to set this up in upcoming lessons.

Once you have the TOTP app installed and understand how it works, you can protect other resources like your primary email address, social media accounts, or financial accounts using the same app. I recommend looking into whether your desired institution offers it and setting it up where possible if extra security is important to you.

I use Proton Mail for my personal account and I’ve set up 2FA for it, for example.

Ubuntu - SSH - 2FA - Apps You Can Use

In this lesson, we’ll look at a few of the apps you can choose from to prepare for enabling 2FA.

There are many free and paid options including:

Google Authenticator - Free
LastPass Authenticator - Free and Paid Versions Available
Microsoft Authenticator - Free
Authy - Free
Duo - Used to be Free, but is now a paid service with Cisco

Wherever possible, you’ll want to choose one and stick with it. I can tell you from experience, it’s kind of a pain to search through them looking for the one you need for a particular login.

I use LastPass Authenticator paid version, but free works fine.

If you’d like to use LastPass, go to https://lastpass.com/create-account.php and create an account.

If found the paid version priced very reasonably at $3 per month if you’re billed annually ($36 once per year). I like to support great work if the price is reasonable. Along with providing an app for 2FA authentication, LastPass is also a password vault. Just be sure you secure it with a good passphrase and enable 2FA on your LastPass account!

The free version will do everything you need for enabling 2FA on your server though.

Once you sign up for an account and confirm your email address, you can download the app to your phone, open it, and enter your credentials.

The process will be similar for other offerings if you choose one of those.

Once the server is set up, you’ll open your app when the Quick Response code (QR code) is displayed and capture it with the phone's camera for your app. The app will then have all the information it needs to synchronize with the server and generate your codes.

Ubuntu Server - SSH - 2FA Server Setup

In this lesson, we’ll configure the server to be able to support 2FA.

We’ll be installing an application called libpam-google-authenticator. Although supplied by Google, Google will not receive any data from your server. It is a stand-alone application.

Before starting, be sure the font size supported in your terminal emulator is relatively small. If you don’t, the QR code will not fit and won’t scan properly. If this happens, you’ll have to type the long secret key into the app manually. It’s a bit of a pain, but do-able.

Install the app by typing:

sudo apt install libpam-google-authenticator

Then run google-authenticator by typing:

google-authenticator

As you go through the prompts, be sure to select “y” when asked “Do you want authentication tokens to be time-based (y/n).”

Do you want authentication tokens to be time-based (y/n) y

You’ll get a QR code displayed on your screen.

You’ll also get some emergency scratch codes. You’ll want to copy these and put them in a safe place for storage, like a password encrypted USB drive.

Those are also one-time use, and can be used to log in if your phone is lost, stolen, or broken and you can’t get to your app.

Select “y” for the following questions:

Do you want me to update your “/home/username/.google_authenticator” file? (y/n) y
Do you want to disallow multiple uses of the same authentication token?…(y/n) y
By default, a new token is generated every 30 seconds by the mobile app… (y/n) y
If the computer that you are logging into isn’t hardened against brute-force… (y/n) y

Now, configure SSH to use google-authenticator during authentication.

Assuming you followed the process for managing SSH using a configuration file in /etc/ssh/sshd_config.d/, add the following lines:

UsePAM yes
ChallengeResponseAuthentication yes
AuthenticationMethods publickey,keyboard-interactive

Restart SSH:

sudo systemctl restart ssh

Now we’ll modify PAM to use 2FA for SSH logins.

sudo cp /etc/pam.d/sshd /etc/pam.d/sshd.0
sudo vim /etc/pam.d/sshd

Add the following lines to the file:

@include common-auth
#One-time password authentication via Google Authenticator
auth required pam_google_authenticator.so

I strongly recommend testing to be sure it’s working right before you disconnect the active session. This way, if something’s amiss, you can straighten it out before terminating the active session.

SSH - 2FA - Recovery

In this lesson you'll learn how to get on your server if you phone is lost or damaged or you don't have access to your 2FA App.

The obvious and easiest solution is to use one of your scratch codes.

If you kept those handy and you can find them when you need them, you'll use one of those. Copy it into your Operating System clipboard by selecting it then hitting Ctrl-C.

Begin the login to your server. Put in the passphrase for your SSH key. When prompted for your 2FA key, paste in the scratch code.

Once you're in, you can run google-authenticator again and it will generate a new code you can scan into a new phone.

What if you don't have access to your recovery codes? That could never happen, right? Of course it can happen. It's happened to me.

If you don't have your recovery codes, you'll need console access to your server.

If you're running a VirtualBox VM, just access it through the VirtualBox console.

If it's on DigitalOcean, you can bring up a console by clicking on the elipsis (...) to the right of the server you want and select Access Console.

You'll log in with a username and password.

Once you're on, you can look at the .google-authenticator file in the root of your home directory.

The top line is the one you'll want to put in your new phone if you want to use existing credentials.

Or, again, just run google-authenticator and generate a new code and scan it with your new phone.

That's it!

SSH Aliases with an SSH config file

In this lesson, we'll learn how to use an alias to SSH to servers we go to often.

When you SSH to something at the command line in Linux, you have to type something like:

ssh <username>@<ip-address>

Or

ssh theo@192.168.254.127

Along with the username and IP Address, you may have to type some additional options such as port number if you're using a port other than 22.

Wouldn't it be cool if you could type much less and get the same SSH connection?

You can, by using a config file in the .ssh folder of your home directory.

When you type ssh, one of the places ssh looks for instructions is a file called config in the .ssh folder of your home directory.

Just navigate to the .ssh folder in your home directory:

cd .ssh

Add a file called config with the following content at a minimum:

Host <name of this shortcut>
Hostname <ip-address of host>
User <username to authenticate with>

So, if I want to SSH to a server I want to call utuntu1 that has an IP Address of 192.168.254.127 using the username theo, my file would contain the following:

Host ubuntu1
Hostname 192.168.254.127
User theo

This seems like it's not a great benefit if you're only accessing one server with SSH. If you have 20, or 10, or even 5, it can be worth while to have shortcuts to them.

The name you specify for Host can be whatever you want and does not have to be related to the actual server, but it makes sense to name it the same as or similar to your server's actual name so you'll be able to keep track of what you're logging into.

If you use the same settings for many or all servers, you can have a wildcard entry at the bottom of the file for those common settings.

The following entry:

Host *
User theo
IdentityFile ~/.ssh/id_rsa

would specify a User of theo for any Host entry that does not contain a username in it.

It would also use the id_rsa identity file in the .ssh directory of your home folder, which is the behavior we want for key based authentication. It is default behavior so you don't have to specify it. It's just used as an example of what you can put there.

If you use a custom port and it's the same for all hosts, you can specify it with the wild card. If we use port 2222 for all our servers, it would be:

Host *
Port 2222

If you specify a different port for a particular host, your alias will use that instead of the wildcard port.

So:

Host ubuntu1
        Hostname 192.168.254.127
        User theo
        Port 4567

would use port 4567 whenever you type ssh ubuntu1.

Once you've defined some hosts in the file named config in your .ssh directory, you can just type ssh <hostname> to connect. How simple is that?

In the above example, to connect to ubuntu1, I would just type

ssh ubuntu1

That's it. Now you know how to define aliases for SSH in the config file.

Moving Files - scp Between Linux Systems

Eventually, if it hasn’t happened already, you’ll want to move files to or from your Linux server at the command line,

We’ll cover one of your options for doing that in this lesson.

Copying to the Command Line

If you try copying a file to your Linux server by dragging and dropping to a terminal, it won’t work.

The command line does not know how to interpret any of the signals generated in that attempt.

Instead, you have to use a program or command that operates at the command line.

One easy to use, secure program for doing this is the Secure Copy or scp.

scp Overview

Secure Copy operates on port 22, the same as SSH, by default, and it operates using the same mechanisms that let SSH work.

If you have SSH configured, you automatically have the ability to use SCP.

No additional configuration is needed.

It can be used to upload (take files from a location to your server) or download (put files on your server on a different computer) files and directories.

It is a secure way of authenticating and uploading or downloading files. The connection to the server and movement of files are all encrypted in transit.

Using scp

The syntax for scp is fairly straight forward once you’ve used it a few times. The two things you can do are download files or directories and upload files or directories.

Downloading

scp <username>@<from ip>:<file name> /local/directory/

where scp is the command, username is the username you want to connect with, from ip is the ip address of the server you want to download the file from, and file name is the file you want to download. /local/directory/ is the directory on the server you’re connecting to where the file is located.

It would look like this:

scp theo@192.168.254.105:ssh_settings /home/theo/

Would connect to 192.168.254.105 with the username theo and try to find a file called ssh_settings in the default directory on the remote server, and if it’s there, download it to the /home/theo/ directory locally.

The default directory on the remote server will be the home directory of the user you’re connecting with. In this example, it will be /home/theo/ on the remote server.

Uploading

To have a local file uploaded to the server you’re connecting to, the syntax is similar but reversed.

scp <file name> <username>@<to ip>:/remote/directory/

This will copy a file from the directory you’re in when you run the command to the directory you specify on the remote server.

scp ssh_settings_modified theo@192.168.254.105:/home/theo/

will copy the file from the present working directory named ssh_settings_modified to the server 192.168.254.105 connecting with username theo, and put it in the /home/theo/ directory on the remote server.

Copying Directories

You can add the -r flag to do a recursive copy, copying the contents of the directory you specify.

scp -r ssh_stuff theo@192.168.254.105:/home/theo/

Will copy the directory ssh_stuff and its contents from the local directory you’re in when you run the command to the /home/theo/ directory on 192.168.254.105.

Copying Between to Two Remote Hosts

You’re not limited to copying between the system you’re connecting from and the one you’re connecting to with scp.

You can also copy directly between two remote hosts.

The syntax is the same, except you’ll specify a login and password for both locations.

scp <username>@<from ip>:/remote/directory/file <username>@<to ip>:/remote/directory

So, the command

scp -3 theo@192.168.254.134:/home/theo/ssh_stuff_new theo@192.168.254.105:/home/theo/

will copy the file ssh_stuff_new from 192.168.254.134 to the /home/theo/ directory on 192.168.254.105, with the command being run from a completely different computer.

The -3 option will make the transaction occur via the computer running the command, so only its keys have to be on each remote server.

Otherwise, the public keys for remote server one will have to be on remote server 2, and vice-versa.

Troubleshooting

If you run into any issues, make sure SSH works for the username and systems you’re trying to copy files to and from.

Try a simple connection with SSH, without the SCP command.

If this works, check your syntax.

Also, make sure the user you’re logging in with has permission to manipulate the files in question.

Moving Files - Windows

If you want to copy files between a Windows computer, and a Linux computer, from the Windows PC, you’ll need an extra piece of software on the Windows side.

There are many options, but we’ll be using Cyberduck. I recommend against using FileZilla, but you may want to check out WinSCP if you don’t like Cyberduck for some reason.

Cyberduck

Navigate to cyberduck.io in your favorite web browser on your Windows system, and click on the download link for your operating system. It’s available for Windows, Windows 10, and MAC OS X.

As with any program, be very careful where you download the file from. From the source is best. Others may repackage it with bundled malware.

Double click the downloaded file and accept the defaults to install Cyberduck.

Using Cyberduck

Cyberduck is pretty intuitive for experienced Windows users. 

Find Cyberduck in your start menu and double-click on it to open it.

Click on Open Connection to create a connection for your server.

From the protocols dropdown list, select SFTP (SSH File transfer Protocol), make sure port 22 is selected, and put in your username and password.

Once connected, the gui interface is pretty intuitive. You can browse the file system on the remote server or click the Upload icon on the top menu bar to see a window allowing you to browse the local system for the file(s) you want to upload.

Key Based Authentication with Cyberduck

If you’re connecting to a server that requires key based authentication, Cyberduck can do that too.

Just go to Open Connection again, and after completing the other steps above, click on Choose… next to the SSH Private Key input window and browse to your private key file. It will end in .ppk if you generated it with PuttyGen.

When you connect, you’ll be prompted for the password you assigned to your private key file.

After that, all the rest is the same. Browse around and move your files as needed.

That’s it. Good job!

Moving Files - wget

As you learned in the last lessons, scp is an excellent tool, but it may not always be the best method for getting a file to your server.

You’ll come across situations where you want to get a file from a web site to your server, for example.

Thats where wget comes in.

We’ll cover wget in this lesson.

wget

We’ll use wget to download a tarred and zipped file with the latest version of they free Lynis security auditing tool from CISOfy’s web site.

Browse to https://cisofy.com/documentation/lynis/get-started/#installation-manual and scroll down to section 3, Installation via direct download.

We’ll deviate from the instructions provided and add a directory to /opt.

cd /opt

sudo mkdir lynis

Now, we’ll use wget to download the latest version from CISOfy to our server.

wget https://cisofy.com/files/lynis-<version>.tar.gz

As I’m creating this lesson, the latest version is 2.6.2-100, so we enter

sudo wget https://cisofy.com/files/lynis-2.6.2-100.tar.gz

It’s as simple as that!

Do an ls to make sure the file is there.

Verifying File Integrity

You can verify that the file you downloaded is the one created by CISOfy, by typing sha256sum and the file name, then comparing the answer to the one provided on the download site.

theo@Ubuntu-Remote:/opt$ sha256sum lynis-2.6.2.tar.gz

629efa33d684b61f0d9d5fe8ae1d3fe87a94dae5897cf90d5733af1552a55498  lynis-2.6.2.tar.gz

Compare the value starting with 629e… to the sha256 signature provided on the site:

SHA256 hash: 629efa33d684b61f0d9d5fe8ae1d3fe87a94dae5897cf90d5733af1552a55498

And observe that they’re the same. 

You can just check the first few and last few characters if you’d like.

This process ensures that the file was not tampered with on CISOfy’s site after it was uploaded by CISOfy, or in transit, or through any kind of redirection to another download.

Other wget Stuff

Please have a look at the man page for wget for more stuff it can do, but we’ll cover one here.

Using the -r option for recursive, you can download an entire copy of a web site!

If you type wget -r https://linuxliteracy.com, you’ll have an entire, local copy of my web site and all its pages.

It can be useful for looking at the innards of web sites if you’re interested.

In the next lesson, we’ll look at another useful tool called curl.

Good job!

Moving Files - curl

The last way we’ll cover to move files onto your server is curl.

curl, like wget, is a program that comes by default on Ubuntu.

curl

We’ll download Lynis again, this time using curl.

We’ll remove the Lynis related files and directories from /opt.

cd /opt

sudo rm -r lyn*

Now we’ll download the latest version of Lynis again using curl this time.

sudo curl -O https://cisofy.com/files/lynis-2.6.2.tar.gz

Make sure you specify the -O (that’s a capital letter “o”) option to tell curl to save the output as a file of the file name you’re downloading, or it will spew the output to standard output.

This is not the behavior we’re after. :-)

Other curl Stuff

Although you can’t do recursive downloads with curl, it supports many more protocols than wget, and supports application program interface (API) calls, and arguably do checking and troubleshooting on a web site more easily. 

We use curl’s header checking feature in our lesson on securing nginx.

curl -I https://linuxliteracy.com

returns

HTTP/1.1 200 OK

Date: Tue, 20 Feb 2018 00:47:38 GMT

Server: Apache

X-Powered-By: PHP/5.6.33

Link: <https://linuxliteracy.com/wp-json/>; rel="https://api.w.org/"

Content-Type: text/html; charset=UTF-8

I’m not super happy with the php version being displayed, so I’ll likely tweak that, but you get the idea.

That’s it for curl.

Good job!

Moving Files - Activity

Now that you know how to move files from your host to your Ubuntu Server, from Server to Server, and how to download files from web sites, it’s time to test your skills.

In this three part activity, you will

1. Download the update.txt bash script from this lesson then upload it from your host to your Ubuntu Server using scp.
2. Download the latest version of Lynis from the CISOfy web site to your server using curl.
3. Download a copy of the web site LinuxLiteracy.com your server using wget.

This activity will not walk you step-by-step through how to do the tasks, but will clearly spell out what you are to accomplish.

That gives you an opportunity to put the grey cells to work and hunt around a little if you have to.

You can return to the applicable lesson to refresh your memory.

Upload a File Using scp

Click on Resources Available in the upper left of this lesson, and click on the link on the left that will let you download update.txt.

Once it has downloaded to your host, rename the file to update.sh. Sorry to make you rename it, but Udemy won’t allow me to upload a file with a .sh extension.

Open the file and understand the content. If you’re not sure what it does, look up the commands in the man pages to make sure they won’t harm your system.

Upload update.sh to your home directory using scp.

Make sure it uploaded correctly, then change it to be executable.

Run it with sudo permissions.

This is a pretty useful file you may want to copy to all of your servers and run regularly.

Download Lynis

Download Lynis to the /opt directory your server using curl.

The site you’ll download it from is https://cisofy.com/files/lynis-<version>.tar.gz.

Replace <version> with the latest version. This can be found under “Version: “ on this page:

https://cisofy.com/downloads/lynis/

Once you’ve downloaded the file, make sure it’s legitimate by checking its hash, which is also available at the page just listed.

In the /opt directory, make a lynis directory.

Uncompress the *.tar.gz file you downloaded.

Navigate to the /opt/lynis directory and give it a run.

Download a Copy of LinuxLiteracy

The last item in this task is to download a recursive copy of the LinuxLiteracy.com web site to your server.

Navigate to your home directory.

Make a directory called LinuxLiteracy.

Change directory into LinuxLiteracy.

Download a recursive copy of https://linuxliteracy.com.

Look in the linuxliteracy.com folder that is created during the download and make sure there are files and directories there.

Look around as you like, then remove the files and directories you created from your home directory.

Excellent work! Doesn’t it feel good to put that knowledge to use?

Moving Files to and From Your Server

Part 1 - Upload a File Using scp

Download the file from the lesson called update.txt by clicking on Resources in the upper left of the assignment, then clicking on the update.txt link to download the file.

Browse to the file’s location and rename it to update.sh.

If you’re using Windows or MAC OS X and want to use one of the GUI’s, and you’re having trouble, please see the appropriate lesson from the section to review and try to fix the issue.

If you’re using the Linux command line, type the following to upload the file to your VM:

scp update.sh theo@192.168.254.105:/home/theo/

This command uploads a file in the present working directory on the local VM called update.sh to a server at 192.168.254.105 logging into it with the username theo, and putting the file in the /home/theo/ directory.

You’ll want to change to the directory where the file is stored using the cd command or put the full path to the file, and change theo to your username, the ip address to the ip address of the server you’re connecting to, and /home/theo/ to /home/<your user name>.

If you’re having difficulty, make sure SSH is working first by typing ssh <username>@<ip address>.

From the example above, that would be:

ssh theo@192.167.254.105

If that works, check the syntax of your scp command.

The auth.log file may provide some helpful clues.

You can also run scp with the -v option to run in verbose mode. It may tell you where the transaction is failing.

If you’re still having difficulty, please ask a question in the Q&A for the course and one of your fellow students or I will get to you as soon as we can.

Part 2 - Download Lynis

To download a file directly from the Internet, you have many options, but you’ll use the curl command in this assignment now.

In the /opt directory, create a new directory called lynis.

cd /opt

sudo mkdir lynis

Now download the zipped file from CISOfy.com

sudo curl -O https://cisofy.com/files/lynis-2.6.2.tar.gz

Change 2.6.2 to the latest version, as this may not be the latest.

You can find the latest version at https://packages.cisofy.com/

Look under Packages, and Lynis.

Also, be sure to specify the -O option. If you’r curious to see what would happen if you run the command without -O, please try it. You won’t hurt anything, but you’ll get interesting output to the console.

Without -O, curl outputs directly to std-out. Output can be wild when you sent a binary file to the console.

Do an ls to make sure the file was downloaded.

theo@ubuntu-server:/opt$ ls

lynis  lynis-2.6.2.tar.gz

Unzip the files.

sudo tar -xvzf lynis-2.6.2.tar.gz

If you’d like to run lynis to have it scan your system and generate a report you can use to secure your system, type

sudo ./lynis audit system

You can look at the report and harden some settings if you like.

Part 3 - Download a Copy of LinuxLiteracy

Now, we’ll make a local copy of a web site using wget.

We’ll make a copy of my LinuxLiteracy web site because I know it’s relatively small, yet has some complexity to it.

Make a directory to store a copy of the site in your home directory

cd ~

mkdir LinuxLiteracy

cd LinuxLiteracy

Now, download a recursive copy of the site using wget

wget -r https://linuxliteracy.com

Browse around a bit to see the files that were downloaded.

cd linuxliteracy.com

theo@ubuntu-server:~/LinuxLiteracy/linuxliteracy.com$ ls

2018              index.html?p=112  index.html?p=87

about             index.html?p=117  index.html?p=90

author            index.html?p=12   index.html?p=93

category          index.html?p=124  index.html?p=97

comments          index.html?p=127  linuxliteracycourses

feed              index.html?p=131  robots.txt

hello-world       index.html?p=4    top-5-free-tutorials-for-linux-beginners

index.html        index.html?p=64   wp-content

index.html?p=1    index.html?p=68   wp-includes

index.html?p=100  index.html?p=73   wp-json

index.html?p=103  index.html?p=78   wp-login.php

index.html?p=105  index.html?p=81   wp-login.php?action=lostpassword

index.html?p=109  index.html?p=84   xmlrpc.php?rsd

That’s it! You’ve completed the assignment and learned some valuable new skills. Great work!