Linux for Data Engineers (Hands On)

Explore the filesystem hierarchy standards, including bin, boot, device files, and essential libraries. Understand how removable media are mounted and how system binaries and bootloader components are organized.

Explore the Ubuntu file system by navigating files and folders, using the terminal to access the command prompt, and understanding 32-bit vs 64-bit differences in root and boot configurations.

Adjust playback speed, video quality, and captions to tailor your viewing; captions are auto-generated, use the full transcript for reference, and consider leaving a review.

Learn to use apt as a packaging manager to install, update, remove, and search software from the terminal, with a hands-on demo of installing nano.

Explore man pages, the Unix manual documentation, and use the man command to access software docs; navigate topics like make and print through practical demonstrations.

Learn basic Linux operations for data engineers in this hands-on session, practicing cd, ls, and cat on a Hadoop environment.

Practice mkdir, rm, and exit as you create and remove directories, clear the screen, and exit the terminal, demonstrated by navigating to /home/bigdata in a hands-on linux data engineering drill.

Explore basic operations of two commands to locate source or binary files, display the present working directory, and perform a practical demonstration of locating Java installations.

Learn how the Linux head and tail commands display the first and last lines of a file, with practical examples to view content efficiently.

Master hands-on Linux file operations for data engineers. Practice listing with ls -la, updating times with touch, and renaming or removing files using mv and rm.

Learn to monitor and manage processes with ps and top, terminate unresponsive tasks using kill, and copy files with cp for efficient data workflows in linux for data engineers.

Learn basic Linux compression operations by practical demonstration of gzip and zip, creating and decompressing sample files to illustrate common compression workflows.

Continue basic operations by downloading the official data package, unzipping it into a target directory, and examining how data is organized in a Hadoop 3.2.0 directory through a practical demonstration.

Continue exploring basic operations, search for patterns, display matching lines, and count lines, words, and characters with wc in a practical demonstration.

learn to efficiently view large log files with less, navigate with page up/down, search with slash, and use the date command for formatting and changing the system date.

Explore practical use of df, du, and top commands to monitor disk space, file sizes, and running processes, with examples and tips on usage and interpreting outputs.

Discover how the history command shows the list of commands you have executed in the terminal, including the last commands like clear, scoop import, and Hadoop commands.

Explore system log files in /var/log, including auth log, lastlog, and faillog, to monitor authentication events, failed logins, and package installations or removals using the package command.

Learn how to set and interpret file permissions with chmod, assigning read, write, and execute to owner, group, and others, using numeric modes like 777 and 766.

Learn to use nano, a simple Linux editor, to open or create files and edit content. Save with Ctrl+X as shown in the hands-on demonstration.

Explore the vi editor in Linux, learn to open and edit files, insert text, and save or exit with commands like :w, :q, and :wq.

Craft and execute a basic bash shell script with a /bin/bash line, edit with nano, and print two lines 'big data' and 'engineer' by running the script.

Explore features of shell scripts, including automating tasks, reducing error risk, and turning long sequences of commands into a single, reusable command using various utilities.

Learn to build a bash interactive shell script that reads a user’s name and displays it back, using read and echo commands in a practical hands-on example.

Learn how to define and call functions in shell scripting, using a function name and a block of statements to produce before and after outputs.

Learn how to pass and access script parameters in bash, using $0 for the script name and $1, $2, $3 for the first, second, and subsequent arguments, with practical demonstrations.

Explore the if statements syntax, compare equal and not equal conditions, and use a practical script with variables to demonstrate if, else, and execution.

Learn how boolean expressions evaluate to true or false using and, or, and not operators, with a practical demonstration and a sample script dataset showing conditional checks.

Discover how the case statement works with a fruit variable, matching patterns in a case expression and displaying orange when it matches, while executing corresponding commands.

Explore how a for loop iterates over items, repeating a command for each item in a list, with a bash example and a practical demonstration establishing the flow.

Explore how a while loop evaluates its condition and terminates when false, demonstrated with a script that counts from 10 down to 1.

Learn how the until loop executes until its condition becomes true, repeatedly evaluating commands. The demonstration prints numbers from 1 to 10 using an increment expression like i++.