Linux Bash | Automate Like a Pro-Start Solving Real Problems

Master Linux Bash to automate real-world problems, elevating efficiency and problem solving, while exploring quotes as food for the mind to fuel practical thinking.

Learn the basic structure of a bash script, including the shebang, dynamic log file naming from the script name, command substitution, and a function-based logging workflow.

Back up files safely in bash by renaming 2024 files to .bcp, moving them to the data files backup folder, and syncing with rsync using include and exclude rules.

Implement purging by encapsulating deletion of log files older than thresholds such as 140, 100, or 70 days, using the find command to free disk space and log deletions.

Archive and compress a data folder in Linux using tar and gzip, creating a tar.gz file with verbose logging and date-based naming, then extract with tar -x.

Automate archival and compression by passing directory and compression type as script arguments, capturing them with $1 and $2, and validating inputs before tar or Jeep compression.

Learn to compare files and directories in Linux using diff, rsync, and compare, illustrated with data one and data two folders, covering file-level, line-level, and byte-level differences.

Learn how the split command divides large files into smaller chunks using -B and -l. Configure numeric suffixes and line-based splits to preserve complete records.

Explore automating file splitting in bash with split by bytes and split by record, using read input and awk to compute sizes and generate numbered chunks.

Calculate total disk usage and monitor space with du and df, show human readable output, and identify the top ten space-occupying directories using descending sort and summarize options.

Develop linux bash automation skills to solve real problems, while drawing inspiration from quotes that feed the mind.

Learn to extract meaningful insights from log files using grep with options like -i, -L, -n, -e, -A, -B, and -C to search for errors and warnings in webapp.log.

Learn the sed stream editor to substitute patterns with delimiters, print lines or ranges, delete data, and perform in-place, global, case-insensitive substitutions with -e and y.

Automate bank-customer data processing with sed, updating short status 'in' to 'inactive', extracting inactive records with grep, and appending header and footer lines with current date.

Learn to use awk to process structured data with begin, main, and end blocks, extract columns, and compute per customer, per source totals and counts for bank transactions.

Master advanced awk to aggregate customer transactions by source, using begin and end blocks, associative arrays, sum and count, and pattern filters with -v for name matching.

Learn how to perform character-level data transformations with the translate or transform utility (tr), including case conversion, delimiter changes, character deletion, squeezing repeats, and selective complementing to keep only numbers.

Mask sensitive data in fixed-width records by hiding the last eight digits with stars, reversing characters 18–20, and preserving the first 17 characters using cut, rev, and translate in bash.

Learn to use sort and unique to organize flight data: sort by fields, use a comma delimiter, apply numeric (-n) and reverse (-r) options, and identify duplicates (-D) or uniques (-u).

Use bash scripting to extract flight insights from a comma-delimited dataset with awk, sort, and uniq, revealing most frequent airline Delhi to Mumbai, total unique routes, and most delayed flight.

Explore how a quote acts as food for the mind while using Linux bash automation to start solving real problems.

Explore how quotes nourish the mind while mastering Linux Bash workflows to automate tasks and start solving real problems.

Explore the world of regular expressions as a powerful pattern-matching tool, and learn how the regex engine substitutes and uppercases matches with a power utility.

Learn regex substitution in bash using dot and the global flag to replace characters with asterisks. Perl provides look ahead and look behind for tasks like validating or masking emails.

Explore how the regex engine scans input left to right, testing patterns at each position, using quantifiers like L plus and L star to match and substitute.

Discover how the dot wildcard matches any single character in a linux perl one-liner, learn substitution with s/./#/g, and note that the dot does not match newlines, with practical examples.

Explore the star quantifier in regex to match zero or more occurrences of any character or group. See practical Perl examples with dot-star substitution and newline handling via chomp.

Explore how the question mark quantifier matches zero or one occurrence, and apply perl -n -e to print and substitute patterns with hash using group patterns.

Explain how the plus quantifier matches one or more preceding character or group in regex, using perl -n data to substitute matches globally with a hash and illustrate greedy behavior.

Learn to use the curly brace quantifier in regex to match six-digit zip codes and filter records, with boundary checks and Perl one-liners.

Learn how the escape character in regex treats special characters like dot and dollar as normal, enabling precise printing of records containing these literals.

Learn to use regex anchors to match patterns at the start or end of a record, apply negation with !^, and perform substitutions with escaping dots in Perl.

Explore how character classes in regular expressions use square brackets to match sets of characters, including ranges, anchors, and negation, with practical bash examples of alphanumeric patterns.

Learn capturing groups in regex with Perl, build and reference groups, swap id and name through substitutions, and use backreferences to print, replace, and deduplicate patterns.

Decide between greedy and non-greedy regex patterns by choosing the biggest match or the first match, and learn to use dot star, escaping, and the question mark to control greediness.

Learn to use the word boundary \b in regex to identify word positions and extract words like hello and world by substituting a delimiter.

Explore how to use positive and negative look ahead and look behind in regular expressions, with real-world examples from banking transactions and email domains to control printing of matches.

Explore positive and negative lookahead and lookbehind in regex with practical examples, including printing transaction IDs for ATM transactions and validating emails.

Learn linux bash to automate tasks like a pro and solve real problems with efficient scripting.

Tackle regex challenges in Linux Bash, including masking emails, validating IP addresses, extracting hashtags and mentions, printing lines that start and end with same word, and replacing digits outside words.

Master practical regex techniques in Bash to solve real problems, from masking email addresses and validating IP addresses to extracting hashtags and mentions, using boundaries, lookahead, backreferences, and substitution.

Explore Linux Bash strategies to automate tasks and solve real problems, guided by the idea of feeding the mind with quotes.

Develop a self-healing Linux bash workflow that keeps web, db, and ETL services up by using Redis and netcat, a daemon script, a Redis watchdog, and a cron job.

Install Redis server and netcat on ubuntu with apt-get, then start and verify Redis on localhost port 6379, preparing for bash automation in the next tutorial.

Automate redis checks with a bash script using netcat in a while loop to detect readiness. When down, start the redis server and daemonize the script with nohup and ampersand.

Build a bash watchdog that monitors the Redis db process and restarts it when it stops, using ps and grep to detect status and logging with nohup.

Learn how to use crontab to schedule the watchdog, keeping the Redis DB up, and understand cron syntax from minute to day-of-week with a two-minute example.

Learn how machines talk to each other using SSH for passwordless authentication across development, testing, and production environments, including file transfer, upstream/downstream workflows, and using authorized_keys.

Learn to connect from a cloud shell to a remote Linux server via SSH, use external IPs and internal IPs, configure firewall rules, and implement public key authentication with authorized_keys.

Use sftp to securely transfer files between local and remote servers, navigating directories with commands such as pwd, and using get and put to copy files.

Demonstrate scp one-liner transfers: copy SAP_remote.dat from remote data/files to the local directory, then SAP_local.dat from local to remote data/files.

Automate file transfers between cloud shell and a vm with scp and sftp in bash. Parameterize username and ip, define local and remote paths, and manage push and pull operations.

Learn to automate tasks with Linux bash, start solving real problems, and explore quotes about food for the mind.

Explore practical Bash concepts like variable handling, command substitution, and graceful exit to automate tasks and avoid leaving temp files when a script is interrupted.

Demonstrate indirect variable referencing in bash by using the exclamation mark to first resolve var name to its value and then resolve the referenced variable’s value.

Explore how the bash eval command evaluates a greetings variable to produce hello world, and learn the security risk of using eval, including sequel injection.

Learn to trap signals in bash to ensure graceful exits by cleaning up temporary files. This lesson demonstrates handling sigint and other signals with trap and cleanup routines.

Learn how to capture the exit code of the last command with $?, interpret zero as success and non-zero as failure, and recognize 2 and 126 to guide Bash logic.