Full Spectrum Comprehensive Testing & Error Reading 101

Name: Full Spectrum Comprehensive Testing & Error Reading 101
Rating: 4.8 (6 reviews)

Interview Prep: Solving Tickets for Real life Job Like scenarios for Python Big Data Science 101 Courses

Created byShivgan Joshi, abhi verma, Prakash Chandr Nandanwar

Last updated 10/2024

English

What you'll learn

Full Spectrum Comprehensive Testing & Error Reading
Solving Tickets for Real life Job Like scenarios
Learn how to fix errirs at differnt stages for differnt repos
Many checkpoints to prepare you for the interview

Course content

7 sections • 30 lectures • 32m total length

Introduction0:41
Explore full spectrum testing and error reading across unit tests, Jenkins builds, and big data implementations. Learn to handle, debug, and test errors with full stack and exploratory engines.
Instructions to take the course0:49
Complete all checkpoints and true/false questions, then apply the provided steps to the final test and project. Reach out with questions to maximize learning.
Content of Course0:21
Content of the Course2:49
Master ticket-based learning across full stack, simulation, and exploratory repos with local, Jenkins, grid, and full-stack runs. Learn to diagnose Jenkins, PySpark, and environment-related errors to get things done.
Ticket based Learning1:16
Explore ticket-based learning by examining raised failures, fixing them quickly, and producing high quality notes. Tackle Jenkins and PySpark errors by categorizing tickets and crafting solutions.
Three types of Repos2:03
Identify the three repo types: full stack frameworks, engine, and exploration; explain their roles, data orientation, and how full stack integrates engine, exploration, and user interface for end-user workflows.
Four types of Run2:12
Explore the four run types across engine, exploratory engine, and full stack framework, covering spark submit runs, data formats like hive, parquet, and snappy, and UI vs shell-based workflows.
Four Stages of Run0:50
Identify and analyze errors across four stages of run, from local unit testing to Jenkins builds, grid runs on big data, and UI input, to learn how to solve them.
Ticket based Learning

Local Runs Intro1:52
Local runs on our system use a window-based virtual environment to run and debug any repository with pytest, and step through code in VS Code or PyCharm.
Checkpoint Local Run
Fixing Local vevn in Python to make local test pass0:07
Different ways to Install Venv Python local0:08
Env Variable for User in Windows0:05
Java Pyspark Hadoop local folder etc0:06
Java Hadoop Pyspark for Local runs0:39
Configure and install local Java, PySpark, and Hadoop with matching versions for full-stack simulation runs. Align Hadoop 2.7, Spark 4.0, and JDK 8 by reading requirements and setting environment variables.
env variables in Windows1:34
Learn how windows environment variables support full stack runs, enabling local hive tables, spark sql views, and the required Hadoop and PySpark setup.
Diff ways to install venv for Python Local0:49
Learn the different ways to install a Python local virtual environment, using PyCharm or custom scripts, with pip install requirements and Git Bash considerations.

Jenkin failures for Diff Repos4:31
Explore Jenkins failures across three repositories and four stages, navigate ci-cd yaml settings, learn how to disable tests or deployment, handle timeouts, and address issues in full stack builds.
Checkpoint Jenkins Failure
Checkpoint Connection Timeout Jenkins
Connection time out issue for Jenkins1:26
This lecture explains common Jenkins connection timeout issues and deployment errors. It highlights causes like parallel builds, wrong table names, and the role of restarts in full stack environments.

Checkpoint Full stack Sim Framework
Workflow vs Task level testing0:39
Examine workflow versus task level testing in full stack computation, noting that workflows are more complex and prone to errors; test locally, then on Jenkins, then on the UI.
Fix local venv for full stack0:50
Fix the local venv for full stack by updating the virtual environment to the latest version and packages, following framework-specific steps beyond pip install requirements.
Deployment Errors in FT frameworks0:05
Deployment Errors in FT frameworks1:07
Testing Connections to Big Data HIVE Tables1:09
Test connections to big data Hive tables when a table is not found by verifying existence, live status, or changes via Hugh Portal, PySpark, or a Jupyter notebook.
Testing and matching golden copy runs - Grid results matching1:12
Compare golden copies with current outputs to validate grid results, aligning parameters like years, paths, and UI inputs, while injecting data frames into testing packages.
Checkpoint - Testing and matching golden copy runs - Grid results matching

Top 5 cause Jenkins failures0:04
Top 5 workflow based error0:01
Top 5 causes full stack sim framework1:45
Identify the top causes of failure in a full stack simulation framework, including conventions, object oriented programming and decorators, hive connections, and cross-workflow dependencies.
Top 5 causes of Jenkins Failure1:21
Identify the top five Jenkins failure causes, from timeout in full stack simulations to unit test value mismatches, import and relative referencing issues, and ODBC data for unit tests.

Requirements

101 course for Begineer, no coding needed

Description

Interview Prep Full Spectrum Comprehensive Testing & Error Reading 101

Solving Tickets for Real life Job Like scenarios

Ticket Based Learning
Three type of Repo
Four types of Run
Checking YAML and CICD for Jenkins build
1. Different kind of Jenkins failure
2. Connection timeout issue for Jenkins
Full Stack Workflow Computational UI runs
1. Workflow vs Task Level Testing in full stack computation Framework
Checking Py-spark Errors
Fixing Local Vevn in Python to make local test pass
Different ways to Install Venv Python
Env Variable for User in Windows
1. Java Pyspark Hadoop local folder etc
Top 5 causes of each failure types
Release Process Errors

Ticket based Learning

Is the most common request Jenkins Failure? True
Local failure are most common due to pyspark? True
Do we get fixing errors as our main ticket? True
Is the most common ticket Jenkins failure? True
Is the most error we will deal is with full stack simulation framework? True

Top 5 cause Jenkins failures

Timeout
Unit tests: Values mismatch
Import problem
Connection to tables using odbc
Many Jenkins build of same code running in parallel

Top 5 workflow based error

Axis Node Name mismatch
You forgot to run workflow integrity test locally
Hive connection failure
Same class instance used by another test
Latest version not used for dependent repos, FT Sim Framework depends on different repos like engine, simulation, ui etc

Who this course is for:

Beginner

Full Spectrum Comprehensive Testing & Error Reading 101

What you'll learn

Explore related topics

Course content

Introduction8 lectures • 11min

Local Unit Testing8 lectures • 5min

Fixing Jenkins Build2 lectures • 6min

Full Stack Workflow Computational UI runs6 lectures • 5min

Grid Spark Errors1 lecture • 1min

Top 5 Causes of Diff Errors4 lectures • 3min

Final Project & Takeaway1 lecture • 1min

Requirements

Description

Who this course is for: