Practical Apache Airflow
What you'll learn
- Airflow, Developing Data Pipeline, Python, Pandas
- Hands on data pipeline development and comparison with other technology
- Installation, Configuration of Airflow
- DAG's, Creating workflow, Operators, Tasks, dependency management, Hooks, Connections
- Different Executors like Local, Celery and Sequential and differences
- Airflow Architecture in detail
- Advanced concepts like Xcoms, Branching, Variables and DAG Chaining
- Authentication and Log storage to S3
- Airflow on Docker
Requirements
- Beginner Python experience will be helpful
Description
Data engineering is a field that can be thought as a superset of business intelligence and data warehousing which brings more elements from software engineering. The reason data engineering exists today is because companies have massive treasure troves of data, but to provide value the data must be extracted. Data engineering provides the toolbox and is how we make sense of that data quickly and effectively.
When it comes to managing data collection, munging and consumption, data pipeline frameworks play a significant role and with the help of Apache Airflow, task of creating data pipeline is not only easy but its actually fun. Originated from AirBnb, Airflow soon became part of the very core of their tech stack.
The data infrastructure ecosystem has yet to show any sign of converging into something more manageable. It seems like we’re still in a huge phase of expansion where every new day bring new distributed database, new frameworks, new libraries and new teammates. As these systems get more complicated and evolve rapidly, it becomes even more important to have something like Apache Airflow that brings everything together in a sane place where every little piece of the puzzle can be orchestrated properly with sane APIs.
So in this course we will be learning as how to reach feature completeness with this amazing orchestration tool called Apache Airflow. You will not only learn to setup the environment but also learn how to create workflow pipeline with real world example so don't wait and sign-up today and get started.
Looking forward to seeing you in this course!
Who this course is for:
- Students who want to learn developing data pipeline
- Students and Professional wanting to learn Airflow
- Professional looking to implement Airflow for their team and organization
Instructor
![DevTechie Inc](data:image/svg+xml,%3Csvg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64"%3E%3C/svg%3E){kind=avatar}
- 3.8 Instructor Rating
- 239 Reviews
- 1,006 Students
- 58 Courses
Hello World from DevTechie. We have several years of experience working for big tech companies to small startups with 40+ years of combined experience in developing apps and softwares. We believe in sharing the knowledge gained over the years and would like to invite you to join us in this journey. We cover end to end development for mobile, web, backend as well as data science and machine learning.