The Complete Hands-on Introduction to Airbyte
What you'll learn
- Understand what Airbyte is, its architecture, concepts, and its role in the MDS
- Install and set up Airbyte locally with Docker
- Connect Airbyte to different data sources (databases, cloud storages, etc)
- Configure Airbyte to send data to various destinations (DWs, databases)
- Develop a data pipeline from scratch with Airbyte, dbt, Soda, Airflow, Postgres, and Snowflake to run your first data syncs
- Set up monitoring and notifications with Airbyte
Requirements
- Prior experience with Python
- Access to Docker on a local machine
- A Google Cloud account with a billing account (for BigQuery)
Description
Welcome to the Complete Hands-On Introduction to Airbyte!
Airbyte is an open-source data integration engine that helps you consolidate data in your data warehouses, lakes, and databases. It is an alternative to Stich and Fivetran and provides hundreds of connectors mainly built by the community.
Aibyte has many connectors (+300) and is extensible. You can create your connector if it doesn't exist.
In this course, you will learn everything you need to get started with Airbyte:
What is Airbyte? Where does it fit in the data stack, and why it is helpful for you.
Essential concepts such as source, destination, connections, normalization, etc.
How to create a source and a destination to synchronize data at ease.
Airbyte best practices to efficiently move data between endpoints.
How to set up and run Airbyte locally with Docker and Kubernetes
Build a data pipeline from scratch using Airflow, dbt, Postgres, Snowflake, Airbyte and Soda.
And more.
At the end of the course, you will fully understand Airbyte and be ready to use it with your data stack!
If you need any help, don't hesitate to ask in Q/A section of Udemy, I will be more than happy to help!
See you in the course!
Who this course is for:
- Data Engineers
- Analytics Engineers
- Data Architects
Instructor
- 4.6 Instructor Rating
- 18,471 Reviews
- 105,753 Students
- 5 Courses
Hi there,
My name is Marc Lamberti, I'm 27 years old and I'm very happy to arouse your curiosity! I'm currently working as Big Data Engineer in full-time for the biggest online bank in France, dealing with more than 1 500 000 clients. For more than 3 years now, I created different ETLs in order to address the problems that a bank encounters everyday such as, a platform to monitor the information system in real time to detect anomalies and reduce the number of client's calls, a tool detecting in real time any suspicious transaction or potential fraudster, an ETL to valorize massive amount of data into Cassandra and so on.
The biggest issue when you are a Big Data Engineer is to deal with the growing number of available open source tools. You have to know how to use them, when to use them and how they connect to each other in order to build robust, secure and performing systems solving your underlying business needs.
I strongly believe that the best way to learn and understand a new skill is by taking a hands-on approach with just enough theory to explain the concepts and a big dose of practice to be ready in a production environment. That's why in each of my courses you will always find practical examples associated with theoric explanations.
Have a great learning time!