
Understand data pipelines as a three-step flow from sources to transformation to destination, enabling data from databases and SaaS to be loaded into Snowflake, Redshift, or BigQuery.
Explore the three ETL stages—extraction, transformation, and loading—and how data moves from sources through a staging area to a data warehouse for analytics.
Identify and address traditional ETL challenges, including everchanging schema, synchronized data sources, manual pipeline creation, and ongoing maintenance to prevent pipeline breakage.
Understand how Fivetran acts as a fully managed data pipeline that centralizes data sources to data warehouses using 150+ connectors, with no coding and UI-driven transformations for effortless ETL.
Create a free Fivetran account and start a 14-day trial to access data sources, connectors, and prebuilt data models. Verify your email, explore the dashboard, and set up a connector.
Explore five Fivetran plans, from starter for small teams to business critical for global enterprises, detailing real-time synchronization, security, compliance, and fully managed connectors.
Connect a destination to Fivetran to synchronize data from sources into cloud data warehouses, databases, and data lakes, using destinations like Azure Synapse, BigQuery, Snowflake, and SQL Server.
Explore two destination account types in Fivetran, create and test a destination—own or managed—and view your project's ID.
Sign up for a free Snowflake trial, choose the enterprise edition on AWS in Northern Virginia, and complete the terms. Wait for the activation email as the account setup begins.
Create a Snowflake free trial account, sign in to the workspace, switch between the classic console and the latest UI, and prepare for data uploading in the next video.
Set up a snowflake destination in Fivetran by configuring the warehouse type, creating the necessary Snowflake roles and database objects with a guided script, and testing the connection.
Connect to a snowflake destination from a Google Sheet source, review the schema, opt to block or hash sensitive columns, then sync data and monitor logs and notifications.
Create a Redshift cluster by accessing the management console, selecting production defaults, changing the password, and monitoring its status as it moves from modifying to completing.
Learn to view a redshift cluster’s general information, including namespace, status, node type, and endpoints, and configure network and security, database settings, default database, port, and admin username.
Add redshift as a destination in fivetran and follow the setup guide to connect, configuring host, port 5439, database, user, password, region, and security group rules.
Access the cluster's property section to view namespace, status, node type, and endpoints, and configure network, security, and database settings including the default database, port, and admin username.
Learn how transformations use SQL scripts to shape raw data from multiple stores into a consistent format before loading into a destination, with automated reruns and raw audit retention.
Connect Google Sheets as a data source in Fivetran, configure the destination employee data schema, set the sheet name range, authorize access, test the connection, and run the initial sync.
Learn to create a full name transformation by concatenating first and last names, build a view with sql, and run, schedule, edit, or delete the transformation on employee data.
Install and configure dbt transformations for a snowflake destination by setting up a conda environment, installing the dvd core and plugins, and cloning a GitHub repository.
Learn to set up a GitHub repository: create a private repo named BTV one, add a readme, clone project, and sign in with the code for the first dvd project.
Create a new dbt project, connect to snowflake with account, username, password, and admin role, configure the database and public schema, then run and verify sample models.
Build a sample dbt model within the DVD project, configure profiles and snowflake transformation, add a github deploy key, define a demo model schema, and prepare integration with fivetran.
Push your local files to an empty GitHub repository, check the status, and prepare the first commit to set up the project in version control.
Navigate project files, including analysis and micro models, note the HTML file deployment, and edit the HTML file, then fill the form to prepare for next week.
Set up a deployment.ml file to schedule dbt commands for a db project, defining multiple jobs with steps like run models, dbt run, and test models.
Save and test the dbt transformation by resolving connection issues, aligning deployment and project folders, validating the connection, and running the transformation with run now to confirm results.
Run dbt transformation workflows by executing run models and test models, review job logs and execution status, and verify contracts across multiple models.
Add configuration details from the profile, set default schema name, select credentials, enter the warehouse, database, user, and role, input the password, then save and test the connection.
Review logs to fix errors, remove the hyphen from selection criteria, and run all models; fix the failing test by editing the example model and verify data in Snowflake schema.
Explore the Fivetran architecture from data sources through the connector, core, and writer to the destination. Learn how pull and push connectors securely transfer and load data using encrypted connections.
Master how Fivetran handles syncs with initial historical sync, incremental updates, and resync, including how Google Sheets data is copied, updated, or overwritten and how to set frequencies.
Learn how to set up and connect a BI tool to a managed destination, authorize Tableau or other BI tools, edit or delete connections, and add new BI tools.
Master account management with destinations, teams, users, and roles; configure access control, assign roles like account administrator or destination creator, and manage billing, settings, and usage across connectors and transformations.
Create and manage a custom role in fivetran by selecting destination level permissions, enabling users to manage destinations, logs, transformations, and connectors, then save and edit or delete as needed.
Learn how to add users, assign an account analyst role, invite them, and manage team members in the teams section to enable access once invitations are accepted.
Monitor the current status of vital services on the status page by subscribing to incident maintenance downtime updates, and access destination details, dashboards, transformations, and APIs for about 510 services.
Master Fivetran notifications to monitor connector, transformation, and account events via email alerts. Configure recipients and settings to receive timely updates on errors, delays, and new users.
Explore alerts that flag issues in your connectors or transformations, distinguishing errors that require fixes from warnings you can glance at. Resolve errors to maintain synchronization and quickly diagnose problems.
Imran introduces Fivetran as a cloud data integration platform, explains its architecture and connectors, and demonstrates a day one setup connecting Google Drive to Snowflake, performing an initial sync.
Explore fivetran deployment options—cloud, private, and hybrid—and master connectors, destinations, and elt workflows, including SQL Server, Snowflake, and GCP managed setups, with data types and transformations.
Explore naming conventions and release phases, set up Google Sheets and SQL Server connections, and implement initial and incremental data sync across Snowflake and AWS.
Build an end-to-end data pipeline from SQL Server to Snowflake, perform initial sync and raw-to-model transformations with DB2 and dbt, and manage deployment via Git and deployment.yaml.
Requirements
Basic understanding about ETL and Basic SQL will be useful, but not necessary.
I will take you through everything necessary to learn this course.
Description
Welcome to Fivetran Bootcamp! Fivetran helps you centralize data from disparate sources which you can manage directly from your browser. In this course you will learn to extract your data and load it into your data destination. This course will help you in preparing and mastering your Fivetran ETL tool concepts.
Highlights of the Course:
Designed to keep only précised information no beating around the bush.
Real-time implementation, learn with Practical.
It will help you to showcase your experience in interviews and discussions.
Involve complex architecture solution which is aligned with industry best practices.
Added the Best Practices to be used with Fivetran
Topics Covered in the Course:
What is a data pipeline
What is ETL
Challenges in the traditional ETL
what is Fivetran
Create A Free Account
Fivetran plans
Fivetran types of destinations
Types of Accounts and Create Managed destination
Create Snowflake Free Trial Account
Create Redshift Cluster
set up Redshift destination
set up snowflake destination
Fivetran and DBT set up
what is Transformation
Fivetran Data Transformations
Fivetran Acrhitecture
Fivetran: Account Management Overview
Fivetran helps you centralize data from disparate sources which you can manage directly from your browser. We extract your data and load it into your data destination. If you already have an account, setup takes anywhere from 5 minutes to 2 hours (depending on the complexity of your internal firewalls and integration setups). The first step is to connect a data warehouse. After that, you'll be taken to the Dashboard, where you can add new connectors. Adding a new connector will lead you to the setup page with detailed notes on the screen and verification for the configuration. Once your first connector is created, it will begin syncing immediately and once that is done you can start using that data in your data warehouse. You can add as many connectors as needed to complete your initial setup.
Who this course is for:
Data engineering Professionals
University students looking for a career in Data Engineering
IT developers working on other disciplines trying to move to Data Engineering
Data Engineers/ Data Warehouse Developers currently working on on-premises technologies, or other cloud platforms such as AWS or GCP who want to learn Azure Technologies
Data Architects looking to gain an understanding about cloud based ETL tool
Data Scientists who want to extend their knowledge into data engineering