
Learn to load data with Snowflake and Informatica, build dashboards to validate mappings, and develop Streamlit apps for updating data and dimensions using Python.
Sign up for a 30-day Snowflake trial, choose AWS and Sao Paulo, and complete account details. Use the attached data, mapping, calendar, and Informatica transformations to create tables.
Register and activate your Snowflake account using the activation link within 72 hours, set a strong username and password, and begin creating the tables for the course.
Create and organize Snowflake objects by using a sysadmin role to set up the IX course database, establish stage, PSA, and data warehouse schemas, then analyze data and create tables.
Create the product mapping table as a transient stage table, replace text keys with a numeric product code, and set up brand and line extension lookups with a primary key.
Create a touchpoint mapping table with a touchpoint key, a touchpoint code key, and media, sub media, and platform lookups, then design a date-rich calendar dimension on the psa layer.
Create the media spend fact table from date, brand, line extension, and touchpoint mappings. Enforce not null constraints and use lookups for stage and PSA tables to preserve history.
Create the final course table with start date, product code, touch point code, value, metric and modified timestamp, include media spend, and apply mapping logic with lookups.
Create an Informatica IICS data integration trial account by entering your details, verify the email, and choose a cloud region aligned with your Snowflake on AWS to reduce data transfer.
Download and install a secure agent, link it to your Informatica instance with a token, and verify services run to enable connections to your data sources.
Create and test a flat file connection by assigning a project tag, specifying a runtime path, and adjusting date format and UTF-8, while addressing administrator access for the secure agent.
Enable the Snowflake data cloud connector in Informatica, configure standard authentication, set the account identifier, select the compute warehouse and sysadmin role, test and save the connection.
Set up a parameter file to manage database connections across dev, uat, and production in Informatica, linking a global db name to the environment and preparing for environment migrations.
Troubleshoot and fix Snowflake object ownership in Informatica connections by switching between sysadmin and account admin roles, and granting ownership across databases, schemas, tables, and warehouses.
Define and execute a stg product mapping from a flat file to Snowflake, generating a CRC32 product code and a parameterized task flow for scalable data migration.
Prefer messenger ingestion over mapping tasks for loading files, and use archiving and error handling to skip the flow when no new files arrive.
Learn to map the PSA calendar from source to target in a new Informatica workflow, adjust delimiters, handle date time fields, and troubleshoot date-lookup errors.
Implement and validate the final media spend upload using Informatica to load a flat file into Snowflake, handling date and decimal formats and mapping fields.
Review and fix mapping tables by removing primary keys from product and touchpoint keys, applying not null constraints, then recreate stage and psa mappings to preserve business keys.
Execute PSA touchpoint mapping by configuring stage and target with a snowflake connection, building pre-sql, and validating data population across mapping tables.
Explore data warehouse mapping with Informatica by building multi-step data flows, including lookups for product and touchpoint, filters, and returning key fields for the PSA media spend table in Snowflake.
Aggregate daily source data weekly using a calendar lookup and a grouped metrics approach, summarizing grp, impressions, and investment while reducing data size for faster mapping.
Apply the normalizer to unpivot data, creating a single value column and a metric name while mapping week start touch point code and product code in the unpivot matrix.
Execute final data warehouse mapping by applying pre sql, handling unpivoted week start as a string, normalizing dates, and enforcing delete-then-insert on the snowflake media spend target.
Design and validate the pre-SQL for the data warehouse layer by building joins between stage, product mapping, and calendar, and ensure accurate week start and product code matching.
Leverage medispan data and mapping tables to build runtime product and touchpoint masters for Snowflake dashboards. Capture brand, product key, and subbrand logic with case expressions to enable reliable aggregation.
Build and replace touch point tables in Snowflake, map media, touch point codes and keys, and design dashboards driven by source and target dimension mappings.
Design a Snowflake dashboard by creating worksheets and dashboards, adding tiles to visualize product and touchpoint mappings, and identify unmapped spend with a SQL query.
Create a touchpoint mapping tile from a sql worksheet, linking media and platform to spend touchpoints, and validate mappings to reveal missing data for charts and dashboards.
Create a Snowflake dashboard by building views that combine product and touchpoint mappings with spend data. Compare mapped and unmapped values using charts and a calendar join to surface gaps.
Discover how Streamlit lets you build data apps with Python without HTML, share quickly, and support data mappings and validation, with Snowflake integration and local or cloud runs.
Design a product master app to ingest data from the PSA table via Streamlit, update the PSA table using Informatica, and prevent manual inserts, ensuring access to the latest data.
Design a product master app using Streamlit and Snowpark, establishing an active Snowflake session to run queries, configure a wide page layout, and manage an environment-agnostic database name.
Persist data across a Streamlit app with session state, query the product master from Snowflake, and display it as a pandas DataFrame; design selectors to add brands and sub-brands.
Design and ingest a psa product table in a data warehouse using Informatica, Snowflake, and Streamlit, handling brand and subbrand levels with dynamic ui and ddl creation.
Define product master columns and a reset data frame, then implement a streamlit brand input that enforces uppercase, informs users with info messages, and checks duplicates via session SQL.
Build and test a product master workflow in Streamlit by initializing and validating a pandas data frame in session state, previewing new rows, and handling duplicates with reset logic.
Designs a product master app that ingests data with brand input, validates duplicates by product key, previews before ingest, and writes to a Snowflake table via pandas, with Streamlit feedback.
Design a dynamic product master app that adds brand and sub brand via input-driven controls, filters duplicates, ensures unique keys, and prepares data for PSA and data warehouse ingestion.
Create an object mapping to populate the product master from PSA media product, connect to Snowflake, create a CRC32 product code, and drop duplicates with a pre-step cleanup.
Design and implement a Streamlit app for product mapping, displaying products, brands, subbrands, and missing mappings in a PSA media layer with a compose key.
Recreate mapping table without product code, keep the product key, replace code with a compose key; build select boxes for mapping and product keys from the data warehouse media product.
Designs a product mapping app workflow that ingests data, uses a session state data frame, and submits mappings with brand, line extension, product key, and compose key.
Refine the touchpoint mapping app by updating fields like media lookup, platform lookup, and touchpoint key, reusing components, and validating dynamic mapping inputs in Streamlit.
Design, map, and validate touchpoint data in the touchpoint mapping app using the latest Streamlit app, syncing fields, creating touch point keys, and updating mappings for consistent dashboards.
Unlock the power of data integration and transformation with this comprehensive course designed to elevate your skills in handling data and leveraging Snowflake tables through Informatica Intelligent Cloud Services (IICS).
Throughout this course, you'll embark on a transformative journey, starting from the basics of extracting, transforming, and loading data (ETL) sourced from CSV files into Snowflake tables using IICS Data Integration. Delve deep into understanding the intricacies of data movement and manipulation, gaining proficiency in streamlining processes for seamless integration.
As the course progresses, you'll transition to exploring Snowflake's Dashboard capability, unlocking new avenues for data validation and analysis. Dive into creating visually appealing and insightful dashboards that provide a comprehensive overview of your data landscape, empowering you to make informed decisions with confidence.
But the learning doesn't stop there. We'll take it a step further by introducing you to Streamlit, a powerful tool for building interactive data applications with Python. Discover how to harness the capabilities of Streamlit to develop custom data apps that not only enhance the ETL process but also offer dynamic functionalities that can potentially replace certain steps altogether.
By the end of this course, you'll emerge equipped with a robust skill set in data integration, validation, and application development, ready to tackle real-world data challenges head-on. Whether you're a seasoned data professional or just starting your journey in the world of data engineering, this course provides invaluable insights and practical knowledge to propel your career forward.