Pentaho(PDI) Fundamentals and DWH Concepts

Learn dimensional modeling for data warehousing, including denormalized dimension and fact tables, dimension attributes and keys, and how facts combine with dimensions to drive faster query performance.

Explore star, snowflake, and galaxy schemas in data warehousing: star centers a fact table with surrounding dimensions to boost performance, snowflake normalizes dimensions for integrity, galaxy supports multiple fact tables.

Learn to design a dimensional model by identifying the business process and granularity, and build a fact table with dimensions such as month, salesperson, territory, province, and customer.

Install and configure the my school database, then establish a Pentaho data integration connection to it and test the database connection after installing the JDBC driver.

Explore Pentaho data integration components: spoon for UI-based jobs and transformations, pan for command-line transformations, kitchen for command-line jobs, carte for load-balanced server execution, and encryption for passwords.

Learn to work with pentaho data integration (pdi) input and output steps by building transformations that extract data from sample files and create tables and outputs for testing.

Learn advanced PDI input steps: read multiple files from a directory, map fields to a table, create and populate an employee_history table, and verify results with transformation metrics.

Explore transformation steps to use constants, add a sequence number, generate department-specific sequences, and apply the calculator for arithmetic, date extraction, and string operations.

Explore more transformation steps, including string concatenation with separators between name and department, and using the number range step to categorize physics marks as excellent, good, average, or bad.

Explore transformation steps in Pentaho PDI, including replacing strings, selecting and removing fields, editing metadata, changing data types, renaming fields, mapping field values, and sorting data.

Explore transformation steps by using split fields to rows to turn a semicolon-delimited category field into multiple rows, yielding separate A, B, and C category records.

Split a field into first and last names by semicolon, apply string operations like case and trim, and explore unique row, hashset, substring, and gender value mapper.

Pentaho PDI fundamentals: transformation steps part-07 introduces utilities such as null-if, clone row, delay row, and zip file and compare for differences.

Learn how flow transformation steps in Pentaho PDI use a table output to block downstream inserts until the first table is filled, then insert into the target table.

Explore inner join in sql, which combines records from two tables by matching department id and returns only matching rows, illustrated with implied and department tables.

Explore the Cartesian product, or cross join, which combines every row from the first table with every row from the second table, stressing performance on large tables.

Explore left outer join, right outer join, and full outer join, learning how each returns rows from the left, right, or both tables with nulls for unmatched rows.

Explore pentaho joins steps part-01, learning how to perform inner, left outer, right joins and cartesian products using employees and departments, with sorted merge and multi-stream merge join.

Explore merge rules in Pentaho join steps to compare source and production data, sorting by employee id and applying flags for identical, changed, new, and deleted.

Master database lookup by matching a key field to a lookup table and returning a value; handle default null when no match, and contrast it with stream lookup.

Explore the database join step in Pentaho PDI by joining employee and department tables on department number, using a parameterized where clause and exploring left, right, and full outer joins.

Apply data validation rules in Pentaho Data Integration to enforce non-null departments, valid marks ranges, and total marks limits, routing invalid data to error handling for review.

Learn to consume a soap service in Pentaho by using a web service lookup and generate row inputs to perform add and multiply operations, yielding results like 25 and 150.

Explore slowly changing dimensions in data warehouse by comparing type 1, type 2, and type 3 approaches to capture current and historical customer attributes.

Implement scd type 0 to 2 to maintain historical customer location changes using a lookup/update process, with start and end dates and an effective date for each record.

Learn to implement change data capture in Pentaho PDI by comparing source and target tables with a merge row and flags, driving insert/update or delete.

Use change data capture method 2 with maximum load_time as the end date instead of current timestamp to avoid time zone differences; update the lock, source, and destination tables.

Learn how to manage files in Pentaho PDI jobs, read and parse part files, connect transformations, generate logs, compare folders, and configure repository, file transfer, and database checks.

Learn how to override a parameter within a specific scope in a transformation step, keeping the global value intact while applying a local change such as London.