Azure Data Engineering Masterclass: ADF, Databricks, PySpark

Summary

This lecture shows how to create an Azure Data Factory (ADF) service from the Azure portal. You search for "Data Factories", click Create, fill in the project details (subscription, resource group), give a globally unique name, pick a region, keep version V2, optionally configure Git, then Review + Create. Once deployment completes, you launch ADF Studio and tour its main areas: Home, Author, Monitor and Manage.

Theory

• Every Azure service creation follows the same pattern: Subscription → Resource Group → Name → Region.
• The Data Factory name must be unique across all of Azure, not just within your account.
• Choosing a region physically deploys the ADF software there; keep all project resources in the same region to reduce network latency.
• Git integration can be configured at creation time or later ("Configure Git later").

Example

A Data Factory named "adftraining" is created under the free-trial subscription with an existing resource group (RG) in the Central India region, then opened from "Go to resource".

Key Takeaways

• Author = where you build pipelines (the core of this course).
• Monitor = view current and past pipeline run results.
• Manage = linked services, integration runtimes, Git configuration, ARM templates, triggers and global parameters.
• Validation must pass ("Validation passed") before the Create button provisions the service.

Summary

This lecture explains what a Resource Group is: a container that holds related resources for an Azure solution. All the services a project uses (Data Lake, SQL Server, Data Factory, Synapse Analytics, Key Vault, etc.) are grouped logically under one name, and the lecture demonstrates creating a resource group in the Azure portal.

Theory

• Azure hierarchy: Management Groups → Subscriptions → Resource Groups → Resources.
• An organization may hold multiple subscriptions (per project or sub-category); each subscription contains many resource groups.
• Every resource you create in Azure must belong to exactly one subscription and one resource group.
• The resource group name must be unique, and you choose a region where its metadata resides.

Example

A resource group is created via Home → Resource groups → Create, choosing the free-trial subscription and South India region. Later, while creating a Data Factory, the same resource group appears in the dropdown — you can pick an existing group or create one on the fly with "Create new".

Key Takeaways

• Resource group = logical grouping of all resources a project needs, managed as one unit.
• You can create a resource group in advance or on the fly during any resource creation.
• Validation must pass before the Create button is enabled.

Summary

This lecture introduces the top-level concepts (key components) of Azure Data Factory that you will use daily: Pipelines, Activities, Datasets, Linked Services, Data Flows and Integration Runtimes, and shows where each lives inside ADF Studio.

Theory

• Pipeline – the container where you develop your data-movement work.
• Activities – the individual steps (move, transform, general, iteration & conditional) inside a pipeline; learning them is the core of this course.
• Linked Service – like a connection string to a data store (database, blob storage, API endpoint).
• Dataset – points to a specific object (table, file, container) inside the store the linked service connects to.
• Data Flow – used for data transformation, with many transformation activities.
• Integration Runtime – the compute infrastructure (CPU, memory, VMs) that runs pipelines in the background; three types exist and are covered in a separate lecture.

Example

In ADF Studio: the Author section holds pipelines, datasets and data flows with the activity toolbox; the Manage section holds Linked Services (choose a store type such as Azure Blob Storage), Integration Runtimes (Auto-Resolve is the default), Triggers and Git configuration.

Key Takeaways

• Linked service connects to a data store; a dataset targets a specific object within it.
• Author = build pipelines; Manage = connections, runtimes, triggers, Git.
• These components appear in almost every real-time ADF task.

Summary

This lecture explains and demonstrates the two concepts every ETL operation in ADF depends on: Linked Services (connection information to external resources) and Datasets (named views pointing to the specific data you want to use as input or output).

Theory

• A linked service is much like a connection string: it defines the connection info (credentials/keys) needed to reach a source or destination data store — database, blob storage, file system, Amazon S3, Salesforce, API endpoint, etc.
• A dataset identifies data within a data store: a table, file, folder or container, used by activities as input or output.
• Every dataset requires a linked service; the linked service reaches the store, the dataset reaches the object inside it.

Example

Manage → Linked services → New → Azure Blob Storage: name it (e.g. LS_blob), keep Auto-Resolve integration runtime, authenticate with Account Key by selecting the subscription and storage account, then Test Connection ("Connection successful") and Create. Then Author → Datasets → New dataset → Azure Blob Storage → CSV: name it (e.g. DS_emp_blob), select the linked service, browse to the container/file (emp_table.csv), enable "First row as header", then preview data and schema.

Key Takeaways

• No linked service, no dataset — no ETL: they are prerequisites for any copy or transformation.
• Always use Test Connection before creating a linked service.
• Enable "First row as header" for CSV files, otherwise the header row is treated as data.
• Datasets can point at container, folder or file level; parameters for them come in a later lecture.

Summary

This lecture covers Azure Blob Storage — Microsoft's object storage solution for the cloud and the main data store you will use throughout the course — including what it is for and a full walkthrough of creating a storage account, container and file upload.

Theory

• Blob storage is optimized for massive amounts of unstructured data (text, binary, images, CSV/Excel/Parquet/JSON files).
• Hierarchy: Storage Account → Containers (like folders) → Files (blobs).
• Designed for: serving images/documents to browsers, distributed file access, streaming audio/video, writing log files, backup/restore & disaster recovery, archiving, and data for analysis.
• In Advanced settings, enabling "hierarchical namespace" turns the account into Data Lake Storage Gen2; leave it off for a pure blob account.

Example

Search "storage accounts" → Create: pick subscription, resource group and region (keep resources in the same region to cut network time). The first name attempt fails because storage account names must be unique across all of Azure, so a new name is chosen. After deployment: Containers → + Container "sourcefiles" (lowercase, no spaces) → Upload emp_table.csv → view metadata, Edit to see raw CSV, Preview to see it as a table.

Key Takeaways

• Storage account names must be globally unique, lowercase and space-free (container names too).
• Containers organize files by category; ADF pipelines read from and write to them constantly.
• Blob storage is a mandatory service to know — it is the key data store for the whole course and real projects.

Summary

This lecture explains pipeline variables in ADF — values you can initialize and then modify during a pipeline run (unlike parameters, which cannot change) — demonstrated with a small pipeline using Set Variable activities.

Theory

• Variables are created at pipeline level under the Variables tab, with a name, type and default value.
• They can be used in activities, passed into expressions, SQL scripts or dynamic content anywhere in the pipeline.
• The Set Variable activity changes a variable's value at run time.
• Type conversion matters: string variables must be cast with int() before arithmetic, and results wrapped back into a string expression with @{...}.

Example

Three variables are created: x = 5, y = 10, z = 7. A Set Variable activity changes x to 15. A second Set Variable activity computes z = x + y using dynamic content: add(int(variables('x')), int(variables('y'))) — producing 25 with the updated x value.

Key Takeaways

• Variables CAN be modified during a run; parameters cannot — this is the key difference.
• Use the Add dynamic content editor and built-in functions (add, int, string) to build expressions.
• Chain activities so the variable update completes before dependent activities read it.

Summary

This lecture explains parameters in ADF — read-only values defined at pipeline, dataset or linked-service level that are supplied at run time — and demonstrates how they make pipelines dynamic and reduce the number of pipelines you need, by copying a file whose name is passed as a parameter.

Theory

• Parameters are defined at pipeline level (also dataset and linked-service level) and cannot be modified during the pipeline run.
• They control pipeline/activity behavior, e.g. passing connection details or the path of a file to process.
• Math analogy: in f(x) = x² + 4, x is the parameter — calling f(2) substitutes the value; ADF parameters substitute values the same way at run time.
• Parameter chain: pipeline parameter → passed into dataset parameter → used in the dataset's file path via dynamic content.

Example

A destination container "destinationfiles" is created in blob storage. A dataset gets a dataset-level parameter "filename" referenced in its connection path (@dataset().filename). A pipeline gets parameter "pl_filename"; the Copy activity's source dataset maps the dataset parameter to the pipeline parameter via dynamic content. On Debug, ADF prompts for the value — emp_table.csv — and the file is copied from sourcefiles to destinationfiles.

Key Takeaways

• Parameters = run-time inputs; variables = changeable values. Parameters cannot be modified mid-run.
• Parameters exist at three levels: pipeline, dataset, linked service.
• One parameterized pipeline can replace many hard-coded pipelines — heavily used in real time.
• When a dataset has parameters, any pipeline using it will ask for those values.

Summary

This lecture explains Integration Runtime (IR) — the compute infrastructure used by Azure Data Factory and Synapse Analytics to provide data integration capabilities (data flows, data movement, activity dispatch) across different networks — and compares its three types.

Theory

• IR supplies the background computation resources (CPU, memory, bandwidth, VMs) whenever a pipeline, data flow or data movement runs.
• Azure IR (Auto-Resolve) – the default, created automatically with the Data Factory; fully managed by Microsoft (patching, scaling, maintenance); supports data flow, data movement and activity dispatch in public and private networks.
• Self-Hosted IR – runs on infrastructure and hardware managed by YOU (your servers); used between on-premises and cloud or private networks; supports data movement and activity dispatch but NOT data flows.
• Azure-SSIS IR – VMs running the SSIS engine to natively lift-and-shift and execute SSIS packages; managed by Microsoft; works in public and private networks; used only for SSIS package execution.

Example

In ADF Studio → Manage → Integration runtimes, the AutoResolveIntegrationRuntime already exists (type Azure, status Running). Clicking + New offers the three choices: Azure/Auto-Resolve, Self-Hosted, and Azure-SSIS.

Key Takeaways

• IR in one phrase: the compute infrastructure of ADF.
• Choose by scenario: cloud-to-cloud → Azure IR; on-premises ↔ cloud → Self-Hosted IR; SSIS packages → Azure-SSIS IR.
• Self-hosted IR cannot run data flows — only data movement and activity dispatch.
• Auto-Resolve IR needs no creation or maintenance — Microsoft handles patching and scaling.

Summary

This lecture covers Copy Activity — the activity a data engineer uses most frequently in ADF — explaining what it is, which data stores it supports, and which file formats it handles, with two hands-on demos: blob-to-blob file copy and SQL Server-to-blob export.

Theory

• Copy Activity copies data between data stores located on-premises and in the cloud — on-prem→cloud, cloud→on-prem, anywhere to anywhere.
• It sits under Move & Transform; the two essential tabs are Source (where the data is) and Sink (where it goes), each requiring a dataset.
• Source options for files: direct file path, prefix, wildcard file path, or a list of files.
• Database sources offer three modes: Table, Query (write your own SELECT), or Stored procedure.
• Supported data stores and formats are listed in Microsoft's documentation, showing per store whether it works as source and/or sink and with which integration runtime.

Example

Demo 1: emp_table.csv is copied from the "sourcefiles" container to the "destinationfiles" container using two CSV datasets. Demo 2: the dbo.tester table from Azure SQL Database is exported to blob as data_from_server.csv; the run Details show rows read/written, data size and throughput. A query mode example uses SELECT * FROM adf_config to fetch different columns.

Key Takeaways

• Copy Activity = data movement between almost any two data stores.
• Monitor progress and statistics (files, rows, size, duration) in the activity run Details.
• Configuration panels change automatically depending on the dataset's data store type.
• Check Microsoft docs for supported stores/formats when facing a new source or sink.

Summary

This lecture explains Append Variable Activity — adding a value to the end of an existing array variable — plus how to access individual array elements with indexing.

Theory

• Append Variable only works with array-type variables; the Name dropdown shows array variables only.
• Three steps: declare the array variable → initialize it with default values in square brackets → append with the activity.
• Values typed into the append box default to string; wrap with int() to append a number.
• Arrays can mix data types (integers, strings, booleans together).
• Indexing starts at 0: variables('marks')[0] returns the first element.

Example

A "marks" array is created with default [89,50,78,40]; Append Variable adds 90 (first as string "90", then as int(90)). A Set Variable copies marks into an "output" array to inspect the result. Another Set Variable reads variables('marks')[0] into a string variable — wrapping the expression in @{...} (or string()) to fix the "type string" error, returning 89; index [3] returns 40.

Key Takeaways

• Append = add to end of array; assignment between variables uses Set Variable, not Append.
• Mind data types: int() to append numbers, string()/@{...} when assigning to string variables.
• Access array items with [index], starting at 0.

Summary

This lecture covers Delete Activity — deleting files or folders in on-premises or cloud data stores — with demos of deleting a single file, wildcard matches, and prefix matches, plus logging every deletion to a container.

Theory

• Delete Activity (General category) removes files or folders; typical real-time use is cleanup/archival — e.g. deleting processed files or archives older than 3 months.
• Four targeting options: file path in dataset, wildcard file path, prefix, or a list of files (paths in a text file).
• Filters: date range (start/end time) and max concurrent connections for parallel deletes.
• "Recursively" deletes contents of subfolders inside the target folder as well.
• Optional logging settings write a log file (which files, category, status) to a linked-service container you choose.

Example

A "log" container is created for logging. Demo 1: not_sample.csv is deleted via direct file path. Demo 2: the dataset points at the container and wildcard *data* deletes every file containing "data". Demo 3: prefix "dept" deletes the three files starting with dept. Each run's log file lists the deleted files.

Key Takeaways

• Delete files or folders directly from a pipeline; essential for blob-storage housekeeping.
• Wildcard = match anywhere in name; prefix = match at start; list-of-files for arbitrary sets.
• Enable logging so you can trace exactly what was deleted and when.

Summary

This lecture explains Execute Pipeline Activity — how one pipeline (master/parent) invokes another (invoked/child) — demonstrated by a master pipeline calling a child pipeline that fetches data with a Lookup.

Theory

• Execute Pipeline lets a Data Factory or Synapse pipeline invoke another pipeline.
• Common real-time pattern: develop each piece of work in its own pipeline, then integrate them sequentially or in parallel from a master pipeline.
• Configuration is minimal: in Settings, pick the invoked pipeline from the dropdown of all pipelines in the factory.
• The master's output shows a pipeline run ID linking to the child run — useful for navigation and for logging.

Example

An "invoked pipeline" is built with a Lookup activity reading emp_table.csv from the source container (a new dataset per requirement is recommended). A "master pipeline" with Execute Pipeline calls it. After unchecking "First row only" on the Lookup, rerunning the master shows all 17 records in the child's output, reached via the run ID link in the master's output.

Key Takeaways

• Use Execute Pipeline to chain or orchestrate pipelines (master → child).
• Inspect child results by following the pipeline run ID from the master's output.
• The run ID can also be captured for logging purposes.

Summary

This lecture covers Fail Activity — intentionally throwing an error in a pipeline with your own error message and error code.

Theory

• ADF throws errors automatically for configuration/syntax problems, but sometimes you need to fail a pipeline on purpose (e.g. a Lookup returned no data).
• Fail Activity (General category) has just two settings: Fail message (static text or dynamic content combining previous activities' output) and Error code (a unique ID, in the spirit of 200-series = success, 400/500-series = failure).
• When it runs, the pipeline fails exactly like a real failure — the message appears under error details with your custom code.

Example

A Fail activity is configured with message "Pipeline failed intentionally" and code 555; on Debug the pipeline fails and the output shows the custom message and error number.

Key Takeaways

• Use Fail Activity to signal business-rule failures (empty lookups, missing files) so the team is notified.
• Message and code can be built dynamically from previous activities' output.
• Assign meaningful, unique error codes so failures are identifiable.

Summary

This lecture explains Get Metadata Activity — retrieving metadata ("data about data": names, types, sizes, child items) of files, folders/containers or tables, to validate conditions or feed subsequent activities.

Theory

• Metadata describes the data: for a file — name, type, size, created/modified dates; for a table — name, column count, column types (not the rows themselves).
• The activity needs a dataset pointing at the target (file, container/folder or table) plus a Field list specifying what to retrieve.
• The available field-list options change with the dataset type: a container dataset offers child items, item name, exists; a file dataset offers many more (size, column count, etc.).
• Outputs feed conditional expressions for validation or are consumed by later activities.

Example

Pointed at the destination container, "Child items" returns every file and the "data" folder with each item's name and type — the standard trick for reading file names dynamically instead of hard-coding them. Pointed at emp_table.csv, "Item name" returns the file name, "Column count" returns 8, and "Size" returns 723 bytes.

Key Takeaways

• Get Metadata = data about the data, not the data itself.
• Child items on a folder dataset is the key pattern for dynamic file processing (usually feeding a ForEach).
• Field-list options depend on what the dataset points to — explore them per dataset type.

Summary

This lecture covers Lookup Activity — reading and returning the content of a file or table (or the result of a query/stored procedure) so later activities can use it — including its two hard limits: 5000 rows and 4 MB.

Theory

• Lookup fetches data from a file or database table via a dataset; database datasets also support query and stored-procedure modes.
• "First row only" is checked by default — uncheck it to return all rows.
• The output is JSON: value is an array of row objects, so navigate with activity('Lookup1').output.value[index].fieldName.
• Limits: maximum 5000 rows returned and maximum 4 MB total size — exceeding 5000 truncates; exceeding 4 MB fails the activity.

Example

Demo 1: emp_table.csv (17 records) is read from blob — first one row, then all 17 after unchecking "First row only". Demo 2: dbo.tester (14 records) is read from Azure SQL Database. A Set Variable then extracts a single value with lookup output .value[0].ename → "Smith"; changing the index to [1] returns the next row's name.

Key Takeaways

• Lookup = bring data INTO the pipeline; use its output in subsequent activities via JSON navigation.
• Remember .value[index].field — value is an array indexed from 0.
• Watch the 5000-row and 4 MB limits when fetching from large sources.

Summary

This lecture explains Set Variable Activity — assigning values to declared pipeline variables (string, boolean or array) — illustrated with an area calculation (length × breadth) that also teaches type casting in dynamic content.

Theory

• Like other languages, two steps: declare the variable (pipeline → Variables tab; types: String, Boolean, Array) then initialize/assign with Set Variable.
• Values can be static or dynamic expressions built in the expression editor.
• Variables are string-typed by default, so arithmetic needs conversions: int() before math, mul() to multiply, and wrap the whole expression in @{...} (or string()) to assign back to a string variable.
• Order dependencies with activity links — parallel assignments can converge on a final calculation.

Example

Variables length, breadth, area are declared. Set Length assigns 10 and Set Breadth assigns 5 in parallel; after both succeed, Area is set with @{mul(int(variables('length')), int(variables('breadth')))} → 50.

Key Takeaways

• Declare first, then Set Variable to assign — the dropdown only shows declared variables.
• Type casting is unavoidable: int() in, string()/@{...} out.
• Chain activities on success so calculations run only after their inputs are set.

Summary

This lecture covers Wait Activity — pausing a pipeline for a specified number of seconds before continuing, typically while an external dependency completes.

Theory

• Wait (General category) has one setting: the wait time in seconds — 1 minute = 60, 1 hour = 3600.
• Total duration is the wait time plus a second or two of queuing/execution overhead.
• Use it between tasks when something outside the pipeline must finish before the next activity can run.

Example

A Wait of 5 seconds shows a run duration of ~6 seconds. Real-time scenario: after pushing products to a search engine's indexing API, the pipeline waits ~1 hour (3600 seconds) for indexing to complete before running the follow-up activities.

Key Takeaways

• Always convert the desired pause to seconds.
• Simple but powerful for external-dependency timing between tasks.
• Expect a small execution overhead on top of the configured wait.

Summary

This lecture explains JSON (JavaScript Object Notation), the lightweight format for storing and transporting data that data engineers meet everywhere — especially with API endpoints and inside ADF pipeline definitions — and shows how to navigate any JSON structure to reach a value.

Theory

• Data is always in field:value pairs, separated by commas.
• Curly braces { } hold objects; an object's value can itself be another object (nesting).
• Square brackets [ ] represent arrays; array elements are objects indexed from 0.
• Navigation is hierarchical with the dot operator: outer field → inner field (like country → state → district → house number).
• Array rule: every object inside an array must have the same number of fields with the same field names in the same order — only values differ.

Example

A student JSON has name, examtitle, rollnumber (an object) and marks (an array of three objects). To get a nested value: rollnumber.s2. To get the science marks stored in the third array element: marks[2].science → 95 — index first, then field name.

Key Takeaways

• { } = object, [ ] = array, array indexes start at 0.
• Reach any value hierarchically: field.subfield or array[index].field.
• Mastering JSON navigation is essential for dynamic content, REST/API work and reading ADF's own pipeline JSON.

Summary

This lecture explains Switch Activity — the ADF equivalent of a programming switch statement — demonstrated by routing a file into one of three containers based on a pipeline parameter.

Theory

• Switch evaluates an expression and executes the activities of the matching case; cases are checked top-down and remaining cases are skipped once one matches.
• A default block runs when no case matches.
• The expression can come from a parameter or a previous activity's output.
• Use If Condition for two branches; Switch is better when there are more than two cases.

Example

Three containers (output1/output2/output3) are created. A pipeline parameter "containerName" feeds the Switch expression. Each case (output1, output2, output3) holds a Copy activity: source = emp_table.csv dataset; sink = a parameterized dataset whose container name comes from the pipeline parameter via dynamic content. Running with "output1" copies the file into output1; rerunning with "output3" copies into output3.

Key Takeaways

• Switch = multi-way branching on a value; first match wins, default catches the rest.
• Combine with parameterized datasets so one dataset serves every case.
• Pipeline parameter → Switch expression → case block → parameterized sink is a common routing pattern.

Summary

This lecture covers Filter Activity — applying a filter expression to an input array so only the values satisfying the condition come out — demonstrated by filtering strings that contain "data".

Theory

• Input must be an array (parameter or variable); output is also an array of the values where the condition evaluated true.
• Two settings: Items (the input array via dynamic content) and Condition (an expression applied to each element).
• item() refers to the current element; contains(item(),'text') checks for a substring (Collection functions).
• Classic real-time use: Get Metadata fetches all file names, Filter keeps only those matching a pattern, then ForEach processes them.

Example

An array parameter ["edufulness","data engineer","data factory","filter activity"] is filtered with contains(item(),'data'); output contains 2 of 4 items: "data engineer" and "data factory". The same works when the array is a variable instead of a parameter.

Key Takeaways

• Array in → condition per item → array out.
• item() + collection functions like contains() build the condition.
• Typical chain: Get Metadata → Filter → ForEach for pattern-based file processing.

Summary

This lecture explains ForEach Activity — ADF's loop for iterating over a collection and executing the same activities for each value — demonstrated by copying three files from a source container to a destination container, plus the difference between parallel and sequential execution.

Theory

• ForEach (Iteration & Conditionals) repeats a set of inner activities once per element, like for/while loops in any language.
• The input MUST be an array: an array variable or an array output from another activity (e.g. Get Metadata child items).
• Two parts: Settings → Items (the input array) and Activities (what to do per value).
• Inside the loop, item() is the current element; item().fieldName reaches a field of the current JSON object.
• By default iterations run in PARALLEL; check Sequential to run one after another.

Example

A JSON array [{"filename":"dept.csv"},{"filename":"emp_table.csv"},{"filename":"students.csv"}] is stored in an array variable and passed as Items. Inside, a Copy activity uses a parameterized source dataset (parameter "filename" = @item().filename) pointing at the source container, and a sink dataset pointing at the destination container. All three files copy — in parallel (same start times) by default; with Sequential checked, each starts after the previous finishes.

Key Takeaways

• ForEach = same action, different value, once per array element.
• item().field navigates the current object; feed it into dataset parameters.
• Parallel by default — tick Sequential when order matters.

Summary

This lecture covers Until Activity — ADF's do-until loop that repeats a set of activities as long as its condition is FALSE and stops when it becomes TRUE — demonstrated by counting from 1 to 10, including the trick for incrementing a variable (no self-assignment in ADF).

Theory

• Until executes inner activities repeatedly until the condition evaluates to true (condition false = keep looping; true = stop).
• Unlike ForEach, Until runs sequentially by default and has a condition instead of an input array.
• ADF variables can't reference themselves (x = x + 1 fails), so use a temp variable: temp = x, then x = temp + 1.
• Type casting is required: int() for comparison/addition, wrap in @{...} or string() when assigning back.

Example

Variables x = 1 (string) and temp are declared. Until's condition: @greaterOrEquals(int(variables('x')), 10). Inside: Set Variable temp = x, then Set Variable x = @{add(int(variables('temp')), 1)}. The loop iterates producing 2,3,4...10; when x reaches the target the condition turns true and the loop stops.

Key Takeaways

• Until loops while condition is FALSE and stops on TRUE — the opposite mindset to a while loop's guard.
• Self-assignment isn't allowed: always increment via a temp variable.
• Use greaterOrEquals/int/add from the logical, conversion and math function groups.

Summary

This lecture explains If Condition Activity — ADF's if/else — demonstrated by setting an interest rate of 12% or 18% depending on whether an amount parameter is at most 10,000.

Theory

• If Condition (Iteration & Conditionals) evaluates an expression to a boolean: True activities run on true, False activities on false — exactly one branch executes.
• The condition uses logical functions such as lessOrEquals(a,b); its syntax returns true/false.
• String parameters must be cast with int() before numeric comparison or ADF raises "does not have overload that supports argument: string, integer".
• Edit the True/False branches via the pencil icons inside the activity.

Example

Parameter "amount" (string) and variable "rateOfInterest" are created. Condition: @lessOrEquals(int(pipeline().parameters.amount), 10000). True branch: Set Variable rateOfInterest = 12; False branch: = 18. Debug with 9000 → true path, rate 12; rerun with 15000 → false path, rate 18.

Key Takeaways

• If Condition = two-way branch; use Switch when you have more than two cases.
• The expression must evaluate to boolean; cast strings with int() before comparing numbers.
• Test both branches by rerunning with different parameter values.