Argo Workflow: Building and Managing Kubernetes Workflow

Explore Argo Workflow, an open source container native workflow engine for orchestrating Kubernetes jobs, using CRD, YAML definitions, and a user-friendly UI for monitoring, logging, and dependencies.

Explore the Argo workflow architecture, detailing how the workflow controller reconciles tasks and queues while the Argo server coordinates workers, the UI, the CLI, and APIs across Kubernetes.

Expose the Argo server with a node port to enable web UI access, then create a training service account, generate a token, and log in.

Learn to define Argo workflows in yaml, detailing api version, kind, metadata, and a spec with templates and the entry point. Track progress and issues as the workflow runs.

Learn to create and run an Argo workflow with YAML in the UI and CLI, including API version, kind, metadata, spec, service account, namespace, entrypoint, and parameters.

Explore Argo workflow template types, including container templates and the lab, focusing on entrypoints and the blueprint-like template section, plus deploying and monitoring with Argo CLI.

Discover how the script type template in Argo workflow runs a script inside a container using the source field. Compare it to container type; explore lab five with Python scripts.

Pause your Argo workflow with the suspend template, optionally setting a duration or awaiting manual resume, and monitor progress via the user interface, Argo submit/watch, and kubectl describe.

Explore step templates in Argo workflow, orchestrating tasks in order and parallelism; define step one, then run two A and two B concurrently using BusyBox containers and YAML.

Explore the dag template in Argo, a directed acyclic graph that defines execution order and dependencies, showing A before B and C and B and C before D, with parallelism.

Configure an Argo workflow artifact repository using an AWS S3 bucket. Create and attach an AWS user with a policy, then update the workflow controller config map to store artifacts.

Test an Argo workflow that generates an artifact, stores it in an S3 bucket via an artifact repository, and then consumes and prints the artifact content.

Learn how to use output parameters in Argo workflows to generate data in one step and pass it to subsequent steps, not storing artifacts in S3, enabling flexible, conditional pipelines.

Create Kubernetes secrets with cube CTL and reuse them in an Argo workflow as environment variables or by mounting them as volumes.

Demonstrates using a pre-existing pvc to claim a pv and mount a persistent volume in an Argo workflow, covering static and dynamic provisioning and storage class considerations.

Explore loops in Argo workflows, including with sequence, with items, and JSON object-based iterations, enabling dynamic runs and parameterized templates.

Explore error handling and work management in Argo workflows, detailing retry policy with exponential backoff and limit, plus exit handlers and active deadline seconds for timeouts.

Explore how to create and reuse workflow templates and cluster workflow templates, understand the difference between template and workflow template, and apply cluster-wide templates across namespaces.

Learn how Argo workflow supports cron workflow scheduling by defining a cron schedule that runs a set of tasks automatically, with options like concurrent policy and timezone.