Mastering Presto: Hands-On Learning

Discover the Presto cli, the cluster's entry point, a self-executing jar you run in the terminal to connect to a server, catalog, and schema, and run queries.

The coordinator is the central server in a Presto cluster; it parses, analyses, and plans queries, creates a query execution plan, and distributes it to workers via REST API.

The coordinator analyzes a query, creates an execution plan, and distributes it to multiple workers that execute tasks and process catalog data.

Discover how Presto connectors adapt to sources like Hive and databases, configure catalogs per data source, and query across diverse storages with a single SQL.

Leverage the hive metastore as the data warehouse meta store for schema definitions and sql queries, a required Presto component, and ensure high availability.

Explore Presto's query execution model, from statement to query object, stages, tasks, splits, drivers, operators, and exchange, to parallel in-memory data processing across workers.

Present Presto's built-in data types and the subset supported by connectors, and outline integer subtypes tinyint, smallint, integer, bigint with min and max values; cover reserved keywords and double-quote usage.

Master Cassandra's distributed, NoSQL architecture with partition, clustering, composite, and primary keys, then leverage the Presto connector for efficient, parallel queries and cross-system joins.

Explore the PostgreSQL connector in Presto, enabling parallel JDBC querying with filters and cross-system joins to PostgreSQL data, plus managing schemas, tables, views, and columns.

Amazon Redshift is a managed data warehouse; the Presto Redshift connector uses columnar storage and filters to access only needed data, with a single database per cluster and limited SQL.

Learn Kafka basics and how Presto's Kafka connector maps topics to tables, with partitions and replication, plus retention policies and data inconsistency caveats when querying.

configure a Presto cluster by adding etc/node.properties with node.environment, a unique node.id, and node.data-dir, ensuring consistent environment names across all nodes, such as development, staging, and production.

Configure the jvm.config file inside the etc folder and name it jvm.config to define the java vm arguments used at Presto startup, including the Xmx heap size of 32 gigabytes.

Create log.properties in the etc folder to define log levels for Presto loggers; set com.facebook.presto to error to cover its server and hive, with default info when unset.

Explore configuring a Presto cluster with the config.properties file, detailing coordinator, worker, and both roles; set http port, memory limits, and service discovery for scalable, production-ready nodes.

Configure catalog files inside the etc/catalog folder to define a PostgreSQL connector, using a resource name as the catalog identifier for queries like select * from my_postgresql.schema.table.

Learn to install and run docker with a focus on docker for desktop on Mac, and use the setup to build docker images for presto and hive in upcoming lectures.

Build a single docker image for both presto coordinator and workers using a Dockerfile and an entrypoint, with an OpenJDK base and dynamic config for Hive catalog.

Build a Hive meta store docker image from a java base, configure hive-site.xml and entrypoint.sh, install Postgres and AWS jars, and start Hive service for Presto integration.

Explore how Kubernetes automates deployment, scaling, and management of containerised apps using master and worker nodes, API server, etcd, pods, deployments, services, and namespaces.

Learn how Helm manages Kubernetes applications by packaging resources into charts and automating install, upgrade, and delete through the command line interface, with charts, values, templates, and environment overrides.

Install VirtualBox, kubectl, and Minikube to run a local Kubernetes single-node cluster on Mac, then start and stop with cpu, memory, version 1.16.2, and driver VirtualBox settings.

Create a helm chart to run a local Postgres-backed Presto cluster on Kubernetes, configuring ConfigMap, Deployment, and Service, with values.yaml and prod_values.yaml for environment-specific overrides.

Deploy a Presto coordinator in Kubernetes with a Helm chart, using a ConfigMap, a Deployment, and a NodePort service. Configure ports and memory from values.yaml and environment variables from ConfigMaps.

Define a Kubernetes configmap and deployment for Presto workers, set worker-specific environment variables (port, memory settings, worker JVM memory) and the coordinator URL, and configure three replicas in values.yaml.

Learn to complete Presto's Helm chart by configuring the Hive metastore with a ConfigMap, Deployment, and a cluster-internal Service, using env vars from ConfigMaps for Hive and PostgreSQL credentials.

Deploy Presto to a Kubernetes cluster by using Minikube's Docker registry, building images inside it, and deploying the Helm chart; then port-forward the Presto coordinator to access it from localhost.

Install awscli on your mac using brew. Then interact with AWS resources from the terminal to prepare for running Presto queries over parquet files in S3.

Learn to create an IAM user with programmatic access in AWS, generate an access key id and secret access key, and configure the CLI with region eu-west-1 and JSON output.

Attach AWS permissions by choosing attach existing policies directly for a user, review administrator access, and follow best practice to use groups and correct policies.

Explore why parquet, a columnar storage format, enables high read throughput with row groups, column chunks, data pages, data dictionaries, and per-column compression such as snappy, gzip, and lzo.

Create a globally unique S3 bucket, then upload your parquet files with the AWS CLI copy command using recursive to build your data lake.

Launch a local Presto cluster on Kubernetes, define an external Hive table pointing to S3 Parquet data, then query with Presto to count rows and inspect schema.

Add a PostgreSQL catalog to your Presto cluster to join data from multiple sources by configuring a my_postgresql catalog, supplying JDBC credentials, and validating with a Presto query.

Learn to join data from PostgreSQL and parquet files on S3 in one Presto query by referencing catalog, schema, and table names across sources.

Explore Presto's web interface on localhost:7000 to monitor cluster performance in realtime, viewing running and blocked queries, active workers, and rows processed per second, with Hive and PostgreSQL join.

Create an elastic container registry repository for Presto and Hive, tag your local images, and push them to the remote ECR after logging in.

Automate the creation of an AWS EKS cluster named presto with eksctl, provisioning a two-node eu-west-1 setup with VPC, subnets, and security groups.

Deploy the presto helm chart to the Amazon EKS cluster using production values, with 1 coordinator and 3 workers, 32gb jvm memory, and 16gb per node, using ECR docker images.

Explore vanilla Presto, a self-managed Presto cluster you control across clouds or on-premises, balancing lower software costs with higher engineering effort and upgrade risks.

Presto on EMR shows Amazon EMR as a managed service that provisions clusters quickly and supports ad hoc queries, CSV exports, and saved queries, with tradeoffs in cost and performance.

Evaluate AWS Athena, a serverless, Presto-based analytics service on S3 with the Glue catalog, ANSI SQL, and pay-per-query pricing; weigh cost, performance, and security trade-offs.

Explore Starburst Presto, a multi-cloud, managed Presto solution with a Kubernetes operator and extensive connectors. It delivers performance-optimised clusters and strong price-performance advantages.