Ceph Storage Mastery: Complete Guide with 25 Hands-On Labs

1. What is Ceph? Complete Overview

   • Software-Defined Storage explained

   • Ceph vs traditional storage

   • Real-world use cases

2. Ceph Architecture Deep Dive

   • RADOS foundation

Ceph Component overview

Data flow understanding

The Three Access Methods

• RBD (Block Storage)

• RGW (Object Storage)

• CephFS (File System)

1. Hardware Requirements & Sizing

   • CPU, RAM, Network considerations

   • Disk selection strategies

   • Performance calculations

2. Network Architecture for Ceph

   • Public vs Cluster networks

   • Bandwidth requirements

   • Network topology best practices

3. Deployment with Cephadm

   • Why Cephadm for modern deployments

   • Container orchestration benefits

   • Production recommendations

4. Capacity Planning & Growth

   • CRUSH map planning

   • Failure domain design

   • Scalability considerations

1. Introduction to Cephadm (8 min)

   • Architecture overview

   • Container orchestration

   • Modern deployment approach

2. Pre-installation Checklist (10 min)

   • System preparation

   • SSH configuration

   • Time synchronization

3. Understanding Service Deployment (8 min)

   • Service placement strategies

   • Label-based deployment

   • Service specifications

   • High availability patterns

   • Client node configuration

4. Client Node Architecture (8 min)

   • Purpose of client1 node

   • Security isolation principles

   • Why admin keyring never leaves cluster

   • Per-service authentication strategy

   • Storage consumption patterns

   • Best practices for client access

Lab 1: Bootstrap Ceph Cluster

• System preparation

• Cephadm installation

• Initial bootstrap

• Dashboard access

Lab 2: Deploy OSDs on HDD

• Add all cluster nodes

• Monitor placement (3 monitors on nodes 1, 2, 3)

• Deploy HDD OSDs (vdb, vdc, vdd)

• Health verification

Lab 3: Deploy RADOS Gateway (HTTP/HTTPS)

• RGW service deployment

• Configure RGW on node2 and node4

• Verify S3 endpoint

• Basic bucket operations

Lab 4: Deploy CephFS & MDS

• Deploy MDS services with YAML spec

• Configure MDS active on node1

• Configure MDS standby on node3

• Create CephFS filesystem with data and metadata pools

• Verify MDS active/standby status

• Verify CephFS pools created

• Create initial subvolumes (user1, user2, webapp, database)

Lab 5: Deploy OSDs on SSD

• Add SSD OSDs (nvme0n1)

• Configure device classes

• Verify mixed storage types

• Performance comparison

Lectures:

1. Understanding Pools & PGs

   • Pool architecture

   • Placement groups explained

   • Optimization strategies

2. Replication Strategies

   • Replica counts

   • Min_size implications

   • Performance vs safety

3. Erasure Coding Deep Dive

   • EC profiles

   • Use case selection

   • Performance implications

4. Pool Types for Different Workloads

   • Fast pools (SSD-based)

   • Archive pools (HDD-based)

   • Mixed workload strategies

Lab 6: Create Storage Pools

• Create replicated pools for RBD

• Create EC pools for archival

• Configure pool quotas

• Set application tags

Lab 7: Advanced Pool Configuration

• Configure fast-pool on SSDs

• Configure archive-pool on HDDs

• Set up device class rules

• Pool migration strategies

Lectures:

1. CRUSH Algorithm Explained

   • How CRUSH works

   • Deterministic placement

   • Fault tolerance design

2. Device Classes & Rules

   • SSD vs HDD separation

   • Custom device classes

   • Rule creation

3. CRUSH Map Editing

   • Decompiling and editing

   • Testing changes

   • Safe deployment

Lab 8: CRUSH Map

• Extract and decompile CRUSH map

• Create custom rules

• Test with crushtool

• Apply changes safely

Lectures:

1. CephX Authentication System (12 min)

   • How CephX works

   • Key management

   • Security best practices

   • Principle of least privilege

2. User & Permission Management (10 min)

   • Creating specific users per use case

   • Capability assignment

   • Why admin keyring stays on cluster

   • Client isolation strategies

LAB 11: Cephadm Management and Container Orchestration

Objectives

• · Master Cephadm shell environment

• · Understand containerized service management

• · Configure non-root administrator access

• · Explore orchestration commands

• · Monitor services through containers

Lab 12: RADOS Direct Manipulation from client1

• Use client.radosuser keyring (created in Lab 10)

• Configure namespace isolation (userdata)

• Object creation and retrieval with radosuser

• RADOS benchmarking from client

• Verify access restricted to specific namespace

Learning Objectives

By the end of this lab, you will be able to:

· Configure and manage RadosGW users with S3 and Swift access

· Set up and use multiple S3 clients (s3cmd, aws-cli, boto3)

· Implement bucket policies, versioning, and lifecycle management

· Configure Swift API access and containers

· Perform multipart uploads for large objects

· Generate and use presigned URLs for temporary access

· Monitor and optimize RadosGW performance

· Integrate object storage with applications using Python

Lab 14: RBD Basic Operations from client1 (45 min)

• Use client.rbduser keyring (created in Lab 10)

• Image creation in fast-pool only

• Mapping and mounting on client1

• Resize operations with rbduser

• Verify no access to other pools

• Basic performance tests

Objectives

• Create and manage RBD snapshots

• Implement cloning from snapshots

• Test rollback operations

• Understand copy-on-write efficiency

Objectives

• Configure RBD cache settings

• Optimize stripe parameters

• Implement QoS limits

• Compare different configurations

Objectives

• Export RBD images for backup

• Import images from backup

• Implement incremental backups

• Migrate images between pools

Objectives

• Mount CephFS using kernel and FUSE clients

• Test file operations and permissions

• Implement subvolume isolation

• Compare performance between mount types