
Join the Telegram group to discuss Linux inter process communication (IPC) from scratch in C.
In this lecture, I referred malloc/free as system calls, though they are not system calls. Whether or not they are system calls is out of context for this course, pls control your adrenaline - I admit they are not system calls.
This Lecture is for those who are absolute beginners with Linux and have no prior knowledge with Linux Installation. This course do not demands user to use Linux OS. All Assignments can be done on Windows platform also, however, it is recommended to use Linux for Development work. Please switch if you are still stuck with windows.
Pls go through this tutorial which will tour you through the Linux installation steps and supporting softwares. Those who are already using Linux can safely skip this Section.
Understand how sockets enable inter-process communication on Linux, for both same-machine and networked processes. Learn socket APIs and a client-server workflow using typical system calls.
Outline the socket design state machine for Linux IPC, from creating the master socket to accepting new clients and processing service requests with per-client data sockets.
Learn how the accept system call converts a server's master socket into a client communication file descriptor, enabling bidirectional, connection-oriented data exchange with new clients.
Explore implementing a unix domain server in C via the server.c state machine, covering socket creation, 128-byte buffer, and summing numbers until zero.
Explain the unix domain client in C: create a data socket, connect using the server's socket name, send integers, and read the server's reply.
Learn how to implement multiplexing in c using select and accept, maintaining a master socket and per-client fds in an fd set to handle multiple connections.
Demonstrates a multiplexed Unix domain server in C that accepts multiple clients via select, computes per-client sums of integers, and handles new connections concurrently.
Learn to build a unix domain socket IPC system where a routing table manager propagates create, update, and delete operations across connected clients and dumps the table to new connections.
Explore message queue as a Linux IPC mechanism, learn its concepts and APIs, understand enqueue and dequeue workflows, and walk through a kernel-resident queue based sender and receiver implementation.
Create and use a kernel-space message queue from user-space processes to enable IPC, where queues are identified by a string ID that bridges sender and receiver.
Use mq_receive to fetch the oldest message with the highest priority from a message queue, filling the provided buffer and returning the bytes received; blocking behavior is controlled at open.
Learn how the message queue unlink API destroys a queue by releasing the kernel resource after all users close it, returning zero on success and minus one on failure.
Download the source and run the sender and receiver C files to demonstrate message queue IPC, where the sender sends user input to the receiver and the receiver prints it.
Explore how shared memory enables fast inter process communication in Linux by using memory mappings and system calls, with hands-on guidance on implementing sender and receiver processes.
I have a separate Course in which the entire paging is explained in detail.
Course name : Linux System Program Concepts and Techniques.
Explore memory mapping in Linux via the map system call, creating virtual pages that map to physical memory and update the page table, with text files mapped as virtual memory.
Learn how memory mapping via the map system call creates shared memory by mapping external memory into multiple processes' virtual address spaces, sharing physical pages and synchronizing changes.
Demonstrate Linux ipc with shared memory via programs using mmap as a malloc substitute, importing text into address space, and using shm_open, ftruncate, mmap, read, write, and shm_unlink across processes.
Explore design constraints for shared memory as inter-process communication, with one publisher, many subscribers, and infrequent updates. Publishers notify subscribers after updates via inter-process communication mechanisms; subscribers read fresh memory.
Explore how Linux signals enable inter-process communication by signaling between processes, capturing signals, and applying default actions, signal handlers, and registration that pre-empts execution.
Explore Linux well known signals, including SIGINT, SIGUSR1 and SIGUSR2, SIGKILL, SIGABRT, SIGTERM, SIGSEGV, and SIGCHLD, and learn their default behaviors, catchability, and how child termination via fork triggers SIGCHLD.
An Example demonstrating a use of signal.
Synchronize new clients with the rtm server by exchanging pids over unix domain sockets, updating in-memory routing and arp tables via shared memory, and signaling clients to flush on command.
Explore socket programming with a tcp focus, learn client-server architecture in C, and master multiplexing with the select and accept system calls to handle multiple clients.
Analyze the accept system call on the TCP server to establish a bi-directional connection, handle client requests, and return a communication file descriptor for data exchange.
Follow the ten steps of the state machine to implement a tcp server; this lecture walks through setup, socket creation, bind, and listen with a tcp server dot c example.
This lecture explains building a tcp server that uses select to monitor the master socket, accept new clients, and exchange data in an infinite loop.
Read a and b from the client, compute their sum, and reply with the result; close the connection when both values are zero, and loop to await the next client.
Observe a blocking TCP server design where the server handles only one client at a time, blocked during data receive, illustrating limitations and the single-client use case.
Explore tcp server design with multiplexing to handle multiple clients using fd_set and select. Learn how to compile and run the tcp client and server on localhost and exchange values.
Explore the high level design of a multiplexed tcp server that can handle multiple clients using a master socket, per-client file descriptors, an fd array, and the select system call.
Explore a TCP server with multiplexing implemented via select, managing the master socket and client file descriptors, accepting connections, and handling data exchange across multiple clients.
Review the socket APIs, including socket, select, accept, bind, listen, receive, send, and close, and analyze a TCP server with multiplexing and 31 connections, contrasting with UDP's simplicity.
Learn to build a distributed transparent memory system using sockets in C, presenting an eight-gigabyte, multi-node memory as a single, user-visible memory space.
Deploy and simulate a six-node ring topology on a single machine by running the same program in six terminals, each binding to 127.0.0.1 with unique UDP and TCP ports.
Explain the put request algorithm in a ring topology, hashing K with K mod N to locate the node, forward via UDP, and retrieve X from the originator over TCP.
Describe how a node in a ring topology processes a put forward message by hashing the key and comparing the node ID; if originator, store key in hash table.
Follow the pseudo code to implement a tcp/udp ipc project in c using select to monitor tcp, udp, and console, handling put/get messages in a ring topology.
Learn how a single C program models a ring topology with hash function k mod n and roles, supporting put and get requests using tcp/udp servers and clients with select.
Develop a tcp/ip stack memory manager and practice thread synchronization within Linux ipc concepts from scratch. Learn debugging techniques on Linux socket programming and network protocol development from scratch.
About This Course is about learning How Linux Processes Talk to each Other. This is a sub-domain of Linux System Programming. We shall explore various popular mechanism used in the industry through which Linux processes to exchange data with each other. We will go through the concepts in detail behind each IPC mechanism, discuss the implementation, and design and analyze the situation where the given IPC is preferred over others.
We also discuss how applications should be designed to use IPC facilities provided by underlying Linux OS.
You will have Assignments wherever possible, and throughout the course, there shall be one project in which you shall be incrementally applying the new IPC technique you have learned. Towards the end of the course, you would have practiced and applied all IPC techniques learned in this course.
Each IPC mechanism, we will have a detailed code walk in which I show you how actually a given IPC mechanism is implemented on sending and receiving side. When you would join the industry, from day 1 you will witness IPC concepts being applied all over the software in order to facilitate communication between different parts of the software.
Who should do this course ?
This course is meant for UG Computer science students, job seekers, and professional developers. This is a MUST do course for those who want to join MNCs as a developer in System Programming. In System Programming, almost all the time you have to use IPC to carry out data exchange between processes, therefore students graduating in computer science and looking to seek an opportunity in MNCs as a developer should have IPC concepts at his/her fingertips.
Pre-requisite
It shall be advantageous if you know a little about C and OS. We designed this course assuming the student is a complete beginner in Linux IPC and we raise the level of course gradually as we move from Basic to advance concepts wherever necessary.
Also, please just do not sit and watch my codes. Write your own codes, even if it is the same as mine!
Related Courses
RPC (Remote Procedure Calls) is another way of carrying out Inter-Process Communication between two processes running on separate machines in the network. You may also want to check my other course on Linux RPCs where you will learn how to implement RPCs from scratch.
Programming Language used In this course :
We have strong reasons to choose C as a language for this course:
IPC is a facility provided by the OS to developers to carry out data exchange between processes. Learning IPC using C helps you understand what is going on behind the scenes. C language really exposes the low-level details about how the system actually works. In System programming, C is the only language to be used and there is not even a remote substitute of this language when it comes to System programming.
No Third-Party libraries
Whatever logic you implement, you need to implement it from scratch. This course does not suggest taking the help of any third party library to get the jobs done. Use of external libraries completely defeats the purpose of the course. However, it is recommended to use third-party libraries for commonly used data structures such as linked lists/Trees/Queues, etc which saves a lot of time implementing these data structures.
Note1 : Though we use Linux to teach the IPC techniques, conceptually, IPC of Linux is not very different from other OS platforms such as windows, iOS, etc. So, if you are a programmer for other platforms, this course still holds great value for you.
Note2 : This Course talks about various techniques regarding exchanging data between processes, the other related topics such as process synchronization and locking is out of the scope of this course and will be covered separately.
Warning: This course has auto system-generated subtitles which may not be perfect. Please disable subtitles as per your convenience.
Curriculum
Introduction
Table of Contents
Computer Architecture - Overview
Various IPC Techniques
Communication Types
IPC Technique 1 - Unix Domain Sockets
Socket API Introduction
Socket Message Types
Socket Design Paradigm
Accept System Call
Introducing Unix Domain Socket
Unix Domain Socket Server Implementation
Unix Domain Socket Client Implementation
Multiplexing
Select System Call
Multiplexing Server State machine
Multiplexed Server Implementation
Data Synchronization - IPC Project part 1
IPC Technique 2 - Message Queue
Introduction
MsgQ as a Kernel Resource
Open & Create a MsgQ
Closing a MsgQ
Enque Data in MsgQ
Dequeue Data from MsgQ
Unlinking a MsgQ
Using a Msg Q - Design perspective
Bi-Directional Communication
Code Walk - Implementation
Demonstration
IPC Technique 3 - Shared Memory
Overall Design Goals
Concept of Virtual Memory
Program Control Block
Shared Memory Basics
Kernel Memory
mmap() - Memory Mapping
Design Constraint
Shared Memory related APIs
Data Synchronization - IPC Project part 2
IPC Technique 4 - Signals
Introduction
Linux Well knows Signals
Signals Generation and Trapping
Sending Signals using Kill()
Data Synchronization - IPC Project part 3
IPC Technique 5 - Network sockets
Socket Programming Design
Select System call
Accept System call
Concept of Multiplexing
Server State machine
Project on Socket Programming - IPC Project 4
Multiplexing on Different IPCs
Use select() to multiplex on different IPC interfaces