
Explore Linux interprocess communication (IPC) techniques, including Unix sockets, message queues, shared memory, and signals, with demonstrations and C implementations, plus design considerations and an IPC project.
Join the Telegram group to discuss Linux inter process communication (IPC) from scratch in C.
In this lecture, I referred malloc/free as system calls, though they are not system calls. Whether or not they are system calls is out of context for this course, pls control your adrenaline - I admit they are not system calls.
This Lecture is for those who are absolute beginners with Linux and have no prior knowledge with Linux Installation. This course do not demands user to use Linux OS. All Assignments can be done on Windows platform also, however, it is recommended to use Linux for Development work. Please switch if you are still stuck with windows.
Pls go through this tutorial which will tour you through the Linux installation steps and supporting softwares. Those who are already using Linux can safely skip this Section.
Understand how sockets enable inter-process communication on Linux, for both same-machine and networked processes. Learn socket APIs and a client-server workflow using typical system calls.
Demonstrate socket message types in client and server communication, detailing connection initiation request messages used to establish a dedicated connection and service request messages sent after the connection is established.
Outline the socket design state machine for Linux IPC, from creating the master socket to accepting new clients and processing service requests with per-client data sockets.
Learn how the accept system call converts a server's master socket into a client communication file descriptor, enabling bidirectional, connection-oriented data exchange with new clients.
Explore interprocess communication on the same machine using Unix domain sockets, and learn server–client roles, stream versus datagram modes, and the socket lifecycle: create, bind, listen, accept, and connect.
Explore implementing a unix domain server in C via the server.c state machine, covering socket creation, 128-byte buffer, and summing numbers until zero.
Explore how a unix domain server is built from scratch in C, detailing bind, listen, accept, read, and write to receive integers, sum them until zero, and return the result.
Explain the unix domain client in C: create a data socket, connect using the server's socket name, send integers, and read the server's reply.
Explore a Unix domain client–server demonstration in C, showing compilation, binding, and accept calls, data exchange of integers, and a single-client limitation with directions for improvement.
Learn unix domain sockets for client-server IPC, using connect and accept, and write/read or send/recv, with blocking behavior noted. The one-client limit motivates multiplexing to handle many clients.
Learn how multiplexing lets a server handle multiple clients simultaneously, using the Linux select system call to monitor master and client file descriptors via the fd_set data structure.
Learn how to implement multiplexing in c using select and accept, maintaining a master socket and per-client fds in an fd set to handle multiple connections.
Demonstrates a multiplexed unix domain server in C that handles multiple clients using master socket and a monitored fd set with select, reads integers, sums them, and replies on zero.
Demonstrates a multiplexed Unix domain server in C that accepts multiple clients via select, computes per-client sums of integers, and handles new connections concurrently.
Learn to build a unix domain socket IPC system where a routing table manager propagates create, update, and delete operations across connected clients and dumps the table to new connections.
Explore message queue as a Linux IPC mechanism, learn its concepts and APIs, understand enqueue and dequeue workflows, and walk through a kernel-resident queue based sender and receiver implementation.
Create and use a kernel-space message queue from user-space processes to enable IPC, where queues are identified by a string ID that bridges sender and receiver.
Open or create a message queue with mq_open by naming the queue with a slash, setting flags and mode, optionally mq_attr, and returning an mqd_t handle for send and receive.
Close a message queue using mk_close by passing its file descriptor and receiving zero on success; the kernel decrements the reference count and destroys the queue when it reaches zero.
use mq_send to enqueue a message into a queue by passing queue file descriptor, message pointer, its length, and priority; it returns zero on success and may block if full.
Use mq_receive to fetch the oldest message with the highest priority from a message queue, filling the provided buffer and returning the bytes received; blocking behavior is controlled at open.
Learn how the message queue unlink API destroys a queue by releasing the kernel resource after all users close it, returning zero on success and minus one on failure.
Use a Linux message queue IPC: multiple senders enqueue to the same queue, while a single receiver dequeues; multiplex across queues with the select system call.
Download the source and run the sender and receiver C files to demonstrate message queue IPC, where the sender sends user input to the receiver and the receiver prints it.
Demonstrates Linux inter process communication in C using message queues by compiling and running a receiver and a sender, exchanging data via a named queue.
Explore how shared memory enables fast inter process communication in Linux by using memory mappings and system calls, with hands-on guidance on implementing sender and receiver processes.
I have a separate Course in which the entire paging is explained in detail.
Course name : Linux System Program Concepts and Techniques.
Explore memory mapping in Linux via the map system call, creating virtual pages that map to physical memory and update the page table, with text files mapped as virtual memory.
Learn how memory mapping via the map system call creates shared memory by mapping external memory into multiple processes' virtual address spaces, sharing physical pages and synchronizing changes.
Use RAM as a data source for inter-process communication by mapping virtual pages to shared physical memory, enabling instant updates across processes without external memory.
Demonstrate Linux ipc with shared memory via programs using mmap as a malloc substitute, importing text into address space, and using shm_open, ftruncate, mmap, read, write, and shm_unlink across processes.
Explore how the mmap map system call allocates dynamic memory like malloc, by passing null for the address and using read or read-write flags. Learn about private and anonymous mappings.
Map a disk text file into process virtual memory, perform read/write on mapped region, and use the sync system call to synchronize changes back to disk with a shared mapping.
Explore design constraints for shared memory as inter-process communication, with one publisher, many subscribers, and infrequent updates. Publishers notify subscribers after updates via inter-process communication mechanisms; subscribers read fresh memory.
Extend the routing table manager to synchronize mac addresses and their arp table and ip addresses via shared memory, with updates delivered through unix domain sockets to all clients.
Explore how Linux signals enable inter-process communication by signaling between processes, capturing signals, and applying default actions, signal handlers, and registration that pre-empts execution.
Explore Linux well known signals, including SIGINT, SIGUSR1 and SIGUSR2, SIGKILL, SIGABRT, SIGTERM, SIGSEGV, and SIGCHLD, and learn their default behaviors, catchability, and how child termination via fork triggers SIGCHLD.
Explore Linux signal generation: OS-delivered SIGINT, self-generated signals via raise, and inter-process signals via kill. Learn to trap signals with the signal API and implement handlers.
An Example demonstrating a use of signal.
Synchronize new clients with the rtm server by exchanging pids over unix domain sockets, updating in-memory routing and arp tables via shared memory, and signaling clients to flush on command.
Explore socket programming with a tcp focus, learn client-server architecture in C, and master multiplexing with the select and accept system calls to handle multiple clients.
designs a Linux socket based server, detailing the master socket, per-client handles, and connect and accept flows, then handles service request messages and service responses for multiple clients.
Analyze the accept system call on the TCP server to establish a bi-directional connection, handle client requests, and return a communication file descriptor for data exchange.
The select system call lets a server monitor multiple clients and new connections using a master socket and an fd_set, blocking until data arrives or a new client connects.
Demonstrate how to implement a multiplexing server by using select and accept together, managing a master socket and client fds within an fd set to handle multiple connections.
Follow the ten steps of the state machine to implement a tcp server; this lecture walks through setup, socket creation, bind, and listen with a tcp server dot c example.
This lecture explains building a tcp server that uses select to monitor the master socket, accept new clients, and exchange data in an infinite loop.
Read a and b from the client, compute their sum, and reply with the result; close the connection when both values are zero, and loop to await the next client.
Observe a blocking TCP server design where the server handles only one client at a time, blocked during data receive, illustrating limitations and the single-client use case.
design a tcp client in c by creating a socket, connecting to 127.0.0.1:2000, sending a and b, and receiving their sum in a loop.
Explore tcp server design with multiplexing to handle multiple clients using fd_set and select. Learn how to compile and run the tcp client and server on localhost and exchange values.
Explore the high level design of a multiplexed tcp server that can handle multiple clients using a master socket, per-client file descriptors, an fd array, and the select system call.
Explore a TCP server with multiplexing implemented via select, managing the master socket and client file descriptors, accepting connections, and handling data exchange across multiple clients.
Watch a live demonstration of a TCP server with multiplexing built in C, showing compilation, a master socket, select-based multiplexing, and handling multiple clients concurrently.
Review the socket APIs, including socket, select, accept, bind, listen, receive, send, and close, and analyze a TCP server with multiplexing and 31 connections, contrasting with UDP's simplicity.
Master multiplexing across network sockets, Unix domain sockets, and message queues by using select on a shared set of file descriptors, enabling asynchronous data reception by the server.
Learn to build a distributed transparent memory system using sockets in C, presenting an eight-gigabyte, multi-node memory as a single, user-visible memory space.
Deploy and simulate a six-node ring topology on a single machine by running the same program in six terminals, each binding to 127.0.0.1 with unique UDP and TCP ports.
Explain the put request algorithm in a ring topology, hashing K with K mod N to locate the node, forward via UDP, and retrieve X from the originator over TCP.
Explain how a get request is processed in a ring topology: hash the key, forward via get forward messages over UDP, and return x with a TCP get reply.
Define a unified IPC message structure with message IDs for put forward, get forward, put reply, and get reply, including key, x, and the originator's IP address and TCP port.
Describe how a node in a ring topology processes a put forward message by hashing the key and comparing the node ID; if originator, store key in hash table.
Follow the pseudo code to implement a tcp/udp ipc project in c using select to monitor tcp, udp, and console, handling put/get messages in a ring topology.
Learn how a single C program models a ring topology with hash function k mod n and roles, supporting put and get requests using tcp/udp servers and clients with select.
Develop a tcp/ip stack memory manager and practice thread synchronization within Linux ipc concepts from scratch. Learn debugging techniques on Linux socket programming and network protocol development from scratch.
About This Course is about learning How Linux Processes Talk to each Other. This is a sub-domain of Linux System Programming. We shall explore various popular mechanism used in the industry through which Linux processes to exchange data with each other. We will go through the concepts in detail behind each IPC mechanism, discuss the implementation, and design and analyze the situation where the given IPC is preferred over others.
We also discuss how applications should be designed to use IPC facilities provided by underlying Linux OS.
You will have Assignments wherever possible, and throughout the course, there shall be one project in which you shall be incrementally applying the new IPC technique you have learned. Towards the end of the course, you would have practiced and applied all IPC techniques learned in this course.
Each IPC mechanism, we will have a detailed code walk in which I show you how actually a given IPC mechanism is implemented on sending and receiving side. When you would join the industry, from day 1 you will witness IPC concepts being applied all over the software in order to facilitate communication between different parts of the software.
Who should do this course ?
This course is meant for UG Computer science students, job seekers, and professional developers. This is a MUST do course for those who want to join MNCs as a developer in System Programming. In System Programming, almost all the time you have to use IPC to carry out data exchange between processes, therefore students graduating in computer science and looking to seek an opportunity in MNCs as a developer should have IPC concepts at his/her fingertips.
Pre-requisite
It shall be advantageous if you know a little about C and OS. We designed this course assuming the student is a complete beginner in Linux IPC and we raise the level of course gradually as we move from Basic to advance concepts wherever necessary.
Also, please just do not sit and watch my codes. Write your own codes, even if it is the same as mine!
Related Courses
RPC (Remote Procedure Calls) is another way of carrying out Inter-Process Communication between two processes running on separate machines in the network. You may also want to check my other course on Linux RPCs where you will learn how to implement RPCs from scratch.
Programming Language used In this course :
We have strong reasons to choose C as a language for this course:
IPC is a facility provided by the OS to developers to carry out data exchange between processes. Learning IPC using C helps you understand what is going on behind the scenes. C language really exposes the low-level details about how the system actually works. In System programming, C is the only language to be used and there is not even a remote substitute of this language when it comes to System programming.
No Third-Party libraries
Whatever logic you implement, you need to implement it from scratch. This course does not suggest taking the help of any third party library to get the jobs done. Use of external libraries completely defeats the purpose of the course. However, it is recommended to use third-party libraries for commonly used data structures such as linked lists/Trees/Queues, etc which saves a lot of time implementing these data structures.
Note1 : Though we use Linux to teach the IPC techniques, conceptually, IPC of Linux is not very different from other OS platforms such as windows, iOS, etc. So, if you are a programmer for other platforms, this course still holds great value for you.
Note2 : This Course talks about various techniques regarding exchanging data between processes, the other related topics such as process synchronization and locking is out of the scope of this course and will be covered separately.
Warning: This course has auto system-generated subtitles which may not be perfect. Please disable subtitles as per your convenience.
Curriculum
Introduction
Table of Contents
Computer Architecture - Overview
Various IPC Techniques
Communication Types
IPC Technique 1 - Unix Domain Sockets
Socket API Introduction
Socket Message Types
Socket Design Paradigm
Accept System Call
Introducing Unix Domain Socket
Unix Domain Socket Server Implementation
Unix Domain Socket Client Implementation
Multiplexing
Select System Call
Multiplexing Server State machine
Multiplexed Server Implementation
Data Synchronization - IPC Project part 1
IPC Technique 2 - Message Queue
Introduction
MsgQ as a Kernel Resource
Open & Create a MsgQ
Closing a MsgQ
Enque Data in MsgQ
Dequeue Data from MsgQ
Unlinking a MsgQ
Using a Msg Q - Design perspective
Bi-Directional Communication
Code Walk - Implementation
Demonstration
IPC Technique 3 - Shared Memory
Overall Design Goals
Concept of Virtual Memory
Program Control Block
Shared Memory Basics
Kernel Memory
mmap() - Memory Mapping
Design Constraint
Shared Memory related APIs
Data Synchronization - IPC Project part 2
IPC Technique 4 - Signals
Introduction
Linux Well knows Signals
Signals Generation and Trapping
Sending Signals using Kill()
Data Synchronization - IPC Project part 3
IPC Technique 5 - Network sockets
Socket Programming Design
Select System call
Accept System call
Concept of Multiplexing
Server State machine
Project on Socket Programming - IPC Project 4
Multiplexing on Different IPCs
Use select() to multiplex on different IPC interfaces