Linux IPC Project : Develop Asynchronous PUB-SUB System

Design coordinator data structures for publishers, subscribers, and their relationships, enabling registration, tracking of publisher and message IDs, and subscriber mappings in a linux ipc pub-sub system.

Implement management APIs for the pub-sub coordinator to insert, update, and delete publishers, subscribers, and publisher-subscriber records using unordered maps, based on the coordinator db header with 11 functions.

Launch the coordinator as a listener thread plus a configurable set of distribution threads using pthreads, handling a UDP port and preparing for message distribution to subscribers.

Create a makefile to build the static library lib code dot a and a coordinator executable with g++ and pthread, compiling coordinator start dot o and coordinator db dot o.

Implement the cmsg_t structure in the common directory for a Linux IPC pub-sub system, detailing message id, type, sub type, priority, code, id, and a TLV buffer.

Implement the coordinator listener thread to bind a udp socket on a defined ip and port, receive cmsg messages, and process publisher and subscriber messages with replies.

Explain the publisher registration flow in a Linux IPC pub-sub system: parse the name TLV, update the coordinator's publisher database, and return a unique publisher ID.

Implement a socket-based publisher and subscriber as separate processes, linked to a client library exposing register and unregister APIs to interact with the coordinator over a socket interface.

Implement a socket based publisher that registers with a coordinator over a UDP socket on port 50000, receives a publisher ID, and can unregister via the coordinator Unregister API.

Implement client library APIs for (un)registration with the broker in a linux ipc pub-sub project, building a TLV-backed message and dispatching it to the coordinator.

Update the makefile to compile a static client library libclient.a and a publisher socket executable, linking the publisher with the client library and required object files.

Start the coordinator, launch multiple publishers on distinct ports, and observe the coordinator assign IDs and update its database, then send unregister messages to verify registration handling.

Implement coordinator changes to handle sub message add and delete for both publisher and subscriber, introducing publisher publish/unpublish and subscriber subscribe/unsubscribe APIs that update the coordinator's databases.

Test the pub-sub ipc system by running the coordinator, publishers, and subscribers across ports 50,000–50,003, publishing and unsubscribing messages. Verify coordinator databases reflect registrations and subscriptions during changes.

This section outlines building a distribution system for an asynchronous pub-sub setup, showing how the coordinator enqueues messages into distribution queues and how the distribution thread dispatches them to subscribers.

Design a message distribution system with a coordinator listener and a distribution thread using high, medium, and low priority sub-queues to dispatch publisher messages to subscribers in a producer-consumer flow.

design a coordinator distribution queue with priority sub-queues using vectors; each entry pairs a message with its subscriber, while NQ and DQ use a mutex and DQ blocks when empty.

Implement reference counting for the Cmsg object to avoid premature deletion when queued for multiple subscribers in a distribution queue, using increase and dereference APIs, like a shared pointer.

Implement and mutex-protect enqueue and dequeue in the coordinator distribution queue, signal the distribution thread on new data, and release all vector data and resources to complete pub-sub integration.

Publishers send data messages to the coordinator for distribution by the message distribution system, using SUB_MSG_DATA with type 100 and a TLV payload, bridged by a coordinator API.

The coordinator launches a listener and multiple distribution threads, creates distribution queues, and uses a round-robin scheme to dispatch messages to subscribers, enabling load-balanced pub-sub distribution.

The coordinator receives data from the publisher and forwards the cmsg to the distribution system for subscriber delivery using a round-robin queue.

Implement an api in client and cpp that lets subscribers report their ipc channel to the coordinator, encoding udp ip and port or unix socket name in a cmsg tlv.

Show how the coordinator processes a subscriber’s IPC channel add request by updating the subscriber database with the chosen IPC method, handling UDP sockets, TLVs, and IP:port details.

Complete the coordinator_dispatch function for UDP socket based communication, sending messages to the subscriber via sendto, reusing or creating the socket as needed, with shared memory removed.

Enhance the subscriber socket example to register its IPC channel with the coordinator, wait for data messages, and decode TLV data in a final pub-sub demonstration.

Demonstrate an asynchronous pub-sub flow by coordinating a publisher and subscriber through a central coordinator, using UDP sockets to dispatch data messages from publisher to subscriber.

Explore the tlv concept—type, length, value—packing data flexibly, with one-byte type and length fields, enabling recipients to ignore unknown data and understand unit sizes.

Parse a tlv buffer by reading its type and length, compute the number of value units, and loop through strings while advancing to the next tlv without overrunning memory.

Develop an iterative macro to parse a TLV buffer, extracting type, length, and value for each TLV while guarding against buffer overruns.

Explore building an asynchronous pub-sub system on Linux IPC, as a bonus module, to enhance understanding of inter-process communication patterns.