Teach on Udemy

Turn what you know into an opportunity and reach millions around the world.

Learn More

Your cart is empty.

Keep shopping

Build Remote Procedure Calls (RPC) - from scratch in C

Name: Build Remote Procedure Calls (RPC) - from scratch in C
Rating: 4.6 (154 reviews)

System Programming Projects - Server Client RPC Model over a Network Sockets, Data Serialization - Windows/Linux

Created byAbhishek CSEPracticals, Shiwani Nigam, Ekta Ekta

Last updated 4/2022

English

English [Auto],

What you'll learn

Shall be able to build Remote procedure Calls starting from #include <stdio.h>
Will understand the concept of Data Serialization and DeSerialization
Understand the concept of Application state Synchronization
Understand the concept of Application State Checkpointing
Will understand how to manipulate data objects as stream of bytes
Will be able to understand how to manipulate the objects Recursively
Will be able to extend and apply the concepts learned in other programming languages
Will understand the pre-requistes required for data transmission between heterogeneous machines

Course content

8 sections • 65 lectures • 6h 10m total length

Table of Contents3:10
Learn data serialization concepts and streams to flatten data for network transmission. Implement serialization, decentralization, remote procedure calls, state synchronization, and checkpointing from scratch in C.
Join Telegram Group0:13
Join the Telegram group for the RPC from scratch in C course to connect with peers and stay updated on course discussions.

Agenda2:15
Introduction7:24
Explore data serialization and decentralization, converting a program’s internal data into serialized form for network transmission and reconstructing it on the receiving side to enable heterogeneous machines to communicate.
Why We need Serialization ?4:19
Problems Without DeSerialization12:07
Application of Serialization3:14
Serialize in-memory data structures into platform-independent byte streams for storage, file transfer, or network transmission. Decentralize by reading those bytes to reconstruct exact object states on remote machines.
Summary1:23
Explore the challenges of cross-machine communication over a network and learn how serialization and decentralization enable data exchange between similar C structures on sending and receiving machines.

Serialization Basics1:09
Let us Learn the Basics of Serialization and DeSerialization in this Section. After this Section you will understand What is meant by Data (De)Serialization
Serializing Simple Structures3:56
Serializing Nested Structures2:24
Serializing Pointer Structures6:43
Explains how to serialize structures with pointer fields by flattening them, replacing pointers with actual content, removing addresses, and using a sentinel value for null pointers.
DeSerializing Simple Structures3:40
Explore decentralization by deserializing flat data into simple c structures, including nested and pointer members. Reconstruct memory layouts and copy serialized fields to the actual object.
DeSerializing Nested Structures2:49
Learn to deserialize an object with a nested structure by copying fields from the serialized version into a live object, and understand how the compiler adds padding bytes in memory.
DeSerializing Pointer Structures5:36
DeSerialization Example9:05
Apply serialization and deserialization to a nested memory model with person and occupation structures, showing how null pointers become sentinels in serialization and are reconstructed during deserialization.
A Big Picture3:18
Complete Sequence of Steps - FlowChart1:54
Serialization API Prototypes5:22
Summary2:13

What are our Goals ?3:55
C Quick Revision5:28
Serializing Simple Structures11:22
Serializing Nested Structures4:05
Serializing Pointer Structures5:41
DeSerializing Simple Structures6:43
Learn to implement a deserializing routine for a simple structure by reading a serialized buffer, detecting a sentinel, and reconstructing the object with name, age, and weight.
DeSerialization Conventions2:14
DeSerializing Nested Structures3:46
Explore how to implement a deserializing routine for nested structures, reconstructing a person with an embedded occupation by using sentinel detection, invoking the occupation routine, and performing shallow copy.
DeSerializing Pointer Structures5:31
Serializing a Linked List6:27
Serialize a linked list in C by implementing recursive serializing routines for nodes and the head, and deserialize to restore the original structure with sentinel insertion and macros.
PitFall of a Serialization-DeSerialization12:00
Limitations Of (De)Serialization3:16
Demo - Code Walk1:45
Walk through a serialization and decentralization example for structures, download and clone the repo, explore five files, and compile and run the final executable to study the concept.
Code Walk C Structures - Part11:54
Code Walk C Structures - Part29:11
Code Walk C Structures - Part310:09
Code Walk C Structures - Part49:36
Explore serialization and deserialization of a C person object: build fields, serialize to a buffer, send to a remote machine, reconstruct, and verify P1 equals P2.
Exercises0:25
Practice building remote procedure calls (RPC) in C through a series of exercises that reinforce from-scratch design, implementation, and testing of RPC mechanisms.
From QnA archive

Agenda1:41
Understanding RPCs5:47
9 Steps Of RPC7:13
RPC Steps - 1 to 37:15
Describe phase one of the RPC mechanism, detailing steps one to three, including client-side marshalling into a serialized buffer, network transmission, and receiving the result.
RPC Steps - 1 to 3 - Code Walk17:05
Walk through step one to three of rpc in c, implementing a multiply client, serializing and marshaling arguments, and exchanging buffers with a local server via sockets.
RPC Steps - 4 to 65:34
Examine how the rpc server receives serialized data, unmarshals and reconstructs arguments, and invokes the remote procedure call that multiplies two numbers, completing steps four to six in phase two.
RPC Steps - 4 to 6 - Code Walk11:16
RPC Steps - 7 to 83:25
RPC Steps - 7 to 8 - Code Walk5:32
RPC Steps - 9 - Final Step2:35
Complete the rpc cycle by reconstructing the final output on the client from the serialized data received from the rpc server, executing step nine of the rpc process.
RPC Steps - 9 - Final Step - Code Walk7:20
Demonstrate the final steps of a remote procedure call: serialize arguments, transmit to the rpc server, receive and reconstruct the result, and free buffers to prevent memory leaks.
RPC Demo3:34
RPC - Four Special Routines5:49
RPC Identity7:32
Attach a header to serialized RPC arguments to convey RPC identity, including RPC ID and payload size, so the server invokes the correct operation, and note serialized size excludes padding.
Attaching RPC Header On RPC Client7:41
Decoding RPC Header On RPC Server5:00
RPC Use Cases and What Next in the Course8:03
Explore real-world use cases of remote procedure calls, including airlines sharing data with travel sites via secure rpc channels. Preview data serialization and state transfer concepts in distributed systems.
RPC Coding Assignment0:51
Follow the example, multiply.c to complete this assignment. Shoot any Questions in QnA section, I will help you out.
hint : Write down serialie/Deserialize routines for all the structures involved in the question - Linked List, Linked List node, person_t.
For a Given RPC, write down 4 RPC routines.
Implement the same 9 steps we discussed. Remember to free any memory that you malloc after use.

Application State Synchronization - Introduction9:20
Application State Synchronization - Implementation8:36
State Synchronization - Demonstration9:45
Bonus Lecture - Time to get some Candies !!1:16
Explore Linux socket programming, multithreading, and debugging techniques from absolute scratch to project work, boosting your résumé and opportunities in operating system networking and Linux system programming.

Requirements

C programming language - at-least an intermediate level
Minimal Socket programming to the extent to send and receive data is desirable but not mandatory
Linux OS, any flavor, - Running as Native or as a VM
Most Important tool - Enthusiasm and love for knowledge

Description

About This course is about Developing your own Remote procedure calls - I will use Linux OS for this course, however you can use Windows OS if you are used to it. The essence and real strength of this course is No use of any third party libraries. I follow this principle in all my other courses. Whatever you learn through my courses, you learn from absolute ground level. This course does not violate the principle and teaches you how to build Remote Procedure Calls step by step from absolute scratch - No framework, tools, supporting libraries or anything - just pure C.

This course actually lays the foundation of many future System Software Projects. Few of which are below and is a part of this course curriculum.

Remote Procedure Calls (RPC) is a technique to invoke the function/procedure which actually resides on different physical machine running somewhere else in the network - hence the name remote procedures. In this course, you will learn the concepts working behind the scenes. The same concepts can be extended to implement other system programming concepts, besides RPC, such as - Data Synchronization and Check-pointing the application state. This course promise to deliver the complete content on developing RPCs in its initial release.

Data Synchronization - It is a process to synchronize the complete application Heap state to remote machine. The remote machine will build the mirror heap state. In the event, the first machine fails, the remote machine can take over as it has all the state required to resume the operation of failed machine.

Check pointing - It is a process of saving the application Memory state to disk/file persistently, so that, the application can be restarted/resumed any time building the exact same Memory state from Memory snapshot stored earlier to secondary storage.

Check pointing shall be delivered in subsequent releases of this course.

Who should do this course ?

Beginners Please take this course at your discretion. You should be good with C pointers and how C objects are laid out in memory. I expect you to be at-least above beginner level in C programming. This means, that only very enthusiastic students who wants to get an edge over the smartest student in their college should enroll. Average students Pls excuse. Job seekers and professional developers Must enroll. The concepts you learn from this course is language agnostic and having learned them will enable you to implement the RPC/Data-Synch/Checkpointing in any programming language of your choice. If tomorrow you happen to work in Java, you shall be knowing how RPCs work at the lowest level of implementation.

Pre-requisite

C and being good at pointers is a pre-requiste of this course. A minimal socket programming back-ground is desirable but not mandatory. We designed this course starting from absolute basics and building the foundation of learners first before actually pulling the course at full throttle. If you are not good with pointers and memory manipulation in C, Pls enroll only after meeting the pre-requisite criteria.

Also, Please just do not sit and watch my codes. Write your own codes, even if it is same as mine !

Programming Language used In this course :

We have a strong reasons to choose C as a language for this course:

RPC is a technique which if knowing the concepts can be implemented in any programming language of your choice. Learning RPC using C helps you understand what is going on behind the scenes. C language really exposes the low level details about how system actually works. In System programming, C is the only language to be used and there is not even a remote substitute of this language when it comes to System programming.

No Third Party libraries

Whatever logic you implement, you need to implement it from scratch, beginning from #include <stdio.h>. This course do not suggest taking help of any third party library to get the jobs done. Use of external libraries completely defeats the purpose of the course. However, it is recommended to use third party libraries for commonly used data structures such as linked lists/Trees/Queues etc which saves a lot of time implementing these data structures.

Related Courses

RPC is one way of carrying out Inter Process Communication between two processes running on separate machines in the network. You may also want to check my another course in which Linux IPC techniques has been discussed.

Warning : This course has auto system-generated subtitles which may not be perfect. Please disable subtitles as per your convenience.

Curriculum

This Course is divided into two major parts -

1. Understanding the Concept of Serialization and DeSerialization in great detail

2. Using the Serialization and DeSerialization to actually solve/build system. This includes :

Building Remote Procedure Calls
State Synchronization
Check pointing the application state

Section 2, 3, 4, 5, 6 are dedicated to build up the base on thoroughly mastering the concept of Serialization and DeSerialization.

Section 7, 8, 9 are dedicated to build and develop above stated systems

Section 1 - Get Started
- Table Of Contents of the Entire Course
- Linux Installation for Beginners
Section 2 - What is Serialization and Why we need it ?
Section 3 - Concept of Data Serialization and DeSerialization
- Serializing and DeSerializing Simple C Structures
- Serializing and DeSerializing Nested C Structures
- Serializing and DeSerializing Pointer C Structures
Section 4 - STREAMS - A Data Structure
- Design and Implementation
Section 5 - Data Serialization and DeSerialization Implementation in C
- Serializing and DeSerializing Simple C Structures
- Serializing and DeSerializing Nested C Structures
- Serializing and DeSerializing Pointer C Structures
- An Example
Section 6 - Serializing Generic Data structures
- Use Function Pointers to Serialize void *
Section 7 - Implementing Remote procedure calls from Scratch
- Understanding RPC Concept and Design
- Developing Client Stubs - Marshalling of RPC Arguments
- Developing Server Stubs - UnMarshalling of RPC Arguments
- Developing Server Stubs - Marshalling of RPC Return Type
- Developing Client Stubs - UnMarshalling of RPC Return Type
- Concept of RPC Identity
- RPC Use Cases
Section 8 - State Synchronization
Section 9 - Checkpointing (Coming Soon)

**Audit Trial **

30 Sept 2018 - Added Section 8 on State Synchronization

29 Sept 2018 - Added Section 6 on Serializing Generic Data structures

Who this course is for:

Students who want to distinguish themselves from the crowd
Professionals looking to advance their career Or switch jobs
those Who love Coding and build actual System softwares

Build Remote Procedure Calls (RPC) - from scratch in C

What you'll learn

Explore related topics

Course content

Get Started2 lectures • 3min

Introduction6 lectures • 31min

Concept of Data Serialization and DeSerialization12 lectures • 48min

Data Structure - STREAMS3 lectures • 22min

Serialization and DeSerialization Implementation in C18 lectures • 1hr 43min

Serializing The Generic Data Structures2 lectures • 20min

Implementation of Remote Procedure Calls Infrastructure18 lectures • 1hr 53min

Application State Synchronization4 lectures • 29min

Requirements

Description

Who this course is for: