Write High Performance Web Server in C++

Socket Programming in C - GeeksforGeeks

What is Socket Programming?

Socket programming is a way of connecting two nodes on a network to communicate with each other. One socket(node) listens on a particular port at an IP, while the other socket reaches out to the other to form a connection. The server forms the listener socket while the client reaches out to the server.

Socket Programming in C++ - GeeksforGeeks

In C++, socket programming refers to the method of coOmmunication between two sockets on the network using a C++ program. We use the socket API to create a connection between the two programs running on the network, one of which receives the data by listening to the particular address port, and the other sends the data. One of the features of the socket programming is that it allows the bidirectional communication between the nodes.

Server Stages

There are 7 stages to create a socket server, we will discuss one by one.

1. Creating the Server Socket

We create socket by using the socket() system call. It is defined inside the header file.

Syntax

int serverSocket = socket(AF_INET, SOCK_STREAM, 0);

here,

socketfd: It is the file descriptor for the socket.

AF_INET: It specifies the IPv4 protocol family.

SOCK_STREAM: It defines that the TCP type socket.

2. Defining Server Address

We then define the server address using the following set of statements

sockaddr_in serverAddress;

serverAddress.sin_family = AF_INET;

serverAddress.sin_port = htons(8080);

serverAddress.sin_addr.s_addr = INADDR_ANY;

here,

sockaddr_in: It is the data type that is used to store the address of the socket.

htons(): This function is used to convert the unsigned int from machine byte order to network byte order.

INADDR_ANY: It is used when we don't want to bind our socket to any particular IP and instead make it listen to all the available IPs.

3. Binding the Server Socket

Then we bind the socket using the bind() call as shown.

bind(serverSocket, (struct sockaddr*)&serverAddress, sizeof(serverAddress));

4. Listening for Connections

We then tell the application to listen to the socket referred by the serverSocket.

listen(serverSocket, 5);

5. Accepting a Client Connection

The accept() call is used to accept the connection request that is recieved on the socket the application was listening to.

int clientSocket = accept(serverSocket, nullptr, nullptr);

6. Receiving Data from the Client

Then we start receiving the data from the client. We can specify the required buffer size so that it has enough space to receive the data sent the the client. The example of this is shown below.

char buffer[1024] = {0};

recv(clientSocket, buffer, sizeof(buffer), 0);

cout << "Message from client: " << buffer << endl;

7. Closing the Server Socket

We close the socket using the close() call and the associated socket descriptor.

close(serverSocket);

Client Stages

Similar to server, we also have to create a socket and specify the address. But instead of accepting request, we send the connection request when we can to sent the data using connect() call.

Then we sent the data using write() call. After all the operations are done, we close the connection using close() call.

1. Creating the Client Socket

int clientSocket = socket(AF_INET, SOCK_STREAM, 0);

2. Defining Server Address

sockaddr_in serverAddress;

serverAddress.sin_family = AF_INET;

serverAddress.sin_port = htons(8080);

serverAddress.sin_addr.s_addr = INADDR_ANY;

3. Connecting to the Server

connect(clientSocket, (struct sockaddr*)&serverAddress, sizeof(serverAddress));

4. Sending Data to the Server

const char* message = "Hello, server!";

send(clientSocket, message, strlen(message), 0);

5. Closing the Client Socket

We close the socket using the close() call and the associated socket descriptor.

close(clientSocket);

In next section, we will create some simple C++ programs to demonstrate the use of socket programming.

In this lecture we will use socket programming apis we learned in last lecture to create a socket server.

This server is very simple. When a client request a connection to the server, the server can create a connection to that client.

After the connection is established, the server will read data from the client and send some data to the client.

After data is sent successfully, the server will close itself.

Example:

Let's open VSCode and use ssh to connect to Ubuntu. After open our project folder, let's navigate to the folder day01.

In Makefile, we put the following contents to build our project.

Let's use make to build our program.

Let's launch the server program.

Before we write a socket client, we can use a tool called netcat to test our socket server.

netcat

Netcat (or nc) is a command-line utility that reads and writes data across network connections, using the TCP or UDP protocols. It is one of the most powerful tools in the network and system administrators arsenal, and it is considered as a Swiss army knife of networking tools.

Let's test our server with netcat.

nc 127.0.0.1 8888

Now a connection is estables between netcat and the server.

Let's type some message in the client side. Press Entry key to send data to the serve. We can see the message is received by the server.

Meanwhile, we can see the messge sent by the server is received by the client.

All the source code we discussed in this lecture can be downloaded in this github repository.

In next lecture, we will write a Socket Client to interact with our server.

In this lecture we will use socket programming apis we learned in last lecture to create a socket client.

This client is very simple. It requests a connection to the serve.

After the connection is established, the client will send a message to the server and receives a message from the serve.

After data is received successfully, the client will close itself.

Example:

Let's open VSCode and use ssh to connect to Ubuntu. After open our project folder, let's navigate to the folder day01.

All the source code we discussed in this lecture can be downloaded in this github repository.

In the server and client examples, we ignored error return of socket apis which is a bad programming style.

In next lecture we will disscuss how to write robust socket programs.

In previous section, we create a simple server and client program by using socket functions such as socket, bind, listen, accept, connect, etc. We ignored errors of these function calls. We should not ignore any errors or exceptions in our program. Otherwise some hard to find bugs will appear in our project.

For most system calls, it will returns -1 to indicate the error.

The header file defines the integer variable errno, which is set by system calls and some library functions in the event of an error to indicate what went wrong.

This table shows list of error numbers and its descriptions in Linux operation system.

You can see all the error numbers in the ChromiumOS > Reference website.

In order to make our code more concise, we create an error handle function errif . If condition is true, we will print the error symbolic name and the errmsg. Then we will call exit by passing EXIT_FAILURE to exit our program.

Let's use errif function to refactor our socket create and check codes. We only need two lines of code. The program codes we write become much concise.

Exampe:

Let's refactor the socket server with errif function.

Let's refactor the socket client with errif function.

Let's build our program.

Let's run the server.

Let's run the client.

Our server can handle just one client connection.

Let's see what will hanpen if we launch another client.

Let's type ctrl + c to terminate our client. We can see that the server and another client are closed too.

All the source code

In next lexture, we will learn to use epoll api to handle multiple client connections. The epoll technolgy is the cornerstone to build a high performance web server. We will build an echo server which can handle thousands of client connections by using this programming technology.

What Exactly is This "epoll" Thing?

The epoll system call provides an efficient "poll"-ing mechanism to monitor multiple file descriptors for events like data becoming available to read or space opening up to write. It builds this functionality on top of kernel data structures to avoid overhead from repeated user-kernel mode transitions suffered by older APIs.

The epoll maintains an in-kernel list of monitored descriptors per process. This is directly updated by the kernel whenever I/O events occur. The overhead of syncing these sets is bypassed – meaning efficiency and scalability is greatly improved, especially under load with thousands of client connections.

In technical terms, epoll_create() sets up an epoll instance, which tracks two descriptor sets:

Interest List: The file descriptors (sockets, pipes, etc) that the process wishes to monitor events for. This list is controlled from userspace via epoll_ctl().

Ready List: The descriptors that actually have events ready – data available, space to write, errors, etc. Managed completely by the kernel.

So conceptually, epoll is the glue connecting these lists, avoiding round-trips while keeping events flowing

With this foundation, the basis of use becomes:

• Call epoll_create() to initialize a new epoll instance

• Use epoll_ctl() to register descriptor(s) of interest

• Block on epoll_wait() for events ready on those descriptors

• Handle returned events by reading/writing descriptors

Level Triggered vs Edge Triggered Modes

A vital distinction to understand with epoll is the difference between level-triggered and edge-triggered modes. This impacts when exactly event notifications will be delivered by the kernel for a "ready" file descriptor.

The mode can be configured per-descriptor during epoll_ctl() by passing EPOLLET for edge-triggered, or 0 for level-triggered.

Level-Triggered Behavior

In level-triggered mode, the meaning is rather straightforward – notifications are generated and delivered as long as the underlying event condition exists.

For example, if 1KB of data is written to a socket, the kernel sees readable data is available and adds it to epoll‘s ready list. When you call epoll_wait(), that socket descriptor will be returned immediately until you drain the data by reading it. The "data available" condition persists.

The Difference With Edge-Triggered

Edge-triggered mode, on the other hand, delivers notifications solely on state changes. Once an event notification has been sent, the kernel will not continuously notify about the same state until something changes.

Going back to our example:

1KB data written to a socket

Descriptor added to epoll‘s ready list

Your code calls epoll_wait(), socket FD returned

You read 500 bytes from the socket

Now at this point, level-triggered mode would immediately return the socket FD again on the next epoll_wait() since readable data remains in the kernel buffer.

However, edge-triggered mode does not generate another event – because the state has not changed since step 1. The same 1KB data buffer still exists. So your code would block waiting for the remote side to write more data.

The difference is edge-triggered only notifies once on state changes, while level-triggered effectively provides continuous polling as long as the condition persists.

This has important implications on I/O processing patterns…

Impact on Application Code

This blocking behavior can easily introduce latency if application code expects to be processing every read as soon as it arrives. If using non-blocking sockets, for example, read calls may unexpectedly start returning EAGAIN even though data is sitting in the kernel!

To manage this and prevent stalls, properly handling EAGAIN is critical when using edge-triggered mode. Generally code should:

Use non-blocking socket descriptors

Check for EAGAIN after reads

Handle by stopping reads and immediately returning to epoll_wait()

This lets the process resume waiting for the next state change event, rather than attempting to read the same data repeatedly.

In practice, edge-triggered usage often outperforms level-triggered substantially under high load and multi-core servers. The reason is avoiding unnecessary notifications where state has not actually changed.

Example:

Let's rewrite our server programm with epoll system call.

All the source code

Now we have an echo server which can handle thounds of connections.

In next lecture we will use C++ OOP paradigm refactor our echo server.

Object Oriented Programming in C++ - GeeksforGeeks

Difference Between OOP and POP - Scaler Topics

What is Object Oriented Programming (OOP)? - Scaler Topics

Class Diagram | Unified Modeling Language (UML) - GeeksforGeeks

In previous section, we have build an echo server which can handle thounds of connections. Unitl now, the program paradigm we use is the Procedure-Oriented Programming (POP) . We put all the codes in just one file, which is not good programming practise.

As our program will become more powerful and more complex, it's necessary to modularize our program before going head.

Object-Oriented Programming (OOP) in C++ is a paradigm that uses objects to model real-world problems. It aims to implement real-world entities like inheritance, hiding, polymorphism, etc., in programming. The main goal of OOP is to bind together the data and the functions that operate on them, ensuring that no other part of the code can access this data except through these functions

1.

Conclusion

• OOP emphasizes data organization into objects, encapsulating data and the procedures (methods) that manipulate that data. The primary focus is on modeling real-world entities and their interactions.

• POP, on the other hand, revolves around procedures or functions that operate on data. It emphasizes breaking down a problem into steps or tasks to achieve the desired outcome.

In this section we will refactor our echo server with OOP paradigm.

The first class we are going to design is the InetAddres class.

The data and functions that manage the server address are encapsulated in the InetAddress class.

The data and functions that manage the socket are encapsulated in the Socket class.

The data and functions that manage the epoll instance are encapsulated in the Epoll class.

The relationships between the three classes is presented in this class diagram.

Example:

Let's open our project with VSCode. Let's navigate to the day04 folder.

Let's open the InetAddress class. In it's constructor, it setup the socket address.

Let's open the Socket class. It creates a socket instance in the constructor. And the socket instance is closed in the destructor. Other methods in this Socket class are used to manage the socket instance .

Let's open the Epoll class. It creats a epoll instance and a epoll_event array in the constructor. In the desctructor, the epoll instance is closed and the epoll_event array is deleted. In the addFd method, we add the fd and the event into the epoll's interest list. In the poll method, we fetch the ready event from the epoll's ready list.

In the main function,

Now we have built an echo server with OOP paradigm, which makes our program more modular. In next lecture we will introduce the Channel class to manage our event_poll data.

In previous lecture we use OOP diagram to make our program more modular. In this lecture we will ecapsulate our epoll_data in the Channel class.

Until now, we use the fd filed of the epoll_data to store and access the user data. How can we pass more data into the epoll_event?

Let's inspect the epoll_data data structure. The epoll_data is an union type, we can use the ptr field to pointer to an object of a class. We can define this class as the Channel class.

Now the relationships between our defined class is presented in this slide. We use the Channel class to interact with an epoll instance.

Example:

Let's open the Channel class. We can see that this class is very primitive because we only use this class to store the user data to be passed to the epoll event.

Le'ts open the Epoll class.

In the poll method, we get the Channel objects from the ready epoll_events, update the ready events and return the Channel objects.

We create an updateChannel method to add the epoll_event with a Channel object in the epoll tree or modify the epoll_event with a Channel object if it is already in the epoll's Interest list.

Let's build our program with make.

In next section we will learn the Reactor network programming pattern.

Understanding Reactor Pattern: Thread-Based and Event-Driven

Analysis of Reactor network programming model

In this lecture we will learn the widely used Reactor network programming pattern.

The Reactor pattern is widely used in server side develop, for example, Redis, Nginx, Nodejs, etc.

The Reactor Pattern

The reactor pattern is one implementation technique of event-driven architecture. In simple terms, it uses a single threaded event loop blocking on resource-emitting events and dispatches them to corresponding handlers and callbacks.

There is no need to block on I/O, as long as handlers and callbacks for events are registered to take care of them. Events refer to instances like a new incoming connection, ready for read, ready for write, etc. Those handlers/callbacks may utilize a thread pool in multi-core environments.

This pattern decouples the modular application-level code from reusable reactor implementation.

There are two important participants in the architecture of Reactor Pattern:

1. Reactor

A Reactor runs in a separate thread, and its job is to react to IO events by dispatching the work to the appropriate handler. It’s like a telephone operator in a company who answers calls from clients and transfers the line to the appropriate contact.

2. Handlers

A Handler performs the actual work to be done with an I/O event, similar to the actual officer in the company the client wants to speak to.

A reactor responds to I/O events by dispatching the appropriate handler. Handlers perform non-blocking actions.

Reactor under Single Thread Model

In the WebServer scenario, the Reactor is defined as the EventLoop class. It gets the ready event list by call the epoll_wait() system call in a loop. And dispatch the event to the appropriate Handler by the event type.

The Acceptor Handler is to establish a connection with the new incomming connection by call the accept system call. The Acceptor register the new connection fd and the Connection Handler into the epoll tree to observe the interest event. When events on the client fd is ready, the Reactor will dispatch the corresponding event to the Connection Handler.

The Connection Handler will read data from the client fd, handle the data and send response to the client.

We will design the EventLoop class , the Acceptor class and the Connection class in the following sections.

From this diagram we can see that the EventLoop class is the core class of the Reactor Pattern. In next lecture we will design the EventLoop Class.

The EventLoop class create an epoll instance and use this epoll instance to manage the event we are interested.

Example:

Let's open the EventLoop Class. The EventLoop is initailized with an Epoll instance.

In the loop method, it fetch the ready event list from the epoll intance. It dispatches the event to the Handler by call the Channel 's handleEvent method.

The updateChannel method in the EventLoop class is a wrapper method for the Epoll's updateChannel method.

Let's open the Channel Class and navigate to the handleEvent method. This method calls the callback functor passed from the Handle class by the setCallback method.

Now we need a class to manage Reactors and Handlers. We define this class as the Server Class.

Let's review the Server Class. In the constructor, it creates a Socket instance and prepares to handle the new incomming connections. Then it creates a Channel instance, set the newConnection method as the Channel's callback, then put the Channel into the epoll interesing event tree.

When new request arrived, the newConnection method will be called. In this function, we call accept system call to build a connection with the client and create a client socket. Then we create a Channel instance to register the client socket fd and the handleReadEvent method into the epoll tree.

When data is ready to read, the handleReadEvent function will be called. We can interact with the client with the read/write method.

In the server.cpp file, we create a main EventLoop and a Server, then call the loop method on the EventLoop. Now our server is ready to handle request from the client.

Let's look at this slide to see how our server works.

When our server program is launched, the server socket fd in registed in the epoll's interesting list with the EPOLLIN event and a Channel object. Our main thread will be blocked on the loop method, because the epoll_wait method will be blocked until some epoll events are read.

When a client sends a request to our server, an EPOLLIN event will be added to the read list and the EventLoop can get a Channel object from this ready event. The Channel's handleEvent method will be called. Then the Server's newConnection method will be called from the handleEvent. In the newConnetion method, we will call accept to establish a connection with the client. Then we will register the client fd into the epoll's tree with the EPOLLIN event and a Channel object.

When the client send message to the server, an EPOLLIN event will be added to the read list and the EventLoop can get a Channel Object from this ready event. The Channel's handleEvent method will be called. Then the Server's handleReadEvent method will be called from the handleEvent. In the handleEventMethod, the server will interact message with the client by call read and write method.

Example:

In next lecture we will create the Acceptor Class to handle the incomming connection.

In this lecture we will design the Acceptor Class to handle the incomming connections.

Example:

In it's constructor, it create a socket, bind the socket to the host and port. Listen on this socket fd, set the socket to nonblocking. Then create a channel with socket fd, setup the channel's callback with Acceptor's acceptConnection method. Very simple.

In the Server's constructor, we noly need to create the Acceptor instance and setup it's callback, nice and simple.

Let's look at this slide to see how our server works.

When our server program is launched, the Server object will create an Acceptor object. Then the server socket fd is registed in the epoll's interesting list with the EPOLLIN event and a Channel by the Acceptor object. Our main thread will be blocked on the loop method, because the epoll_wait method will be blocked until some epoll events are read.

When a client sends a request to our server, an EPOLLIN event will be added to the read list and the EventLoop can get a Channel object from this ready event. The Channel's handleEvent method will be called. Then the Acceptor's acceptConnection method will be called. And then Server's newConnection method will be called from the handleEvent. In the newConnetion method, we will call accept to establish a connection with the client. Then we will register the client fd into the epoll's tree with the EPOLLIN event and a Channel object.

When the client send message to the server, an EPOLLIN event will be added to the read list and the EventLoop can get a Channel Object from this ready event. The Channel's handleEvent method will be called. Then the Server's handleReadEvent method will be called from the handleEvent. In the handleEventMethod, the server will interact message with the client by call read and write method.

Example:

In next we will create the Connection class to interact data with the client.

In this lecture we will design the Connection class to read data from client and write response data back to the client.

Example:

Let's open the Connection class. In the constructor, it creates a Channel instance and registers this Channel instance into the epoll tree.

The callback is the echo method. In the cecho method, we can read data from the client and write data back to the client.

Let's see what changed in the Acceptor class. In the accepConnection method of the Acceptor class, we call accept to establish a Connection with the client. Then we call the newConnectionCallback of the Server class.

In the Server's newConnectionCallback method, we create a Connection object with the main loop and the client socket. Then we setup the Connection's deleteConnectionCallback and register this Connection object in the Server's connections map.

In the Connection constructor, we build a Channel with loop and socket. Then we set the Connection's echo method as the Channel's callback function. In the echo method, we can interact with the client by call the read and write method. If the connection with the Client is disconnected, we should call the deleteConnectionCallback.

In the Server's deleteConnection method, we should unregester the Connection object in the connections map.

Let's look at this slide to see how our server works.

When our server program is launched, the Server object will create an Acceptor object. Then the server socket fd in registed in the epoll's interesting list with the EPOLLIN event and a Channel by the Acceptor object. Our main thread will be blocked on the loop method, because the epoll_wait method will be blocked until some epoll events are read.

When a client sends a request to our server, an EPOLLIN event will be added to the read list and the EventLoop can get a Channel object from this ready event. The Channel's handleEvent method will be called. Then the Acceptor's acceptConnection method will be called.

In the acceptConnection, we call accept to establish a Connection with the client and create a Socket instance. Then we call the newConnectionCallback, which is the newConnection method of the Server class.

In the newConnection method, it creates a Connection object. The Connection object will register the client fd into the epoll's tree with the EPOLLIN event and a Channel object.

When the client send message to the server, an EPOLLIN event will be added to the read list and the EventLoop can get a Channel Object from this ready event. The Channel's handleEvent method will be called. Then the Connection's echo method will be called.

In the echo method, we can interact message with the client by call read and write method. If the connection with the Client is disconnected, we should call the deleteConnectionCallback method.

Example:

In next lecture we will implement Buffer class to improve our serve performance.

In this lecture we will learn how to improve IO performance with the Buffer class.

In the Connection's echo method, we write data back immediate after we read the data from the client. The write method appears in pair with the read method in a loop.

To reduce the system calls time, we can read all the data from the kernel to our created Buffer object. When the data is processed, we can call the write system call just once to write data back to the client.

Example:

The Buffer class use a std::string object to manage data. It has an append method to store data. We can use the c_str method to get the data. We can use the getLine method to fetch a line of data from the stdin.

Let's open the Connection's echo method. In the echo method, we call the read system call to get data from kernel. If we can get data from kernel, we store the data into the Buffer object. Else if the EINTR error accured, we can continue reading. If the EAGAIN or the EWOULDBLOCK error accured, this means we have read all the data from kernel. So we can write data from the Buffer object back to the client.

We can also use the Buffer in the client. We create a sendBuffer object to get data from stdin and send the data in this Buffer object. And we can create a readBuffer to store the data we read from the server side.

Run:

In next lecture we will discuss the multithreading network programming.

Concurrent programming allows the execution of multiple threads and thus we can write highly efficient programs by taking advantage of any parallelism available in computer system.

C++11 acknowledged the existence of multi-threaded programs and the later standards also brought some improvements.

Thread is basically a lightweight sub-process. It is an independent task running inside a program that can be started, interrupted and stopped. The this figure explains the life-cycle of a thread.

std::thread is the thread class that represents a single thread in C++. To start a thread we simply need to create a new thread object and pass the executing code to be called (i.e, a callable object) into the constructor of the object. Once the object is created a new thread is launched which will execute the code specified in callable.

Let’s break things down here. We created a simple function func that simply prints a line to the screen nothing fancy. std::thread t1(func) is going to launch thread that will execute our function while join() will make sure of waiting until t1 terminates. >>Run code

If we don't call join(), we will see abort() will be called.

Let’s make a few changes to the function fun to examine the workflow. In function func_loop, we create a loop to printf message. >>Run code:

The main function thread execution could be in the beginning or anywhere. The reason is that there is no strict ‘sequence’ of execution, rather both threads execute at the same time (that’s what concurrency is about right!), but main function will always exit at the very end. join() makes sure that t1 terminates before main function exits.

Using function is just one way to creating and initializing thread. We can use any of these 5 ways.

A Function Pointer

A Lambda Expression

A Function Object

Non-Static Member Function

Static Member Function

We will inspect the demo codes one by one.

Example: A function pointer can be a callable object to pass to the std::thread constructor for initializing a thread. We can pass arguments to the threads. The following code snippet demonstrates how it is done.

std::thread object can also be launched using a lambda expression as a callable. The following code snippet demonstrates how this is done.

Function Objects or Functors can also be used for launching a thread in C++. The following code snippet demonstrates how it is done:

We can also launch the thread using the non-static member function of a class. The following snippet demonstrates how to do it.

We can also launch the threads using static member functions.

Joining and Detaching Threads

When you create a thread, you need to tell the compiler what your relationship with it is going to be. We may use the join() method to wait for the thread to finish before we can take some action. If you want a thread to run independently of the main thread then you can use the detach() method.

Detached threads cannot be joined and you cannot undo the detachment.

Mutex stands for Mutual Exclusion. In C++, std::mutex class is a synchronization primitive that is used to protect the shared data from being accessed by multiple threads simultaneously. The shared data can be in the form of variables, data structures, etc.

std::mutex class implements mutex in C++. It is defined inside header file.

Need for Mutex in C++

In C++, when multiple threads modify the same shared resources simultaneously may cause race conditions. It may produce unpredictable output or unexpected behavior while executing the program. Mutex is used to avoid race conditions by locking the current thread so that all the other threads cannot access the shared resources at that time and unlocking it when the current thread is done.

Syntax for Mutex in C++

The use of mutex can be divided into three steps:

1. Create a std::mutex Object

2. Lock the Thread

The lock() function of the std::mutex class locks the thread and allows only the current thread to run until it is unlocked. It prevents the shared resource from being accessed by multiple threads simultaneously.

1. Unlock the thread

The unlock() of the std::mutex function is used to release the lock after execution of the code piece containing the possibility of race condition occurrence. It resumes all the waiting threads.

Example:

Let's see a program Without Mutex Synchronization.

Let's create a shared integer variable, which can be accessed globally inside the program. Create a function to increment the number by 1 for 1000000 times using a for loop. Create two threads named thread1 and thread2 to run the same increment() function.

In this case, thread1 will increment the number by 1 for 1000000 times and thread2 will increment the number by 1 for 1000000 times. So the expected output is 2000000.

However, there is a possibility of occurrences of race conditions when multiple threads try to modify the same resource simultaneously. So the value of the number cannot be predicted.

The same program is executed three times to observe the behavior when modifying shared resources without thread synchronization.

Explanation

It is clearly visible that the output of the program is unpredictable. When two threads are running at the same time causes race cases that create unpredictable output. There is no guarantee that the output to be 2000000. This unpredictable behavior is happening because of the concurrent modification of the same shared variable simultaneously using multiple threads.

Le'ts see the Code with Mutex Synchronization

Output

The same program is executed three times to observe the behavior when modifying shared resource with thread synchronization using mutex.

Explanation

Here, the output is stable. The threads are synchronous. Mutex object pauses all other thread than current thread using lock(). Now, only it allows one thread (current thread) at a time until it is unlocked using unlock() function.

C++ Condition Variable

A condition variable in C++ is a synchronization primitive used to block a thread until notified by another thread. Condition variables are typically used in conjunction with a mutex to ensure safe access to shared resources in a multithreading environment.

The syntax to declare a condition variable is simple:

After that, we use the associated method for different operations.

Typical Usage

CVS are usually used as "conditions" that depend on shared data. The typical usage-pattern is as follows.

Example:

Here is an example demonstrating the use of a condition variable in C++:

In this program, the consumer thread uses the condition variable cv to wait until data_ready is set to true while the producer thread sleeps for two seconds to mimic data generation.

In next lecture we will learn thread pool in C++.

A thread pool is a set of worker threads that pick up and execute tasks. Typically, the user would push tasks to a queue, and the worker threads can pop tasks from the queue to be executed, like shown in the diagram below.

A thread poll a typical application of the producer-consumer model.

A brief overview of the Producer and Consumer Model is provided in this picture:

Producer and Consumer Model Introduction:

The Producer and Consumer Model is a model applied to schedule how concurrent process and thread access shared resources. It consists of:

1. Producer: One or more processes/threads that produce data to the shared buffer OR releases hardware resources.

2. Consumer:One or more processes/threads that take in data from the shared buffer OR takes hadware resources.

3. Buffer: The shared buffer that may be accessed by both the producer and the consumer and is accessed by the producers to store resources and accessed by the consumers to take resources.

A producer could alse be relatively a consumer to the output of another producer and vice versa.

Example:

Here is an example demonstrating a simple thread pool in C++.

In the above code, we have used the following C++ features for the implementation of the thread pool:

1. A vector of worker threads, a task queue, a mutex for synchronization, a condition variable for signaling, and a boolean flag to indicate whether the pool should stop are all managed by the ThreadPool class.

2. The worker threads are initialized by the constructor, who then puts them in an endless loop while they wait for jobs to be enqueued. We use a wrapper class std::function over the given tasks.

3. A job is added to the queue and one of the worker threads is notified to begin executing it using the enqueue method.

4. To guarantee a clean shutdown, the destructor joins the worker threads, sets the stop flag, and informs all threads.

It should be noted that in a real-world situation, you would usually connect the threads or use some other kind of synchronization to make sure that all jobs are finished before the program ends.

Here is an example demonstrating a simple usage of ThreadPool. We create a thread pool and add a print job into the pool.

Let's build and run the program. Here the message is printed.

In next lexture we will write a WebServer Benchmark with thread pool.

In this lecture we will write a benchmark test for our server.

The usage of our benchmark is very simple.

./benchmark [-t thread_count] [-m message_count] [-w seconds]

-t Thread count we are going to create.

-m Message write times per task.

-t Wait time after connection established

Example:

Let's inspect how the benchmark is impleted. At the beginning of the program, we parse the argument. Then we create a thread pool and a function object. The target of the function object is the oneClient function. In the for loop we add tasks to the thread pool.

At the beginning of the oneClinet function, we send a request to server and wait for some seconds. In the while loop, write to server and then read data from server.

Run:

./benchmark -t 10 -m 10 -w 3

Let's increase thread count to 100. Nice and simple.

./benchmark -t 100 -m 10 -w 3

In next lecture we will refactor web server under multi reactor multithreaded pattern.

In Multi Reactor Multithreading Pattern, one main Reactor is responsible for monitoring all connection requests, and multiple sub reactors are responsible for monitoring and processing read / write requests, which reduces the pressure of the main Reactor and the delay caused by too much pressure of the main Reactor. And each sub Reactor belongs to an independent thread, and all operations of each Channel after successful connection are handled by the same thread. This ensures that all the States and contexts of the same request are in the same thread, avoids unnecessary context switching, and facilitates monitoring the response state of the request.

Example:

Let's open our project and navigate to folder day12.

In the constructor of the Server class, we create an Acceptor, setup newConnectionCallback.

Then we will create a ThreadPool with the number of hardware thread contexts available on the system.

And we will create a list of sub reactors.

In this loop, we create a function with the sub reactor's loop method and add the method into the ThreadPool.

In the Accetpor constructor, we create a server socket to listen the request from the client.

We create a Channel object and register acceptConnection method in the main Reactor.

When a request from the client arrived, the Acceptor's acceptConnection method will be called to accept a client and create a client socket. Then will execute newConnectionCallback with the client socket.

In Server's newConnection method, we will select a sub reactor from the sub reactor list at random.

We create a Connection object with this sub reactor and client socket, setup deleteConnectionCallback, and register this connection in the connections map.

In the Connection constructor, we create a Channel object and register echo method in the sub reactor. We alse create a read Buffer.

When a data ready to read event is happed on this socket, the Connection's echo method will be called. We can read data from the client by using read system call and write data to the client by using write system call in the Connection's send method. If the client is disconnected, or the connection is reset by peer, we will execute the delectionConnectionCallback method which is the Server's deleteConnection method.

In the Server's deleteConnection method, we unregester the connection from the connection's map and close the socket.

Run:

Let's build our program with make.

Let's launch the server.

Let's launch the benchmark.

It's works well, nice and simple.

In next lecture we will refactor the connection class to separate the echo method from the connection class. The echo method is a part of the business logic, so it should be implemented as the form of user defined behavior.

In the previous lectures, the echo method is implmented in Connection class. This is not a good desing pattern. The Connection class should not process the business login. The business login should be put in the server.cpp file.

This picture demonstrates how our server program works when a ready to read event happened.

When some ready to read event happened, the Channel's handleEvent method will be called by a sub reactor. In handleEvent method, a readCallback function wrapper will be called. The target of function wrapper is the echo method in Connection class.

In echo method, we can read data from the client, write data to the client and call Server's deleteConnection method.

Let's see how our server program works after we redesign our Connection class.

We can call the Server's onConnect method to register a lambda in server.cpp file. When the Channel's andleEvent method is called, the lambda in server.cpp weill be called.

In the lambda, we call Connection's read method, call Connection's write method and call Connection's close method.

Now our echo method is implemented in the Server.cpp file.

Example:

In Connection class, we add a read method to read data from client. We add a write method to write data to the client. We add a close method to close the socket.

Smart pointers in C++ are a powerful feature that helps manage dynamic memory and resource allocation, ensuring that programs are free of memory leaks and are exception-safe. They are part of the C++ Standard Library and are defined in the header file1.

Key Principles

Smart pointers are designed to automatically manage the lifetime of dynamically allocated objects. They encapsulate raw pointers and ensure that the memory is properly deallocated when the smart pointer goes out of scope. This is achieved through the RAII (Resource Acquisition Is Initialization) idiom, which ties resource management to object lifetime.

Types of Smart Pointers

1. unique_ptr

A unique_ptr is a smart pointer that owns and manages another object through a pointer and disposes of that object when the unique_ptr goes out of scope. It ensures that there is only one unique_ptr pointing to a given object, thus preventing multiple ownership.

Example:

In this example, P1 is a unique_ptr that manages a Rectangle object. When P1 goes out of scope, the Rectangle object is automatically deleted

2. shared_ptr

A shared_ptr is a smart pointer that allows multiple pointers to share ownership of an object. It maintains a reference count to keep track of how many shared_ptr instances are pointing to the same object. The object is destroyed when the last shared_ptr goes out of scope.

Example:

In this example, both P1 and P2 share ownership of the Rectangle object. The reference count is 2, indicating that two shared_ptr instances are pointing to the same object

3. weak_ptr

A weak_ptr is a smart pointer that holds a non-owning reference to an object managed by shared_ptr. It does not affect the reference count and is used to break circular references between shared_ptr instances.

Example:

In this example, P2 is a weak_ptr that observes the Rectangle object managed by P1. The reference count remains 1, indicating that P2 does not own the object

Benefits of Smart Pointers

Smart pointers provide several benefits:

• Automatic

Resource Management: They automatically release resources when they go out of scope, preventing memory leaks1.

• Exception

Safety: They ensure that resources are properly released even if an exception is thrown1.

• Simplified

Code: They reduce the need for explicit delete calls, making the code cleaner and less error-prone2.

Smart pointers are a crucial part of modern C++ programming, providing a robust and efficient way to manage dynamic memory and resources.

Conclusion

Smart pointers in C++ provide a robust mechanism for managing dynamic memory and resources, ensuring that objects are properly deleted when they are no longer needed. By using unique_ptr, shared_ptr, and weak_ptr, developers can avoid common pitfalls such as memory leaks and dangling pointers, leading to more reliable and maintainable code23.

In next lecture we will learn common problems with shared pointer.

There are 3 very common problems with shared pointers list in this slide.

Let's learn them one by one.

Circular references

We all know (I hope) about the classic cyclic reference problem that often arises when two objects both mutually own themselves and refer to themselves at the same time. Something along the lines:

This, of course, inevitably leads to a memory leak as the reference count never drops to zero for both of them.

Example:

The easiest way to break the circular reference is to use std::weak_ptr instead. The above example can be fixed by changing the Parent::mChild to be a std::weak_ptr:

Example:

Double Free with shared_ptr

The double free error occurs when a block of memory is freed more than once. This can happen in a number of ways, for example, when a pointer is deleted twice or when a pointer is deleted after it has already been deleted by another part of the program.

std::shared_ptr has a pointer to a struct that stores the reference count besides the pointer to the actual object. Therefore, when you create a std::shared_ptr from another one, it will increment the count properly (the two std::shared_ptrs point to the same struct).

If you create two std::shared_ptr from the same raw pointer, although they actually point to the same resource, they have no way of knowing it!

Example:

std::enable_shared_from_this

When you are not creating a std::shared_ptr of an object inside itself, you can obviously avoid the problem by creating std::shared_ptr from existing ones. How do we solve the problem if you must create inside the object? Turns out, there is something called std::enable_shared_from_this in the standard library.

std::enable_shared_from_this is a utility in C++ that allows an object to create shared pointers to itself. This is particularly useful when you want to ensure that multiple shared pointers manage the same object without creating separate ownership groups.

Example:

Key Principles

When a class inherits from std::enable_shared_from_this, it gains the ability to create std::shared_ptr instances that share ownership of the object. This is achieved through the shared_from_this member function. The primary advantage of this approach is that it prevents the creation of multiple ownership groups, which can lead to undefined behavior and potential double deletion of the object.

Shared pointers and threads

shared_ptr ownership and threads often lead to a problem that I refer to as “ownership inversion”. What do I mean by that? Have a look at the following example:

Examples:

What’s gonna happen if I run this? The expected result is that it should print “Hello, world!” after two seconds and terminate after three (-ish) more. But here’s what it does:

A owns the Timer which owns the thread t_. The functor scheduled on the timer is the only owner of A once it executes, both A and Timer are being destructed which leads to an attempt to join thread t_ from itself - resulting in an exception being thrown

In other words, the ownership has been transferred from the main thread to the Timer thread.

How to address that? std::weak_ptr is merely a mitigation. Once locked on Timer thread, the main thread might drop all its shared_ptr instances resulting in this problem back again.

The problem is in the ownership model and the lifetime of the objects which is incorrect. In this example, the lifetime of A on main thread has to extend beyond anything that’s happening on Timer thread.

These kind of bugs are unexpected and initially, difficult to spot by just looking at the code yet very common and indicative of bad ownership model, most of the time, caused by over reliance on shared_ptrs.

Conclusion

Ownership model is an important part of software design. Smart pointers are not just simple wrappers for managing resources but define objects lifetime, the dependency graph and ownership model between participating objects and modules.

As mentioned in C++ core guidelines one should prefer unique_ptr whenever possible before considering shared_ptr. This is the rule of thumb I try to follow and hopefully, with the examples I’ve discussed here, I’ve managed to convince you to do the same.

Hello everyone, in this lecture we will use C++ Smart Pointers to refactor our project.

This digram shows our server architecture we will create in this lecture.

Let's review our architecture.

Before we move into smart pointer, let's write a macro to disallow copy constructor, move constructor and assign operator.

Let's review the common.h

Let's review the socket class in folder day13. This class is used by the Acceptor and the Connection class. And we only use the fd property of the Socket class. So we can remove the socket class and move the methods of this class into the Acceptor class.

In the Acceptor's Create method we create a socket.

In the Acceptor's Bind method we bind the socket to the ip and port.

In the Acceptor's Listen method we listen on the created socket.

In the Acceptor's constructor.

We create a socket, bind the created socket to the ip and port, and listen on this socket.

And we create a channel instance, setup call back method. Register this channel into the epoll.

In the Constructor, after we success called the Bind method, this means we have prepared our server socket and ready to accept the connection from the client.

Let's review the Channel class. This is the core class the our web server.

We add the set_read_callback and set_write_callback method.

We save the ready_events and the listen_events.

Let's review the Epoll class.

Let's review the EventLoop class. We modify the epoll type to the unique_ptr.

We modify the Connection class name to TcpConnection.

We modify the Server class name to TcpServer.

In the echo_server.cpp, we create a TcpServer instance and start our serve.

We can use make to build our project. Let's launch our server and benchmark.

In next lecture we will manage TcpConnection Lifecycle with shared_ptr

We make TcpConnection inherit from the std::enable_shared_from_this, it gains the ability to create std::shared_ptr instances that share ownership of the object.

In the TcpServer, we use shared_ptr, to save the TcpConnection instance.

When TcpConnection is inserted into the connectionsMap, it's reference count is set to 1.

When the handleEvent method in the Channel is called, we call the lock method from the weak_ptr to increase the ref count by 1. After the handleEvent is called, the ref count will decreased by 1. As there is a ref from the connectionsMap, so the Connection instance is still alive.

In HandleClose method, it put off execute handleCloseInLoop method by bind a function and puts this function into the todo list. The handleCloseInLoop method will be executed in the next loop.

In handleCloseInLoop, the connection will be erased from the map, it's refrence count decreased to 1. And then the ConnectionDestructor method will be postphoe execute in the todo list of the sub looper.

And after the ConnectionDestructor is called in next loop, the ref count will be decreased to 1, and the Connection will be deleted.

In ConnectionDestructor method, the channel will be registered from the epoll.

Let's build our program and launch the serve.

In next lecture we will to migrate our build system from make to cmake.

In this lecture use cmake to build our web server.

We have three folders: folder base, folder tcp and folder test.

Let's put the following content in the CMakeLists.txt file .

Now we can build our project with CMake.

Let's create the folder build and goto this folder. mkdir build & cd build

We can use cmake to generate Makefile. cmake ..

We can use make to generate executable files. make

Let's launch our server. ./test/echoerver

In next lecture we will create EventLoop in SubThread.

In this leture we will Create EventLoop in SubThread.

Let's inspect the loop method in previous section. We create Sub EventLoops in the TcpServer's constructor, which means the sub EventLoop in Created in the main thread.

As the EventLoop's loop method is run in sub thread, so we need to get the thread id in the loop method.

In order to make our code more readable, we hope to create EventLoop int the sub thread, so we can get the thread id in the EventLoop's constructor.

Let's inspect the ThreadFunc in EventLoopThread class. This method will be run in the sub thread. It create an EventLoop, notify the main thread to run and call the loop's loop method.

The StartLoop method is called in the main thread. It create a thread to run the ThreadFunc method. It will blocked on the wait method until the EventLoop is created.

We create EventLoopThreadPool clas to manage our threads and loops.

In the start method, we create EventLoopThread in a for loop. We save the thread in threads and save the loop in loops.

In the nextloop method, we select a loop from loops.

In the TcpServer's constructor, we create the EventLoopThreadPool.

In the start method, we start the thread pool and the mian reactor.

Let's build our program and launch the serve.

In next lecture we will create a HTTP Server.

We use the HttpRequest class to save the parse result. The following members are used to save the method, version, request params ...

We use the HttpContext class to parse the http request. The parse algorithm is implemented in the ParseRequest method.

The algorithm is written in the State Machine design pattern. The parse state is declared in the HttpRequestParseState enum.

The state machine diagram is illustrated in this slide. We can write the parse algorithm according to this diagram.

We create a test case in test_httpcontext.cpp file.

Let's build our progam.

Le'ts run the test case. The http request is parsed correctly.

In next lecture, we will Create a Http Response

We use the Response class to save the response data. The following members are used to save the headers, status_code, status_message_, etc

In the message method, we convert the Response data to a string.

We create a HttpServer class to parse the http request and send http response. In the Constructor, We create a TcpServer, setup it's connection callback and message callback.

We SetHttpCallback with the HttpDefaultCallback method.

In the onConnection method, we print the connection information.

In the onMessage method, we parse the http request. If parse error, we send Bad Request information. If parse success, we call the onRequest method and reset context status.

In the onRequest method, we create a response with connection state. And we call the response_callback_ method to handle our response.

After response is processed, we send the response message to the client.

We call the connection's HandleClose method if the isCloseConnection method of response returns true.

We create a test case in http_server.cpp file.

In the main function, we create a httpserver, setup http response callback and start the http server.

In the HttpResponseCallback function, we setup response according to the request method and request url.

Let's build our progam.

Our http server works.

In next lecture, we will Create a Timer.

Hello everyone, in the following lectures we will learn to write a Timer with timerfd.

The timerfd interface is a Linux-specific set of functions that present POSIX timers as file descriptors (hence the fd) rather than signals thus avoiding all that tedious messing about with signal handlers.

timerfd_create() creates a new timer object, and returns a file descriptor that refers to that timer. The clockid argument specifies the clock that is used to mark the progress of the timer.

timerfd_settime() arms (starts) or disarms (stops) the timer referred to by the file descriptor fd. The new_value argument specifies the initial expiration and interval for the timer. If the old_value argument is not NULL, then the itimerspec structure that it points to is used to return the setting of the timer that was current at the time of the call.

timerfd_gettime() returns, in curr_value, an itimerspec structure that contains the current setting of the timer referred to by the file descriptor fd.

The following program creates a timer and then monitors its progress. The program accepts up to three command-line arguments. The first argument specifies the number of seconds for the initial expiration of the timer. The second argument specifies the interval for the timer, in seconds. The third argument specifies the number of times the program should allow the timer to expire before terminating. The second and third command-line arguments are optional.

Let's build the program. The following shell session demonstrates the use of the program:

The file descriptor returned by timerfd_create() supports the following additional operations:

Let's create a program to monitor timerfd by epoll.

Let's build the program. The following shell session demonstrates the use of the program:

The timerfd interface is a Linux-specific set of functions that present POSIX timers as file descriptors (hence the fd) rather than signals thus avoiding all that tedious messing about with signal handlers.

timerfd_create() creates a new timer object, and returns a file descriptor that refers to that timer. The clockid argument specifies the clock that is used to mark the progress of the timer.

timerfd_settime() arms (starts) or disarms (stops) the timer referred to by the file descriptor fd. The new_value argument specifies the initial expiration and interval for the timer. If the old_value argument is not NULL, then the itimerspec structure that it points to is used to return the setting of the timer that was current at the time of the call.

timerfd_gettime() returns, in curr_value, an itimerspec structure that contains the current setting of the timer referred to by the file descriptor fd.

The following program creates a timer and then monitors its progress. The program accepts up to three command-line arguments. The first argument specifies the number of seconds for the initial expiration of the timer. The second argument specifies the interval for the timer, in seconds. The third argument specifies the number of times the program should allow the timer to expire before terminating. The second and third command-line arguments are optional.

Let's build the program. The following shell session demonstrates the use of the program:

The file descriptor returned by timerfd_create() supports the following additional operations:

Let's create a program to monitor timerfd by epoll.

Let's build the program. The following shell session demonstrates the use of the program:

Hello everyone, in this lecture we will manage TcpConnection Lifetime with the Timer we created in previous lecture.

This diagram shows how we auto close the TcpConnection automatically. We will close the TcpConnection when socket is idle for 5 seconds.

Let's open our project and navigator to folder day21.

In the HttpServer, we define the macro AUTOCLOSETIMEOUT as 5.

In the onConnection method, we call the looper's RunAfter method. To call the ActiveCloseConn method 5 seconds later.

In the ActiveCloseConn method, we call connection.lock to get the TcpConnection ptr. Then we check whether the connection is expired.

If true, we call the handleClose method of Connection. Else, we will call the ActiveCloseConn method after 5 seconds.

Let's pay attention to the onMessage method. We call the connection's UpdateTimeStamp every time the onMessage method is called to ensure the connection's timestamp is updated.

Let's build our project with cmake.

Let's launch our server.

Let's lauch our client, send some message to the server and wait for about 5 second. We can see that the disconnect message is printed about 5 seconds later.

In next lecture we will create a logger.