Solution Architect Master Class

What is performance?

A software system's performance is determined by how quickly it responds to a given workload and by the current hardware and infrastructure resources available. These two parameters must remain constant when measuring performance, as changing them will cause a system's performance to fluctuate. For example, a system with a large amount of data is probably going to experience greater performance issues than one with a smaller amount of data on the backend. In a similar manner, a high number of requests will increase the likelihood of system performance issues. This means that we must maintain the request load constant when evaluating performance. This does not imply that our system cannot handle additional queries or store more information; simply put, these are the characteristics that should never be changed when evaluating performance.

Performance Objectives:

There are two main objectives to keep in mind when handling performance as it relates to solution architecture. Number one is to minimize latency, and number two is to maximize throughput.Minimizing Latency: The time it takes to process a request or a response is known as latency. It is crucial to keep in mind that during the course of a request or response traversing through the different components of a system, it sometimes waits in a queue for its turn to be processed. Therefore, the total latency of a request or response is the sum of the “wait time" plus the “processing time.". This detail allows us to conclude that whenever we are trying to minimize latency, we are in fact trying to minimize the wait time for processing, and the wait time is the time the requests sit in a queue waiting to be processed.

Software Performance Engineering (SPE) is a holistic approach to designing, implementing, and maintaining software systems with a focus on achieving optimal performance, scalability, and reliability. Performance considerations should be integrated into the software development life cycle from the beginning. The earliest you can identify and address performance issues will reduce the likelihood of costly fixes later in the development process. With this being said, the following software performance engineering principles will allow you to innovate highly performant software systems at scale. 1. Performance Requirement Documentation: Listing performance requirements and clearly defining and documenting expectations on latency, throughput, and utilization based on business goals and user demands will establish measurable performance objectives and provide a foundation for performance modeling, testing, optimization, and ongoing monitoring.

When we talk about network latency, there are two kinds of networks to which we are referring. One is the internet network, which establishes stable connectivity between a web browser, other devices, and a web server. A request has to navigate several roadblocks and checkpoints along the network route. Certain relay and control points might be quick, while others might not be as quick or as dependable. Furthermore, internet communications frequently span several networks and sub-networks over great distances. It goes without saying that establishing a connection between a client and a server via the internet involves a significant amount of latency. On the other hand, intranet communications are pretty reliable, but again, this does not mean that they're one hundred percent reliable. We can safely say that they are much more reliable compared to internet communication

CPU latency exists because of inefficient algorithms and because of context switching. We can manage inefficient algorithms by requiring software developers to use best practices when developing logic-sound algorithms. The second cause is context switching, which we will cover in this session as it is less evident. Thus, first we will examine how context switching degrades a system's performance, and then we will look into different strategies to mitigate the effects of context switching or reduce its impact on CPU latency and overall performance.

Let us talk about performance issues triggered by memory latency. So the first one that we can talk about is finite heap memory. Now, whether a process uses a heap for allocating objects or not, any process running on a machine can utilize only a finite amount of memory because there is only so much memory. Any process that exceeds that amount of memory is bound to fail, and it will crash.

Assuming that we have already addressed the serial request latency issues, including memory, CPU, and network latency, and that we have reduced latency as much as we have been able to. Concurrency is the other item that requires in-depth review in order to improve performance. What is concurrency in request processing terms? Let us try to explain this using the following example. Assume for the moment that the system is handling five requests sequentially. We refer to this as serial request processing. We would refer to this as a parallel process if the machine could instead process them simultaneously. In this case, we can say that the five requests are being processed in parallel instead of being processed in series. The fact is that in any system, what actually happens is that it is neither 100% serial nor 100% parallel.