Backend engineering is an art. Throughout my 18-year career of working with and building backend applications, I've discovered that certain communication design patterns consistently emerge. While there might be several ways for clients to communicate with backend applications, I believe the patterns discussed in this course are the most prevalent. These patterns include request-response, publish-subscribe, short and long polling, and push mechanisms.
Communication Design Patterns in Backend Engineering
Based on these communication design patterns, engineers can utilize numerous protocols for effective communication. Although core transport vehicles are limited to either TCP or UDP, a variety of industry-specific protocols have been developed atop these to address particular problems and use cases. Examples of these high-level protocols include HTTP/1.1, HTTP/2, HTTP/3, gRPC, WebRTC, and many others. For instance, the QUIC protocol, which was built on UDP, aims to enhance HTTP/2 streaming at the transport level. Each protocol comes with its own advantages and disadvantages, making it suitable for specific scenarios. In this course, I explore the most common protocols, offering examples and demonstrations where applicable.
Connection Establishment in Backend Applications
Before a client can send a request, it must establish a connection to the backend. It's crucial to understand how this connection is established, which parts of the connection process are managed by the kernel, and which are handled by the backend application process. Understanding how the backend application accepts connections speedily is vital to prevent the kernel queue from filling up, which would prevent further client connections.
Request Handling in Backend Applications
After the connection is established, the client sends a request. But what happens exactly within the backend application to process this request? Understanding what constitutes a request and the cost of parsing it based on the protocol equips engineers with better tools for troubleshooting performance issues or bugs. Appreciating the work involved in parsing requests is crucial for effective backend engineering.
Execution Styles in Backend Development
Once the request reaches the backend, the application must execute it. There is a wide array of design choices available regarding execution styles, including the use of processes, threads, multi-processing, and multi-threading. The choice of execution style should consider the number of CPU cores or hardware threads. Understanding these differences is essential for selecting the right execution pattern, although one can always innovate new patterns tailored to specific needs.
Course Requirements and Recommendations
This course is designed for engineers who have experience building backend applications. It is an intermediate to advanced level course, requiring certain programming and networking knowledge. If you lack the necessary networking skills, I recommend taking my "Fundamentals of Network Engineering" course before embarking on this one. I hope you find this course enjoyable and beneficial. Thank you for considering it.
