Incoherent ramblings, devlogs, edgy philosophy and other topics of interest
[#114] [Mon, 22 Mar 2021 19:22:02 CST][tech]
Now, I've been meaning to write a few tutorials about some of the subjects that, in my experience, give aspiring programmers the most trouble. But 2021 has so far been a very busy year with no signs of slowing down, since I can't really spare the time to start yet another secondary project, I settled on trying to be more thorough as to how it is that we get actual engineering jobs done and documenting the relevant parts of the programming process in a tutorial format. Thankfully, the client agreed to release the finished software for free, making it a great opportunity to test out the idea.
Since pretty much all modern operating systems concern themselves with providing only the absolute minimum through their networking API, it befalls to library developers to implement a more robust solution for user programs to use as an intermediary. Features like multithreading, OS API abstraction, data serialization, packet queuing, SSL / TLS encryption and connection lifetime management are some of the more common demands of a typical, modern networking app. As for the sockets themselves, both the UNIX and Windows interfaces are very much the same but with enough minor differences to merit some abstraction, ironically the simplest problem to solve. The others? Not so trivial. Thus, we'll be employing the ASIO networking library as foundation.
Like its name implies, the library focuses in providing "Asynchronous Input & Output" that can be used in a variety of ways, networking and serial communication being among the more common. Socket centric projects, in particular, don't really have much of a choice but to adopt an asynchronous approach to combat the aforementioned problems, not so easy in the 90s but in actuality, tools like ASIO and modern C++ optimize the process quite nicely. Hence, the birth and raise of independent, high performance services, like private MMO servers (of which ASIO has sponsored many) and crypto trading bots have helped cement ASIO as arguably the best networking library for C++. So much so that I wouldn't be surprised if it (eventually) gets incorporated into the official C++ standard, not unlike a few other liboost projects.
The main idea is to get around the unpredictability of sockets by calling the relevant functions from a different thread to prevent pausing the rest of the program, and even though ASIO makes this (relatively) easy, there are still a few details to consider.
The client also specified that the program will be running on virtual machines without a GUI, and given the I/O (not to mention A5STH5TIC) limitations of a vanilla console, ncurses was added as a dependency. Even though it's older than me, it has yet to be trumped as the king of console UI libraries, and with good reason. It's very efficient, fast, tested, documented, easy to incorporate and likely already installed in whatever UNIX system the client will be using. However, it has absolutely no multithreading support. Working asynchronously with multiple threads can trigger memory related crashes if an ncurses function is called from anywhere other than its respective thread. A critical limitation for a tool that requires multiple threads to work, but it's by no means impossible.
Consolidating the threading discrepancies between ASIO's internals and ncurses is all about discerning how is it that ASIO will (asynchronously) communicate with ncurses to make sure this only happens in a predictable, thread-safe manner.
This extends to the internal I/O buffer storage and packet queue as well, any thread parsing a packet into a queue for incoming data must do so in harmony with the rest of the threads processing the ASIO io_context that may trigger an operation on the queue while it's already in use. So, for starters, we have to make note of where such events will happen.
The graph pinpoints the critical sections of the executable and their relationships, starting with the beginning of the main thread (top left). As previously stated, all ncurses functions have to be called from the main thread or risk undefined behavior, luckily there are only three relevant functions out of which only one should be accessed by secondary threads; the window output / system logging function. This is the only access to the window's output available, meaning its internal container must be protected to prevent simultaneous writes by the rest of the threads.
Do note that even though the UI drawing function will be invoked solely by the main thread, the container storing the window's output could be modified by a secondary thread as the draw function is processing it. This container has to be protected during UI processing as well, or it could be updated as it's being iterated through. That's makes two sections of the main thread that must be accounted for when writing the program. As for the rest of the threads, the relevant action will be happening between a handful of functions that interact with the socket themselves. Requesting or accepting connections, reading from and writing to the sockets, processing requests and logging through ncurses, it will all be done asynchronously from any of the secondary threads.
Now, what exactly is it to be done to protect these vulnerable data structures? The most common approach is to use locks and mutexes to limit access to the protected resources to one thread at a time. Despite not being the fastest spell in the book, it can still perform at a superb level as long as the locking period of the mutexes remains as short as possible. Another key reason as to why performance sensitive netcode should always be planned ahead of time, if we design a multi-threaded program around locking and waiting but fail to account for the workload of said function, we risk bottle-necking the tool's performance by often difficult to change code.
In our case, the only procedure that risks such behavior is the request resolution function, the rest shouldn't be doing much more than adding or removing a small object (representing a packet) to the connection's internal packet queue, but resolving requests may entail more complex behavior that wouldn't play well with lock based designs. Still, so far we've covered most of the expected pitfalls of working with high-octane netcode in the context of our project.
I'll cover the source code in the next post, this one has grown long enough.