Concurrency in Go
On my journey to build background agent platforms, one of the main reasons I picked Go as my language of choice is its flexibility, its concurrency primitives, and the fact that I have been using it for years at work on a CLI tool my team owns. You can also leverage its concurrency primitives to more easily handle concurrent and parallel computing tasks. With this in mind, I recently finished Concurrency in Go. This book put some fat on my mind and solidified how to use channels, sync groups, and patterns leveraging these things better.
Building Container Orchestrator
Now that I have read "How Linux Works", "Systems Programming Essentials with Go", and "Concurrency in Go", I am pushing through "Build an Orchestrator in Go". This book has been great, too. The book is teaching me how to build a basic container orchestrator leveraging a Manager, Worker, and Schedule components. The author intentionally skips over distributed systems aspects so that he can trap your focus on the basics of a Kubernetes like platform. So far I have built the worker components and added an API layer to each one. These components can take a task and start a docker container to run the task. You can list the tasks mapped to a worker, and stop a task. This is all done using the Chi http server framework and the Go Docker SDK.
Extensions to Book Logic
When I complete the project made with this book, I have some extensions I want to make to it to make the project my own. I would like to make the manager node fault tolerant with leader election or a leaderless approach using a quorum of 3 nodes. I want to swap out Docker to use VMMs like Firecracker Microvm, gvisor, and Cloud Hypervisor. Lastly, Id like to introduce an agent harness and move away from running images/containers and move towards creating sandboxes that the agents use to run their code. This books is pushing me in a great direction to work on all of this.
Non-Coding Reading
I fight Youtube, LinkedIn, and Facebook everyday for attention. The main weapons I yield for this battle are Audible and physical books. My current physical book is "The Data Center as a Computer. ChatGPT said there are helpful parallels between scheduling work for agents at scale in a background agent platform, and the way work and processes are scheduled in Warehouse Scale Computers. This book has been really interesting. I want to commit to it, but the physical copy will not be delivered for several weeks. For now, I am going to keep reading it, but I may pivot to Operating Systems: Three Easy Pieces to continue my research into learning enough Systems Programming and Operating Systems to be dangerous with creating MicroVM and container based agentic coding platforms.