Core principles for software that scales (in practice)
Sandro Maglione
Check out my newsletter ๐จโ๐ปWeb development
24 April 2024
โข5 min read
Sandro Maglione
Web development
Lately I am thinking about this more and more:
What are the principles to write code that scales? ๐ค
Leaving aside the specific language, design patterns, and theory. What should a developer focus in practice when writing large software projects?
Some (initial) ideas:
- ๐ Local reasoning
- Modularity
- Refactoring
- ๐ฉ Error handling
- ๐งช Testability
- Configuration
- ๐ฌ Debugging/Observability
- ๐จ State management
- Data structures
Concepts to understand about error handling ๐ ๐ Error vs Exception ๐ Stack track ๐ Error object ๐ try/catch/finally ๐ Union types (Option/Either) Error handling is at the core of every program Back to refine and master the basics ๐
This is how it looks like in practice ๐
Local reasoning: the secret for software that scales
You work in a car factory. You role is to attach doors to the frame of the car:
- You expect the car's frame to be delivered to you (input)
- You perform your role (attach the door)
- You produce a car with doors (output)
In this process you don't need to bother with anything else: get the frame, attach the door, move to the next frame.
This is what makes factories scale: local reasoning.
It works the same for code:
Local reasoning: property of some code wherein the correctness of the code can be inferred locally (under specified assumptions) without considering prior application state or all possible inputs
(Source)
There is more.
Every week I build a new open source project, with a new language or library, and teach you how I did it, what I learned, and how you can do the same. Join me and other 600+ readers.
Error handling
Error handling is not discussed enough ๐คจ
It's found and required in every software, but there is still a lot of confusion around it
In practice:
- Returns errors as values
- Propagate errors between functions
Testability
Tests cannot be an afterthought ๐๐ผโโ๏ธ
Writing testable software requires thinking about the structure of your code from the start
In practice:
- Pure functions: easy to (unit) test, same input always same output
- Dependency injection (manage effects): when side effects are necessary, inject mock implementations
Debugging/Observability
One thing is making code that compiles, another thing is verifying that it does what you expect (in every case).
Debugging is necessary: you cannot foresee everything that may happen.
When something doesn't work as expected, inspecting internal state at runtime becomes necessary ๐ฉ
In practice:
- Logging: not just text messages, but also timestamps, stack traces, relevant state, and more
- Metrics: inspect performance, count function calls, track external requests
- Tracing: complete overview of the lifetime of requests
Inspect every function call, how much time it takes, external requests, and more
State management
Values change: we want these changes to be performant, predictable, and easy to understand/visualize.
In practice:
- Data structures: optimize performance based on the most common operations
- State machines: define all possible states and make state transitions predictable and easy to visualize
You can read more about these topics below:
Regardless of programming language or area of expertise, these principles will always follow you everywhere ๐ป
These are underappreciated topics, I plan to cover each of them in more details (and in practice).
Stay tuned, this can become something big ๐คซ
See you next ๐