Dear founder,
There’s this famous saying by Paul Graham from one of his most well-known essays: a command to all founders and startup operators to “do things that don’t scale.” Graham explains that at the beginning of starting a business, there are things you should do as a founder that you know will never work once you have hundreds or thousands of customers.
But it’s precisely these things that make it happen.
The learning from these hands-on projects allows you to build a repeatable process that can then be automated or handed over to others.
🎧 Listen to this on my podcast.
This entrepreneurial wisdom works wonders in our community. You see it in small indie hacker projects where the founder helps people integrate a system or service into their stack. You see it on a larger scale too, like with Airbnb, which initially relied on the founders going into people’s homes, taking photographs, and listing them on their website. Doing things at scale can only happen if you do things that don’t scale first.
Over the last couple of days, I’ve been experiencing the technical flip side of that wisdom while building Podscan. I’ve been talking a lot to clients and helping them implement the project, knowing I could never do this with dozens or hundreds of them simultaneously. So far, I’ve been following that advice, but unfortunately, it also bled into my development work.
Here’s the problem with the difference between development and operating a business: It’s very easy in development to build something that works perfectly well for a couple of customers or your testing environment but has horrendous consequences when it actually runs in production and can create a lot of trouble down the road.
Last week, I noticed that every now and then, my website got really slow, then recovered, then slowed down again. It still worked, but it had slowed to a point that I didn’t think was normal anymore. I looked at the server and saw that every couple of minutes, one of the processes serving the website would grab more and more memory until it was either stopped or impeded the function of other programs on the same server.
For a couple of days, I struggled just trying to keep the leak at bay. I set up a new server with more resources to see if this was just a resource-constrained problem, which it wasn’t. I checked my code and looked for potential reasons but really struggled.
Thanks to building in public and having many developers follow my journey, I got some great feedback when I posted about it on Twitter. I figured out the problem: I had built something that didn’t scale.
I had created a caching system to track how many podcasts I transcribe every day and hour, keeping the IDs of those podcasts in memory. This works fine on my home computer system, where I might have a couple dozen podcasts an hour. But I have a fleet of 30 servers that can transcribe up to 5,000 podcasts an hour. Suddenly, I had 5,000 IDs plus additional data in the cache per hour, and a daily cache with 100,000 items.
The real issue was that instead of just sending an update command, my system loaded every single item, added the new one, and then saved every item back to the database. This constant shuffling of 30+ megabytes of data per request, multiplied by 5-10 requests per second, created a memory bottleneck.
This was a problem I could never have seen in my local desktop environment. It only existed because of the resource requirements of my production system. I probably could have foreseen it had I spent more time thinking about how this would impact the system at large.
To solve this, I asked Claude, Anthropic’s AI, to rewrite my cache module in a memory-saving way. Since I was already using Redis as the backend caching system, Claude rebuilt it using Redis’s internal memory management. Instead of loading every single item of an array to add one to it, I now use Redis commands to add an item to an existing Redis array. This operation is much faster and involves almost zero memory usage in my application.
Now, I’m left wondering: Should I have built something else from the start? Should I just deal with the fact that this will happen over and over again? Am I building things that don’t scale until they break and then building something scalable, or should I always try to figure out every single possible contingency before deploying something to my product?
Here’s my answer: I will keep doing what I’m doing. I will build things that work for me and see how they impact my system. But I won’t shy away from deploying or releasing new features just because they might eventually put a resource constraint on my server. It’s a constant prototyping stage of my business anyway, on the way to undeniable product-market fit. I’d rather have introspection into my system than never release a feature for fear it might impact performance.
As a developer building larger systems but still with a lot to learn, I know I can’t foresee every contingency. But I can ensure that I keep an eye on resource usage. I’ve set up alerts, thresholds, and monitoring systems. Ironically, this involves the cache that actually slowed down the system, but now I’ve built a monitoring system for my monitoring system.
Instead of trying to build things that scale forever, I’m building a framework around my prototypes and features that tells me if things don’t scale. I’ll still try to build them as reliably as I can, thinking about the impact that any kind of data collection or operation might have at a different scale. What if this isn’t just 10 items in a list, but 10,000? Could I do it differently? Should I chunk it so it’s not all done at once?
The Podscan backend is a bit special because it doesn’t scale with the number of users but with the number of podcasts released worldwide every day. That floor is at least 30,000 podcast episodes a day, and I need to deal with numbers of that size today, not in some distant future.
So, doing things that don’t scale is wonderful for business operations, but building things that don’t scale is not ideal for technical implementations. Both for a system that can handle more load and for a business that can grow reliably, whatever you do that doesn’t scale needs to provide insight into how you could potentially make it scale.
The big learning from this experience is to keep an eye on significantly larger numbers and use technology that’s already built to facilitate this. I should have used Redis commands from the start instead of shuffling data around myself.
Build things at scale by building things that don’t scale, but put guardrails around them. Keep learning, keep improving, and always be ready to adapt when your non-scaling solutions hit their limits.
I'll share a few updates about my SaaS on the pod, and if you want to track your brand mentions on podcasts, please check out podscan.fm — and tell your friends!
Thank you for reading this week’s essay edition of The Bootstrapped Founder. Did you enjoy it? If so, please spread the word and share this issue on Twitter.
If you want to reach tens of thousands of creators, makers, and dreamers, you can apply to sponsor an episode of this newsletter. Or just reply to this email!