Jeremy Wright is getting some flack (37signals, but also a whole lot more on Memeorandum) for stating that Web 2.0 companies need to scale:
Listen up. If your company relies on the web to stay alive, you’d damn well better be using at least some of the following “ladder to high availabilityâ€:
Backups, Redundant, Failover, Cluster, Distributed, Grid and finally Mesh.
JMasterson over at 37signals makes a good point:
So. Don’t scale. Don’t worry about five 9’s or even two. Worry about getting something to a point where there’s reason to worry about it.
I want to throw my 2 cents into the jar.
If you’re building a system that has 99.999% uptime today, fault-tolerant data-center and all, big enough to make Wal-Mart turn green, one of two things are going to happen: either it doesn’t do much of anything useful, or some nimble startup will eat your lunch money by doing the same thing years before you get to ship. In either case, there’s no point in investing all that money.
But if you can’t make it scale when the time comes, then some nimble startup will eat your lunch money, or maybe one of the big guys. Imagine the point when you’re starting to see user growth, and users are starting to see dead pages and unresponsive servers. What you’ve just managed to do is teach a wide user base that there’s a useful service behind your obscure domain name, only to see them walk away to a similar service on someone else’s domain.
It happened to Technorati, the saving grace was the lack of any good alternative. It happened to Friendster, and we all know where they stand compared to MySpace. So go ahead and take the risk.
In software, your perfect design is never going to be ready. You take shortcuts, you optimize for certain loads, you focus on the killer features that affect you right here, right now. And you run with that base longer than you’d like, because you’re always busy adding more and more features. Which leaves you with two design options. The one you can fix, and the one you have to redo. The one you have to redo is someone else’s window of opportunity to outdo you.
Build to scale.
Assume change. Don’t worry too much about pre-mature abstractions, they make things harder. Don’t be afraid to hardcode, you’ll be surprised how easy it is to change. Take all the necessary shortcuts. But remember to start with a solid foundation, so you can fix and run. Choose the technologies that are easy today and scalable tomorrow. Design patterns that won’t require a full out rewrite.
If there’s one thing we’re not short of, it technology and experience in how to start small, and scale big. It’s been done before.