There is a very good ongoing series on O’Reilly Radar called Database War Stories. Or in other words, how various people with a lot of users and page views address their database scalabitity issues. It ranges from using no database at all and using flat files, to using a cluster of databases (with one master, and several slaves).
If you ever hope to see your application reach the stratosphere, it is worth a read. Scalability is something to keep in mind, but please, don’t over engineer your application from the get go. Focus on features and usability first, and make it more scalable when you need to. Knowing what it takes might help make your life easier when you get there.
The first 5 posts include stories about:
- craigslist. 1 master and 12 replicating slaves for the classifieds,
- NASA World Wind
- Flickr. A terabyte and growing!
- bloglines and memeorandum,
- Second Life
Of course the, database is only one dimension of scalability, but if you don’t get that one right, you might as well go home.