1. Sep 2nd, 2007

    CouchDB: Thinking beyond the RDBMS

    A flat document storage engine calling itself a database? On any other day, that’s what you’ll get from a collaboration between a vendor and PR agency, pushing new ways to achieve synergy and optimize the bottom line to Global 5000 companies.

    This day, it happens to be CouchDB. And CouchDB on first look seems like the future of database without the weight that is SQL and write consistency.

    It stores document in a flat space.

    There are no schemas. But you do store (and retrieve) JSON objects. Cool kids rejoice.

    And all this happens using Real REST (you know, the one with PUT, DELETE and no envelopes to hide stuff), so it doesn’t matter that CouchDB is implemented in Erlang. (In fact, Erlang is a feature)

    Here’s where it gets interesting. There are no indexes. So your first option is knowing the name of the document you want to retrieve. The second is referencing it from another document. And remember, it’s JSON in/JSON out, with REST access all around, so relative URLs and you’re fine.

    But that still doesn’t explain the lack of indexes. CouchDB has something better. It calls them views, but in fact those are computed tables. Computed using JavaScript. So you feed (reminder: JSON over REST) it a set of functions, and you get a set of queries for computed results coming out of these functions.

    Here’s the kicker. This simple architecture you can partition and replicate any way you want, map/reduce these computed tables on any scale, and deal with the rest on the client.

    And if you’re using a sane person’s programming language, it’s a trivial matter.

    I’m personally convinced that write consistency is the reason RDBMS are imploding under their own weight. Features like referential integrity, constraints and atomic updates are really important in the client-server world, but irrelevant in a world of services.

    You can do all of that in the service. And you can do better if you replace write consistency with read consistency, making allowances for asynchronous updates, and using functional programming in your code instead of delegating to SQL.

    It’s an incredibly hard problem to solve in languages like C or Java, but we’re past that stage, and it’s time to start looking beyond SQL.

    So CouchDB is still an alpha, but a serious project to look at.

    Ruby bindings here, and another take on CouchDB from Tobi.

    Update: For more context: Read Consistency: Dumb Databases, Smart Services

    1. Sep 3rd, 2007

      Jan

      Cheers, that’s one fabulous summary of CouchDb’s features. Thanks!

    2. Sep 3rd, 2007

      Russell

      Actually, I’m a little disappointed in the briefness of the CouchDb summary. I’m not a big fan of RDBMSs either, especially since so many keep some of their business logic in SQL (via stored procedures) and some in the service code (C#, Java, whatever). Ughh… The prospect of using various functional programming paradigms is very attractive.

      But it isn’t clear from what I’ve read that CouchDB provides the functionality that is commonly included with your average RDBMS, particularly the extensive join capability, cascading deletes, etc. SQL is ugly, but it is very expressive in its domain. I’d sure like to see an example of how CouchDB clients handle these questions. I’d be real reluctant to re-implement an SQL-like interpreter in Java, C#, Lisp or Scheme without good reason.

    3. Sep 3rd, 2007

      Assaf

      I think the future is running joins on the client. Not to be confused with indexing and filtering which are horribly expensive on the client in terms of performance. Joining on the client gives you partitioning across data sources, you can pull data from all over the place, local and remote services, etc And the possibilities, when you run the queries outside the database are endless.

      With that in mind, the idea that a database server provides query capabilities is wrong. You should be able to pick a generic library that works best for your environment, and use it with any number of data sources.

    4. Sep 4th, 2007

      Bill de hOra

      ZODB. Seriously.

    5. Sep 4th, 2007

      links for 2007-09-04 « Treat with Jermolene

      [...] Labnotes » CouchDB: Thinking beyond the RDBMS Interesting Erlang-based database technology (tags: databases data webdev REST javascript json) [...]

    6. Sep 4th, 2007

      Sterling Camden

      Thanks, Assaf. After reading all that Damien had to say about CouchDB for months, you’ve finally made it clear to me.

    7. Sep 4th, 2007

      All in a days work…

      [...] CouchDB: Thinking beyond the RDBMS the future of dbs without SQL’s weight, stores in a flat space, no schemas, store (and retrieve) JSON objects, no indexes (uses views instead which are computed tables), use relative URLs, you’ll do better if you replace write consistency with read… (tags: Databases RDBMS JSON REST Erlang Scalability WebServices Functional_Programming) [...]

    8. Sep 4th, 2007

      tecosystems » links for 2007-09-05

      [...] Labnotes » CouchDB: Thinking beyond the RDBMS best summary of the CouchDB project i’ve seen (tags: couchdb map-reduce erlang views non-relational indices writeconsistency) [...]

    9. Sep 8th, 2007

      items for 09.08.2007 « Tzetze Fly

      [...] Labnotes � CouchDB: Thinking beyond the RDBMS SQL databases being replaced by scalable REST web services in the future? (tags: database scaling) [...]

    10. Sep 14th, 2007

      CouchDB – doucument centric ODS « Gobán Saor

      [...] block, the document-oriented database, is less so. One such DBMS,CouchDb, is the latest wunderkid to bubble to the surface, helped by the database’s RESTful inteface , its abandonment of XML in favour of JSON, the [...]

    11. Sep 16th, 2007

      mkaush

      Seems like Notes dB. A notes database is very good for document type storage and this dB seems to have very similar concepts (replication, document id, etc.)

    12. Sep 16th, 2007

      links for 2007-09-17 » SDLC Blog

      [...] Labnotes » CouchDB: Thinking beyond the RDBMS CouchDB summary (tags: couchdb rdbms database data json Design) [...]

    13. Jan 21st, 2008

      Luka Manser

      As a database developer in MSSQL I have to say that this look very interesting. I’m looking forward to see it in action. I’m just installing it on my laptop.

    14. May 9th, 2008

      Andrew B

      Can you say Lotus Domino? I find it hilarious that this is seen as such a hot thing when its basically reimplementation of what Lotus Domino has been doing since the 80s. Say Lotus Domino and people laugh and think its a toy but this is somehow revolutionary and progressive? Don’t get me wrong I think its great, as is Lotus Domino for specific purposes.

    15. May 17th, 2008

      Perspectives

      Erlang and High-Sale System Software…

    16. May 18th, 2008

      Perspectives

      Erlang and High-Scale System Software…

    Your comment, here ⇓