Comments on: Read Consistency: Dumb Databases, Smart Services

By: Eric

Eric — Wed, 29 Oct 2008 13:07:11 +0000

Very nice Assaf! Thanks.

By: MikeD

MikeD — Thu, 01 May 2008 00:40:33 +0000

“Goes without mention that I won’t even dream of duplicating product details inside the order, or listing orders inside a product record.”
You at least need to record the version of the product purchased (not to mention the seller, etc.) in case someone changes the title later.
Which leads into why a storage service that natively supports versions is pretty useful.

By: Scaling Databases and Google’s App Engine at robubu

Scaling Databases and Google’s App Engine at robubu — Mon, 14 Apr 2008 03:42:10 +0000

[...] Assaf Arkin: if anything I wrote sounds vaguely familiar because you somehow managed to dumb your RDBMS into storing structured data in BLOBs, added versions and timestamps on all records, grappled with minimizing transactions and locks, denormalized data like there’s no tomorrow, or relied too much on a message queue, then time to rethink. Are you using a hammer to polish your china? (Tip: not a good idea, invest in soft cloth) [...]

By: Internet-Scale Services « Todor Ivanov’s Weblog

Internet-Scale Services « Todor Ivanov’s Weblog — Sat, 05 Apr 2008 19:24:56 +0000

[...] interesting post following the database topic I found in Labnotes blog is Read Consistency: Dumb Databases, Smart Services – the article is a bit longer but there are good points reminding us what is important in DB world. [...]

By: Internet-Scale Services « Todor Ivanov’s Weblog

Internet-Scale Services « Todor Ivanov’s Weblog — Wed, 26 Mar 2008 12:23:06 +0000

By: Oleg Andreev

Oleg Andreev — Tue, 25 Mar 2008 00:36:43 +0000

Great post, Assaf!

StrokeDB is an attempt to address all the issues, described in this post. Check it out on strokedb.com

By: Giles Bowkett

Giles Bowkett — Thu, 24 Jan 2008 15:10:20 +0000

on structured search, find and listen to Clay Shirky’s talk “Ontology Is Overrated” on IT Conversations dot com. the difference between structured search interfaces and simple type it-see it search bars was the difference between Yahoo’s loss of the number one spot and Google’s ability to take it and keep it.

By: Assaf

Assaf — Mon, 14 Jan 2008 07:29:21 +0000

lubosh, there’s a delay between the app adding the record (or changing an existing one), and all the database nodes reaching consensus to show the new record (or the update).

The update feed can (and should) deal with that consensus by ordering records as hey become available, so B will show up followed by A. Remember, the delay is not measured in hours, but in milliseconds, a simple consensus algorithm can take care of that.

By: lubosh

lubosh — Mon, 14 Jan 2008 04:07:20 +0000

Hi Assaf, very interesting read but update feeds are more complicated problem. some changes might take longer to become fully available to all nodes, so your client application might miss on those.

for example,
- record A gets created
- record B gets created
- record B will get available to all nodes
my client application will pull updates (only record B) and remembers time of last update
- record A will get available to all nodes

in this case record A will never get noticed because of its earlier timestamp.

how would you deal with this problem? perhaps only message queues so we don’t depend on time unit.

By: Plasmoid

Plasmoid — Tue, 27 Nov 2007 04:37:20 +0000

Well, that’s a nice pool of random ideas.
Unfortunately you gloss over hard problems as if they were trivially solved…
Distributed Indexes, “Partitioning is something that happens, not something you have to work hard for” etc.

What would really help to get an idea of what you’re after
would be some examples with pseudocode.

What exactly happens on INSERT, UPDATE, DELETE, SELECT – what is read/written, and from/to where?
What happens when one node in your Cluster fails? (n-copies, or what?)
How exactly does the partitioning “just happen”?
How does the distributed index work?
How do aggregate queries work?

In some paragraphs you make it sound like you know some magic bullet, something
in between a stream oriented db and a distributed rdbms, in other paragraphs
you just sound like you have no idea what you’re talking about at all…