Comments on: Vanity: Experiment Driven Development for Rails

By: How we do A/B testing in Rails « Amit Kumar Mondol's Blog

How we do A/B testing in Rails « Amit Kumar Mondol's Blog — Thu, 01 Apr 2010 03:53:30 +0000

[...] service." model Account.in_trial end Here I have only described some brief concept Read more: http://labnotes.org/2009/11/19/vanity-experiment-driven-development-for-rails/#ixzz0jZMRkbJi Posted in Web Development. Tags: A/B Testing. Leave a Comment [...]

By: almost effortless » Weekly Digest, 12-11-09

almost effortless » Weekly Digest, 12-11-09 — Tue, 30 Mar 2010 15:48:48 +0000

[...] Vanity: Experiment Driven Development for Rails You’ve got your TDD, your BDD, your load testing, your user testing, your code metrics. All tools for testing your code and improving it. Here’s a question for you: what are you using to test your ideas? [...]

By: Arno.Nyhm

Arno.Nyhm — Sat, 20 Feb 2010 22:51:59 +0000

interesting.

how i can see this extended statistics:

a) how many signup (rate)

b) how many signup (rate) with 3 fields (username, age, zip)
– b.1 fill in field username
– b.2 fill in field age
– b.3 fill in field zip

or how i can setup maybe some more subgoals?

By: Jesse Crouch

Jesse Crouch — Tue, 16 Feb 2010 20:05:11 +0000

If you’re on Windows and trying to use Vanity – maybe give up. Right now it requires Redis which doesn’t easily compile on win. Read a few posts about getting redis running with cygwin sucessfully, but quite a few more that had issues. Some people compiling it with visual studio too, but also some had issues.

Just thought I’d throw that in in case anyone is having difficulties.

Also would love to see a config option to use something besides Redis.

By: Henrik Joreteg

Henrik Joreteg — Tue, 26 Jan 2010 16:47:32 +0000

Absolutely brilliant! There’s no doubt in my mind you’ll soon see ports to django and other frameworks.

Design by committee just doesn’t work. Solid, data-driven decisions are what matter. But, doing good A/B testing has to be simple or it just won’t be done. Very, very cool. Thanks for this.

By: Avishai

Avishai — Thu, 07 Jan 2010 11:27:52 +0000

Looks quite interesting, can’t wait to give it a try in my next app!

By: Assaf

Assaf — Sat, 26 Dec 2009 21:13:11 +0000

That article starts out by taking “release early”, and slippery sloping it into a “release crap” strawman. It attacks design-by-consensus, but either doesn’t mention or doesn’t understand customer development. Strawman are easy to build, easy to defeat, great link bait, but add nothing to the conversation.

Reasonable people start out with good ideas they want to turn into quality products that people will actually want to use. Good ideas still need tuning, few ever get it right the first time. How do you tune? You can do it based on opinions, or you can do it based on facts.

EDD is about tuning your software to be better based on facts.

By: Asaf Atreides

Asaf Atreides — Sat, 26 Dec 2009 18:06:58 +0000

Hi,

How do you think EDD (and perhaps the entire Lean Startup methodology) deals with the arguments presented in this article ?(http://onstartups.com/tabid/3339/bid/11416/Releasing-Early-Is-Not-Always-Good-Heresy.aspx)

And a quick idea – A/B testing should be somehow incorporated into a revision control system, having the system automatically calculate the “best source code paths” for a given scenario/user.

Thank you for the lib and for the blog,

Regards,

Asaf.

By: Eliot Sykes

Eliot Sykes — Thu, 10 Dec 2009 17:38:51 +0000

Clean looking library Assaf, looking forward to trying it out with a Spree install.

By: Assaf

Assaf — Tue, 08 Dec 2009 17:27:30 +0000

You start with an idea for a change that will improve your software. Your baseline cost of development is having both alternatives — before and after the change. Without EDD these alternatives will be separate in time, with EDD you’re going to have some overlap (the duration of the experiment).

For the experiment, you only need a skeletal implementation, you’re not committed to fully developing the feature until after it proves itself. For small changes it makes little difference, the cost is the same.

For complex changes, you can save a lot by not fully developing features that don’t matter. You’re going to know whether a feature matters or not quickly enough, and with data to back up, that you can make the decision to *not* develop it further.

You can also kill unused features early. So these are two ways to reduce development costs using EDD.