Here is another 2 variations on my GSB homepage.

New version

Old Version

The new version has some extra content at the bottom, I measured how much of a boost that gave to the percentage of people grabbing the demo. It is a small, but noticeable difference.

GWO says its within error margins, but over 10 days it’s consistantly outperformed, so I consider it worth switching to. Say GSB earns $50,000 over a year, 2.28% improvement is $1,140, for changing some html. Now you see why I do this stuff :D Imagine how much a change to amazons home page must earn them…

10 Responses to “Website Experiment #2”

  1. Danik says:

    Pretty amazing how a small change like that pays off. All you’ve done is add a small tutorial and another download button. Shows how important attention to detail is.
    Btw: The text in the first column under “In the press” is formatted differently than the rest. I noticed when tabbing back and forth between the old and new versions.

  2. Draxis says:

    Thanks for sharing this kind of information, its really interesting stuff! Also, http://www.positech.co.uk/gratuitousspacebattles/index.html has a spelling error, ‘issue each shop with orders’

  3. Muesli says:

    Well, the new version has two download links whereas the old version had only one link that is a bit lost in the content. I guess that’s what really makes the difference…? Check what button actually is used by the users in the new version :) . I’d also make the demo link in the top navbar more visible.

  4. Schlabbi says:

    You’ll have even better results if your button is red not blue. Just try it…

  5. Anthony says:

    @Schlabbi,

    I’d also be interested in the results if you used a ‘warm’ color (red or orange, say) instead of a cool color for the button.

  6. Fraser says:

    @Schlabbi might be right:

    http://blog.performable.com/631526233/

    Worth an A/B on if you can get 21% clicks…

  7. cliffski says:

    Well I guess I better try that then :D

  8. nana says:

    >Well I guess I better try that then

    Yes, I was going to make the same comment. The first download button is lost in the UI. A different color (or whatever) making it standing out a bit, will be better.

  9. James says:

    I used to do these types of tests until I learned that they are usually worthless since they are not statistically significant. Meaning if you do the same test a month from now the results could be opposite, the change is not reproducible.

    For your test:
    N=2754
    D=16

    D^2 !> N so it is not significant, not even close.