Introducing Navflow

As some of you may know we have been hinting at something new over the last couple of weeks. Not a new feature or a redesign but an entirely new app, and today I am happy to announce that the fruit of our labour is launching under the moniker of navflow.

With navflow we believe we have created a novel approach to testing how people navigate websites and applications. The traditional approach to running usability tests of this nature has mostly involved the testing of live interfaces. While this is a perfectly valid means of gaining valuable user insight, implementing design changes can be prohibitively costly and difficult to implement once a website or application is live.

Our solution was to move testing into the design phase and give designers tools that allow them to run user analysis using mock ups and wire frames. This approach has proven to be very successful on with almost a thousand people participating in and creating tests every day. For navflow we wanted to take design phase testing even further and let designers see how users navigate their designs without needing to build them first. To that effect we have built a testing platform that takes a series of images and creates a conversion funnel from it. The designer is responsible for highlighting areas of the image that result in successful conversions and we allow successful clicks to proceed along the funnel.

Navflow is currently in beta and open to registrations, however new accounts will need to be activated in batches in order for us to stay on top of any bugs or other issues that may arise with a new launch. Bear in mind that this is a beta and so may still be a little rough around the edges, we will be working hard over the coming weeks to get everything up to scratch. We are eager to hear your thoughts so please send any bug reports, feedback or even praise to

And so with that, happy testing!

Single file upload, using plupload

I'm pretty happy with plupload for all my multi-upload needs. One thing that seems to be missing "out of the box", is the ability to limit the number of queued files. It is possible, with a little event wrangling, to force plupload into allowing only a single upload.

The first thing you need to do is disable plupload's "multi_selection" setting.

var uploader = new plupload.Uploader({
runtimes: 'gears,flash,silverlight,browserplus,html5', multi_selection: false,

An important thing to note here. This doesn't prevent queuing of multiple files, it only prevents multiple SELECTION of files.  There are a few options to preventing queuing of multiple files. The one I've used below works for my needs, but there are other options.

uploader.bind('QueueChanged', function(up) {
if ( up.files.length > 0 && uploader.state != 2) {

Basically, this is a bit of a cheat. Whenever we select a file, we check that we're not already uploading and then automatically start the upload. At this point you could then choose to disable the uploader altogether, or in our case, display an icon for the uploaded file and give the user the option to replace it. I've seen other people use the event "FileAdded", but this event looks like it should really be called "BeforeFileAdded", as the newly added file doesn't appear to be added to the "files" collection until after the event has run.

It's worth noting that this method is really only appropriate if you're using your own custom upload interface. Obviously the queuechanged event is triggered when adding AND removing files. Not to mention that our little cheat doesn't actually clear the queue (although there is nothing stopping you doing that).

Using plupload with ASP.NET

I hate file uploaders. The simplest way is always the ugliest, and having anything "nice" requires days of backbreaking labour, and even then you're not even close to being sure it will work across all browsers. It's frustrating to do so much work and then have some users receive seemingly random errors for no apparent reason!

Matt pointed this neat file uploader out to me the other day, plupload. It is very simple and has about a half dozen "fallbacks" for supporting different technologies from Google Gears, Flash and HTML5.

For the most part implementing plupload is a matter of downloading the zip, unzipping and whacking in some sample code. But there IS a catch. Plupload uses BINAY STREAMING, not your bog standard multipart upload. So if you're hoping for a simple drag and drop replacement for your existing code, sorry to disappoint.  But it's not that hard to convert your existing app into a binary stream app!

Of course, there is an easy PHP implementation from the folks that wrote plupload, but I really struggled to find an ASP.NET one. So here's one I wrote based loosely on the PHP version! This is not complete by any stretch (you'd want to handle caching, I/O errors, folder creation etc), but this is the guts of it to save you some research.

Two main things you'll want to take note of.

1. Plupload supports chunks - this is a way to get around upload file size restrictions. Basically, the file can be chopped up and sent as multiple parts, and then put back together again when it arrives. If you look at the requests going to the server, you will see there are actually multiple requests PER FILE. This means we need to stitch the files together when we're done at the other end.

2. Data handling - plupload stores the file name, and chunk info into the querystring, and all the file data is in don't bother trying to work out why Request.Files is empty!

Here is the basics of what you need to implement plupload in! Easy as! If you're using any other file uploader, change it NOW!

int chunk = Request.QueryString["chunk"] != null ? int.Parse(Request.QueryString["chunk"]) : 0;

//open a file, if our chunk is 1 or more, we should be appending to an existing file, otherwise create a new file FileStream fs = new FileStream(Server.MapPath("/files/" + fileName), chunk == 0 ? FileMode.OpenOrCreate : FileMode.Append); //write our input stream to a buffer

Byte[] buffer = new Byte[Request.InputStream.Length]; Request.InputStream.Read(buffer, 0, buffer.Length);

string fileName = Request.QueryString["name"] != null ? Request.QueryString["name"] : ""//write the buffer to a file.

fs.Write(buffer, 0, buffer.Length);


Of course, if you're REALLY lazy, they've just recently updated plupload to support multipart uploads...but that's just boring!

Performance Issues and moving host

Hi all! It appears we may have outgrown our "budget" hosting solution. Over the last few weeks we've seen an anomalous increase in timeouts across the site without a corresponding increase in traffic. I'm pretty sure my programming hasn't gotten worse, and I'm pretty sure our users haven't gone feral on us, so the only remaining variable is the shared host we're on. I guess it's a classic case of "you get what you pay for". So we've decided to move house! I'd be more than happy to hear recommendations from the community for a .NET host supporting SQL2008. If you have any suggestions, drop us an email at or contact us on twitter! The downside of this move is that there may be some interruption to services while we move house. The upside is that when we're back up, things will be LIGHTNING fast....unless the problems ARE because of my which case I apologise in advance! Thanks for your support!


The Karma system is here at long last! It's taken a lot of thinking, a lot of coding, and a lot of tweaking to get this to a point where we're happy with it. Today is the day when I can finally say we're ready to release it! When having a look, please keep in mind that this is only phase one of our system. I can't go into detail as to what else we have planned, but hopefully this is enough to keep everyone happy until "later".

So what is this karma system? 

Well, in a nutshell, we're now rewarding users who do tests by giving them more responses. This sort of happened indirectly before, but now it is very clear and very defined that when you help the community you get rewarded. In the past if everyone did more tests, everyone got more results. With the changes to the payment structure, our "free" users got pretty well burned. We're really sorry about that. The karma system was supposed to happen at the same time, and for various reasons didn't. Well it's here now, and as an added bonus, <i>we're applying karma points retrospectively!</i>. That means if you've been doing a heap a tests for us over the past few months, you'll get all that karma in one lump sum! We even have one loyal user who'll find a stash of nearly 300 karma points!! It's the least we can do to say thanks for your support!

How does it work?

Put simply, you do one test, you earn one karma point. When you get 10 karma points, you can buy 10 more results for one of your tests. Easy! So this now means free tests get can get up to 15 results without you spending a single dollar. Of course, we'd still like you to spend a dollar, so upgrades can only be done once per test.

What's the catch?

There is no catch, but there are a couple of conditions. Firstly, you need to be registered. If you're not registered, you can't earn karma. Secondly, you can only earn 15 karma at a time. This gets reset every 12 hours. Basically, this is to stop folks just clicking through 60 tests without any real thought as to what they're doing collecting all the karma, creating some tests for free and then never coming back to the community. Besides, there's no point in you getting 10 extra results if all the results are garbage! We're far more interested in rewarding regular visitors who frequently contribute to the community. Of course, by doing more than 15 tests, you're still helping out the community, so by all means go nuts!

Mega Karma Chain Super Bonus

We have a couple of extra ways to earn <strong>bonus </strong>karma! If you consistently do 15 or more tests a day, you'll start to earn a Karma Chain Bonus! This bonus increments each day you keep the chain going up to a maximum of 15. This means you could actually earn 30 karma a day! That's enough to upgrade 3 tests! The trick is to come back at least 5 times in a week and do 15 tests, that's when you'll start seeing the chain bonus. 

Community moderation

On top of that, you have the ability to give other users bonus karma directly! In your test results page you now have the ability to reward users by clicking the little green thumbs up. This gives them an extra karma point for that excellent feedback! Of course, if someone submits a garbage response you can always deduct a karma point! Deleting someone's response will quickly have their karma going backwards, so play nice!


We'd love to hear what you think of our new karma system, so please either leave a comment below, send us an email or follow us on twitter. There will be many more features surrounding this including a top 10 list, "credit for karma" and some other things I can't even tell you yet!

Even BETTER value for paid tests

As part of our upcoming Karma system, we've been collecting a bit more information about who is doing your tests. One bit of information we're collecting is referrals. Up until now if you bought a gold test and got 45 people doing your test, it wouldn't matter whether those people were "random" testers or friends of yours that you had sent a link to. This meant that if you sent your link to 26 people you'd essentially be getting 26 less "random" tests. Rather than discouraging this, we now ENCOURAGE it! If someone does your test having come from a link you sent out, we no longer deduct that from your "expected" results. What this means is that if you buy a gold test AND send your link out to 10 people, you can expect to get 55 results instead of 45! By sending your link out, not only are you getting more results for your money, but you're also helping other test owners get more results as well! Everyone wins! Another minor update: We're in the process of ditching "tokens". They'll still be used in the background, but we're reverting to $ credit. Basically 1 token is worth $5. Everything will remain the same, it's just that it will hopefully be less complicated than trying to work out how many tokens you need, how much the cost etc etc.

Questions, Answers and the "Karma" system

We've had a couple of people contact us regarding the changes to Not complaints, just suggestions. One user (Christopher) made a great response to my last post. I started to reply but then decided I'd just make a new post instead, as it is well worth the discussion!

You may or may not have noticed the last line of my original post, but it made reference to a "Karma" system. It is pretty much what Christopher described and it IS coming, we just haven't released the exact details yet!

A couple of suggestions have been made, which I'd like to respond to here. 

1. "Why not just set a cap on results to a certain number"

We can't set "hard" caps on tests and responses. It may not be obvious, but the number of responses a test get varies with site traffic. Basically, the problem is that if we have a slow period and we have a hard cap we will end up with tests taking days and days to get all their results, or worse we may end up with no tests "waiting" in the system on a particularly busy day. We can't afford to have people going to go do a random test and be told there are none to do!

In actual fact, when are traffic spikes it is often either just people DOING tests or just people CREATING tests and this makes coming up with a "cap" nigh on impossible. The number of "expected" results per test can vary from 9 responses all the way up to 20 or more (with premium tests being a multiple of that). 

Now, what used to happen was that paid tests "expected" results were based on the average usage for the week. This means that variations to that number would occur slowly over time. When we saw a spike in traffic, rather than paid users reaping the benefits, free tests were "absorbing" that extra traffic (because when there are no paid tests awaiting results, we just grab the lowest free tests). We'd get free tests getting 20 or more results and bronze test (worked out on the weekly average) only getting 22 or 23. 

What we're now doing is making sure paid tests are absorbing that extra traffic, rather than free tests....hence why free tests are now fixed at 5 responses. When we get a spike in traffic, those extra results go to people have handed over their hard-earned cash. Of course, we still need to smooth the numbers a bit so that we have a reasonably consistent number from day to day, so to a degree free tests still absorb some of the traffic spike (at the moment most free tests are getting 7 or 8 results)....just not as much as they used to.

2. "Why don't you reward people for doing tests?"

The karma system (which IS coming soon) will work pretty much how Christopher described it (i.e do X tests and get a free token). But it's not as easy at it seems. At the moment tests have very little "noise" in their results. People are doing tests because they want to help someone else out. By relying on the good nature of people, we ensure that test results are good quality. People are filling out tests because they want to help, not because there is a reward for doing so.

The concern is that if we start giving away tokens for responses, we will be encouraging "noise" (i.e. people doing tests purely to get credits, not to be helpful). So what need to do (for memory tests) is allow test owners to vet their responses. They will be able to discard worthless responses and "reward" very useful responses. The result isn't then just about how many tests you did, but also how useful your comments are.  The problem here, however, is validating "useful" clicks in the click test. We are yet to come up with a good way to reward "good" clicks and deter "bad" do you even tell the difference? That is the real stumbling block to the Karma system.

3. "can I just buy a subscription?"

We'd love for you to give us more money! But seriously, we're not sure how a subscription results in better value for our users. Subscription models are great for the vendor, but I'm not convinced they're better for the end user. The big upside for users, of course, is that they don't have to deal with tokens (which we admit can be a little confusing!). The downside though is that having a certain number of tests available per work or month isn't really how we're seeing people work. I don't ever want a situation where someone has bought 4 tests per month, and all of a sudden needs a 5th, but that requires upgrading to the next plan. 

For me personally, the pay for what you use method is much better value for money. The vast majority of our sales are from people who buy tokens in bulk, and use them when they need them. If you need more, buy some more. We've just now included larger token packs to make this a whole lot easier to do this. Of course, if people keep nagging for subscriptions...we will add them! So keep your feedback coming!

4. Last but not least, "can I just see English/French/Chinese/German tests".

Another tricksie problem! There are two ways to look at this problem. Test owners don't gain much value from people doing their test if they can't read the writing in it, and similarly users can get frustrated doing tests in other languages. The trick here is marrying the two together. We want to give the test owner the ability to exclude anyone who doesn't "accept" that language, but we also want end users to be able skip all tests that don't match their language choice. Doesn't sound hard does it?

The problem arises in that most of our test "doers" are anonymous. Beyond checking their browser language we don't know what they do or don't understand. We also don't want to trouble users while registering with too many demographic questions. So the solution at this point is to do a little of both. We'll be adding the ability for registered users to submit their demographic information (if they choose!), and we'll be adding the ability for test owners to exclude all but certain groups from doing their tests. We'll also then have the ability for test owners to filter on that demographic information. We then just need to sort out the anonymous users....

Well, I hope that answers some of the questions floating around! We are still actively making changes, and so your feedback definitely DOES help us decide what to do next. We're both pretty busy on other projects, so changes don't happen quite as quickly as we'd like, but they ARE coming.

Thanks again for all your feedback and support!

A dollar more, but more results for you!

Hi All! We've just made a couple of minor changes in preparation for some larger changes coming up in a few weeks time. The first, is a bit of good and bad. The good news is that paid tests will now get more results than they've been getting! Hooray! The bad news is that it is at the expense of free tests..... Now, now...let me explain! Previously we worked out who got how many results based on the number of tests being created, and the number of responses being made. We simply divide one by the other. Free tests are worth 1, a bronze test is worth 2, silver - 3 and gold - 4. Add up all the responses in the last week, and divide by the number of tests (including the weighted values for premium tests)...that's the number of "expected results". Simply put Bronze tests get double the results of free tests, silver gets three times and gold gets four times. Easy. With the increase in traffic to the site in the past few months, we've had a matching increase in responses to tests. Originally, we'd planned on the basic free test getting 5-6 results, the first level of premium tests getting about 12-15 results and so on. What has happened, however, is that free tests are now getting 12-15 results! In short, free users are reaping the rewards above our loyal paying customers. So, it pains us to do it, but we're capping free tests at 5 responses. We've spoken to a lot of people, and we feel that 5 responses is still enough to give a reasonable indication of a trend in testing. The upside of this is that the results that free users were soaking up will now be redirected out to paid users. Should you like more results at any stage, you can now upgrade an existing test to get more results. Of course, as some of you may have noticed, we've been trialling a "bonus" upgrade to some free users. We'll be ramping this up and upgrading more free tests (for registered users only) to "Gold" tests. Hopefully this will make up for those that feel they're getting a raw deal! If you haven't received one of these yet, hopefully it will be your turn soon. Keep in mind you have to be a registered user to be a winner! The other change we've made is to pricing. Single tokens have gone up $1 to be a nice even $5. Buying in bulk still gives you a discount, but we've now added a 25 and 50 token pack due to popular demand. The cheapest per token price is still $2.50, and this is available when buying 50 tokens. We hope that the increase in test results you will now be receiving makes up for this small increase. As always, we value your feedback, and we appreciate your support! Stay tuned for more changes coming soon, including the long overdue Karma system.

New test options, huzzah!

First up, we want to say a big thank you to all those that have bought premium tests already! We really appreciate your support. The response has been great, and as a result we've rolled out a new set of upgrade options. We now have four levels of tests. 1. FREE - Free tests are still free, and will always remain free. We're not taking anything away from you. 2. Bronze -These up until now were called premium tests. Bronze tests reward you with double the results, and get you your results much faster than free tests. 3. Silver - Silver tests get even more results! They're also prioritised higher than Bronze and Free tests. Most importantly, however, you get to give your test some context by adding a custom question to the start of your test. E.g. you may ask, "Where would you go to sign up?", or "What are today's specials?". This enables you to get a much more focused response from users. 4. Gold - Gold tests get everything Silver tests get, but more of it! In addition, Gold tests can ask an additional question AFTER the user has completed the test. You also have the option to show the test image again (for an unlimited amount of time) to gain more detailed feedback on your design, or choose to hide the image to really test how much your users remember from the five second test! We look forward to seeing how all of you these new tools. Whether it be just to give your test some context, or to asking the user to perform a specific task, we hope these additions will make five second tests that much more useful. As always, we'd love to hear your feedback on these changes, and we look forward to bringing you more soon!