usability testing - 5-second tests vs. the short-term memory

I've recently been participating in random tests on http://usabilityhub.com and I find myself cringing when the 5-second tests come up and feeling guilty at my seemingly useless responses once I'm finished. I frequently can't remember the details when it comes to question time so I'm afraid my responses are not helpful to people.

I can have an opinion or overall impression within 5 seconds, but remembering multiple, specific details is another matter entirely.

In a perfect world, if the UI were clear and obvious enough, I suppose I would not have issues remembering the important things. It's also a lot to take in in 5 seconds and the questions span a large range of topics including logo, colors, site purpose, call-to-action identification, etc. If they want to know if I absorbed what services they offer or what their name was and I could only focus on the gigantic stock photo or the pretty colors, I am having a difficult time seeing those responses of mine being beneficial to someone.

Is this the whole point of the 5-second test or am I a terrible tester? Or, is this a case of people using the 5 second test instead of the click test because they want the free-text answers?

Answer

This is a great question. I believe the only purpose the 5 second tests serve to see is where is the immediate focus of the users and what their next steps are after the first impression.

They would be extremely ineffective if you have a site where there are a lot of images or a lot of content but they would be effective when you have a site which has a definite agenda or is trying to convey a message

In the above example the video icon is the prominent figure in the image and hence drives the user to potentially watch the video and hopefully download dropbox

Below Firefox's download image dominates the page with its color focus and size and hence its something most users should notice almost immediately

To quote what this article "5-Second Tests: Measuring Your Site's Content Pages" in the User interface Engineering site has to say on the subject with regards to benefits:

Limiting the viewing time to 5 seconds, we get a valuable glimpse into what happens during the first moments a user sees a page. When we give users more than 5 seconds to study the page, we've found they start looking at the page more like a designer, noticing details they would normally miss or misinterpret.

Because this technique is quick and easy to implement, it is perfect to run in locations where we can gather many users at one time, such as trade shows, conferences, and the company cafeteria. We can gather large amounts of user data in a short time.

With regards to disadvantages of the test,quoting them :

We've found the technique is best when we use it on pages designed with a single primary purpose. Home pages and major navigation pages don't yield as valuable results, because they often serve many different tasks.

For example, the home page for RedCross.org serves the needs of donors, sponsors, volunteers, medical professionals, victims, and the press, each with their own set of tasks. Each of these different users would probably see different things on the page, depending on their context and immediate goals. Other techniques, such as traditional usability tests and inherent value tests would be better instruments for judging the effectiveness of this page.