Thursday, 23 April 2009

Usability: A second class citizen?

I’m sure we’ve all been in the situation where you get an opportunity to test a product so you sit down and you start testing for functionality, you run some system tests, you run some security tests and maybe a performance test or two before you then run some usability tests. Your defects are raised, most of the functional defects get fixed, the big security and performance issues are fixed but then you look at your usability defects are they’re set as “Fix in the Future” or “Not a problem” or “Proposing not to fix” – Argh! Why is this?

Usually, as we discussed in our team meeting this morning, it’s for two reasons:
1. You’re testing for usability too late in the cycle
2. Time pressures mean the focus is on making the product functional, not usable

Quite often it seems that, what appears to be a small usability issue, actually requires a significant (> 1hour) amount of time to fix because the fix requires some re-working of how the UI works, or it would have a significant impact on other functionality – which tells us we’re testing for usability too late in the software development lifecycle.

Our other battle is members of the project who lack empathy with the end user. “Why on earth would we want to hold up a project to fix these paltry 10 usability defects?” (I recognise that sometimes it’s absolutely the right decision to release with those 10 defects unfixed). So we sit in defect reviews and watch as our usability defects get set to the statuses shown above, usually because we’ve failed to articulate the real end-user impact of those issues.

So, what are we doing about it?

Well, we’re doing a few things about this. We’re trying to use the power of the checklist to create a common set of usability heuristics which can be run against a design doc, a prototype, an early version of the product or a final version of the product. And as that implies, we also recognise that we need to test for usability earlier in the development of the product. It’s clear that we’re testing for usability across most of our products but generally we’re leaving it until too late in the day to do this so it’s key we test for usability as early as we can in the development of a product and continue to focus on usability throughout its creation. Finally, we recognise that we need to get better at influencing our project team on the impact of these defects. We need to be better at understanding our customers and also understanding those people within the business who also understand those customers and who can therefore help us articulate why these issues are important.

Approximately 50-60% of calls our support teams take are “How to” calls which suggests usability of our products is an area in need of improvement so hopefully some of the measures we’re putting in place will start to reduce this in future product releases.

Very interested in hearing other’s experiences with usability testing – have you seen the same problems? If so, what are you doing about it?

1 comment:

Rob Lambert said...

Really cool post Simon. I've been studying usability and sitting courses in it for some time now and the only real way to solve the usability issues in my opinion, is to start running your projects in an iterative way.

I've done the design up front ways and it doesn't work. Things change or the design is wrong or it gets implemented badly.

I've done the post implementation usability review and you're right - this takes too much time.

But in the last year or so I've been on agile projects where the UI is considered and implemented each sprint. At the end of each sprint you show the customer. The customer feeds back, the tester feeds back and all of this information is reviewed.

the outcomes are worked on in the next sprint (if prioritised) etc etc.

By the end of the project the usability, accessibility and general look and feel of the products is pretty good, cos you've spent weeks/months/years tuning it and tweaking it.

There's a good article here on pretty much the same concept: