Expert panels & user trials
Which? employs a number of leading experts, but sometimes it's important to have a number of external experts consulting on a project, or speak to lay people to truly understand a product or service's broader implications.
We will always endeavour to ensure we have the right expertise informing our work. If that can't be provided in-house, then we look externally to form expert panels to help us design our methods and establish our criteria for assessment.
For example, in our funerals panel we brought together a head of trading standards who runs the coroners office, an experienced funeral director and an expert in bereavement issues. They each brought expertise that the others didn't have, and this allowed them to shift positions and reach a much better-informed and robust consensus view that we could defend.
Choosing the right expert and avoiding conflicts of interest
If we are going to be critical of practice, it’s important to think carefully about who we will need to help. For example, we may want a Trading Standards Officer to comment on selling practice, an industry expert to evaluate practice, and an advertising expert to be credible in their analysis of marketing material. And it's always essential that we check whether there could be any conflicts of interest in advance.
When Which? tests products we want to make sure that their most important features work well, that they are up to the job and that they will suit a wide range of people - from experienced users to first-time buyers. It's not always possible to create meaningful tests of these things under laboratory conditions - for instance while we can measure the airflow speed from a hairdryer, there's no such thing as the perfect speed for drying hair - some people want one that blasts their hair dry as quickly as possible, whereas some need a more gentle airflow to create their perfect style.
The way that Which? gets round these problems is to run user trials throughout the year - giving people products to try and asking them for their feedback.
Depending on the product we either ask people to use them under controlled circumstances - such as taking pushchairs around a standard obstacle course - or give them to people to try in their own home, as we do when we ask men to test razors for us. Which? buys all of the products we use in our user trials, costing us thousands of pounds each year.
Analysing the data
Using our insights into how people use specific products and what makes a good or bad one, we create standardised questionnaires and ask our users to rate the products. Questionnaires explore aspects including how easy it is to use the product, whether it does a good job, particularly good or poor features and the value of any unique selling points.
Using large numbers of people in our trials helps us to get enough ratings and comments about each product to be sure that we are making robust conclusions from our research - so it's not influenced by the previous experience or prejudices of the people involved in the trial.
Ready to become a Which? member?
If you'd like to become a Which? member, you can take out a trial £1 subscription. Not only will you receive access to all our reviews online, but you'll also receive our monthly magazine containing reviews, investigations and advice. Plus, you'll be able to use our personal Legal and Money advice lines, and use Which? Local to read about services recommended by other Which? members living in your area.
Discover the benefits, and try Which? for £1.