How we test
How we test mpg and emissions
By Adrian Porter
Article 2 of 2
Want to know your car’s real mpg? The true range of your electric car? We reveal why official figures can’t always be trusted.
Cars with the best mpg offer the greatest savings on fuel. Which? research has discovered that the difference between the most and least fuel-efficient medium-sized hatchback is more than £800 a year – enough money for a holiday.
But the key to saving fuel is getting to the real mpg figures, as our tests show the advertised figures can’t usually be trusted.
Which? car tests discover not only a car’s true mpg, but also how reliable it is, the usable boot and passenger space, and how easy it is to drive. Find a brilliant new or used car – we reveal the best cars.
Claimed mpg figures explained
Car manufacturers are required to publish fuel economy figures (usually expressed as miles per gallon, or ‘mpg’), along with exhaust emissions data (air pollution), each time they launch a new car. These are the figures you tend to see in adverts and brochures.
And up until 1 September 2017, these figures all come from the previous European Commission test protocol, the NEDC (New European Driving Cycle) which has been widely criticised as easy to dupe, open to loopholes and not being challenging for modern cars.
The same test cycle was also used to derive how much air pollution, such as NOx (oxides of nitrogen) and particulate matter, cars create while driving. You can read more about the different emissions and their effects, as well as find out which cars are the dirtiest, in our air pollution guide guide.
Fuel economy figures are split into three different categories:
- Urban represents towns and city driving.
- Extra urban refers to driving outside of towns and cities.
- Combined merges urban and extra urban together to give an overall average. This is the figure you’ll most likely see in adverts and promotions.
New official mpg and emission tests from September 2017
With support from our own ‘Come clean on fuel claims’ campaign, brand new generations of cars released after 1 September 2017 (and those that undergo significant facelifts), will be subject to the Worldwide-Harmonised Light vehicles Test Procedure (WLTP). All remaining new cars have to go through the WLTP as of 1 September 2018.
This is a positive step forward, as its a more challenging and realistic test, and similar to one that Which? has been using for years as part of our independent research into fuel economy and emissions.
The toughness of the test should help produce more realistic fuel economy figures. Also, according to the NGO, Transport and Environment, the new test’s conditions will also close the old loopholes that manufacturers could have exploited in the past to gain better results (scroll down for what these loopholes were).
But whereas the NEDC measured both fuel economy and emissions, the WLTP will only be used to measure CO2 output and fuel economy. Emissions will be measured by the new Real Driving Emissions (RDE) test.
RDE – new emissions tests
The new Real Driving Emissions (RDE) tests are used to measure ‘real-world’ emission levels of NOx (oxides of nitrogen) and PN (Particulate Number).
Unlike the WLTP, which is carried out in a lab to ensure comparable results, RDE tests will take place outside and be measured using a PEMS machine (Portable Emissions Measurement system), as shown in the picture above.
RDE dates and conformity factors
Similar to WLTP lab tests, all brand new generations of cars released after 1 September 2017 will be RDE tested. All other new cars won’t have to be RDE tested until 1 September 2019.
There are two stages of RDE tests:
RDE step 1 (RDE 1)
Currently, all new diesel cars sold today must emit less than 0.08 g/km of NOx in lab tests to meet the latest emission limit (known as Euro 6).
But new RDE tests will initially allow a conformity factor (CF) of 2.1. This is RDE 1.
This means cars can produce 2.1 times the 0.08g/km limit, or 0.168 g/km of NOx, in RDE tests and still be sold in the UK and Europe.
RDE step 2 (RDE 2)
In 2020 the conformity factor changes to 1.5 (so up to 50% above the actual limit). This is RDE 2. All new generations of cars will have to meet this limit from January 2020, whereas all other new cars sold will have until January 2021 to comply.
A conformity limit of 1.5 means cars will be able to emit up to 0.12 g/km in RDE tests and be declared legal to be sold.
The conformity factor is there partly because results gathered from PEMs testing will vary, as it will be influenced by factors such as temperature and traffic conditions.
Are conformity factors too lenient?
While the conformity factor does seem lenient, manufacturers may not have it easy.
From our own lab-based tests, the amount of NOx we’ve measured from cars that officially meet Euro 6, is 0.24 g/km. This is exactly twice as much as the new 2020-21 limit, and is still noticeably more excessive than the 2017-2019 limit.
Despite the conformity factors, the RDE and WLTP should help bring about more realistic emission and fuel economy figures – but how more realistic remains to be seen.
How Which? finds a car's real mpg
As with the official cycle, the Which? fuel economy test is carried out on a rolling road in a lab. This simulates wind resistance and allows the test to be repeated in the same way in the exact same environment, as we use the same lab for each car test.
We use the Worldwide-harmonised Light vehicles Test Procedure (WLTP) cycle, running both a ‘cold cycle’, where the engine is cold, and a ‘hot cycle’ where the engine is already warm.
We also have a motorway-driving cycle where we accelerate up to, and sustain, motorway speeds. This cycle catches out a lot of cars that are efficient at low speeds but burn a lot of fuel on the motorway.
If you do a lot of miles, remember to check out the motorway mpg figure that we publish in every one of our independent car reviews.
We also set the conditions of the test to be as realistic as possible:
- We test cars in the default setting they start up in, rather than switching to a more economical driving mode.
- Our additional test cycles are conducted with the air conditioning on, lights on dipped beam and the radio working. Our trained drivers also stick to the test’s speed restrictions.
- We don’t tamper with the cars by removing extra weight or inflating the tyres any more than specified in the owner’s manual.
Finally, we don’t adapt our results in any way at the end of our procedure. What you see in our car reviews is what you get.
What about hybrid and electric cars?
Hybrid cars are subject to the same test cycles so that we can compare them directly to their petrol and diesel counterparts.
For non-plug-in hybrid cars
The test is conducted with a 60-70% state of charge displayed on the on-board computer.
For plug-in hybrid cars
We start each test cycle with a full battery charge and continue to repeat them until the state of charge drops below 50%, taking a measurement of the electric energy supplied for charging. We then conduct the same cycles with an empty battery once. Our calculation then takes all of these factors into account.
Electric cars are tested slightly differently. We use a test cycle that combines the WLTC cycle, and our own motorway cycle.
Before the test, the car is charged to full capacity and, just like other vehicles, the air conditioning is set to 20 degrees and daytime running lights (or low beams) are on throughout the cycle.
Through repeat tests, we are able to work out a realistic range that an electric car can travel. The table below reveals the tested ranges for all electric cars we’ve tested so far; you’ll have to log in to find reveal which car is which.
Thinking of buying an electric car? Choice might be limited, but the worst models are both expensive and impractical. To see which battery powered motors we recommend, head for our roundup of the best electric cars we've tested.
What about air pollution?
In the same test, we collect air-polluting emissions data: the quantity of exhaust emissions, such as oxides of nitrogen (NOx), carbon monoxide (CO), hydrocarbons (HC) and particulate matter (PM).
Any vehicle that produces a very large amount of air pollution in our tests cannot be a Best Buy car.
Volkswagen was caught out in September 2015 by employing a ‘defeat device’ in its diesel cars to dupe US tests over NOx emissions. This has become known as the VW emissions scandal, or ‘dieselgate’.
In Europe, 8.5 million cars (1.2 million of which are in the UK) had the same device.
How we stop manufacturers from cheating our tests
There is a risk that cars (from any manufacturer, not just VW) could use similar means to detect they are in a lab testing environment and switch to a more economical running mode. To stop any manufacturer confusing our tests, whenever we are suspicious of the NOx emissions we record in the lab, we now run additional test cycles using a Portable Emissions Measuring System (PEMS).
This device is placed in the car, and the emissions are recorded while driving on real roads. We then compare this NOx emissions data with that recorded in the lab and penalise any cars that have an incredibly high real-life reading.
While this is a realistic way of testing cars, we couldn’t replace our lab testing with PEMS testing alone. Being outside the lab, we are not able to control factors, such as temperature or road conditions, which can affect the result. So results from PEMS are not directly comparable to each other, unlike our lab tests.
Problems with claimed mpg from before September 2017
Back to official fuel economy figures, and for cars that were not released after 1 September, they may still only be using the NEDC - but don’t be fooled by the name. The New European Driving Cycle (NEDC) hasn’t been updated since 1997, and the basis of the test was first introduced in the 1970s.
The old rules allow manufacturers to arbitrarily knock 4% off the results at the end of the cycle.
The test’s numerous loopholes and lack of real-world driving scenarios make the figures it generates unachievable when you actually get behind the wheel of a car.
Here are some of the most notable:
- The test cycle includes urban (in-town) and extra urban (out-of-town) driving. But, while it reaches a top speed of 75mph for 10 seconds, it doesn’t include any sustained motorway driving – this is the type of driving for which many cars consume the most amount of fuel in our tests.
- Many modern cars have adaptable driving modes to make them more economical or sportier. Manufacturers can elect to carry out the official test cycle using an eco mode, but we think few drivers actually opt to use them day to day because they tend to make a car feel sluggish and unresponsive.
- The test is conducted with all ancillary loads turned off, including air conditioning, lights and heated windows, thereby improving efficiency.
- There is a tolerance for the testing to be carried out at 1.2mph below the required speed, meaning less fuel is used, although the speeds used are already quite pedestrian.
- Roof rails, extra lights and even the door mirror on the passenger side are allowed to be removed. This makes the car lighter and, therefore, more fuel-efficient.
- There was no restriction on the air pressure level in the tyres, meaning manufacturers can use higher-than-recommended pressures to reduce rolling resistance. This takes load off the engine and reduces fuel use.
- There was no official body in place to police the testing procedure and monitor the consistency of results between labs.
- All manufacturers follow the same test procedure but can select any accredited lab to use for the test. It’s very hard to get truly repeatable, comparable results when using multiple labs.
- If all that wasn’t enough, the rules also allow manufacturers to arbitrarily knock 4% off the results at the end of the cycle.
Fuel economy you can rely on
While some rival publishers only show the manufacturer’s official mpg figures in their car reviews, and some now offer a limited number of real-world figures, we show our own realistic mpg data for every car we’ve reviewed. That’s currently more than 700 vehicles, and we test around 100 new cars every year.
Whatever your need, our independent tests reveal the full picture: how fuel-efficient, reliable, easy to drive and comfortable cars really are. To help you find the best new or used car at the right price, see our round-up of the best cars.