Security software: How we test security software
Which? test lab
Which? tests almost 3,000 products every year, including more home technology products than anyone else. We look at new products as soon as they are announced, helping you to understand new technologies, and whether they are worth your money.
Which? is different because...
• Our tests are based on years of testing experience
• We don't accept adverts in our magazines or website
• We buy everything we lab-test
• We're completely independent of any manufacturers
Which? exists to give consumers impartial advice, which means that if a product isn't any good we'll say so, without fear of penalty. We examine everything that matters across all products, including performance, features and how well they work in real life – so you'll know exactly what to expect. Our unique, comparative lab tests mean you can trust our Best Buy and Don't Buy verdicts and choose with confidence. Which? works for you, providing trustworthy advice without a hidden agenda.
Try Which? today for just £1 to find out which tablets we rated as Best Buys.
Security software testing in brief
Our security software testing is based on two important aspects:
- Does it respond to viruses and other security threats in a manner in which your computer and others' is kept safe?
- Is it easy to purchase, install and interact with the software on a day-to-day basis so that your security is maximised?
To test this, we run each software through a series of scenario based tests involving real security threats.
How we choose products for testing
Dedicated researchers carefully select the products Which? tests, scouring the market for the latest releases and the most innovative products. Our researchers aim to cover a very high percentage of the market in any one product area. And, while we do test some premium products, we avoid testing those which you would need a second mortgage to afford. We also make sure we test free options when they are available. We take the following criteria into account when choosing security software.
• Popularity - we test the biggest selling security software, for both Mac and PC
• Innovation - we test security software with interesting, new features
• Cost - we typically test software costing up to £35.
Testing security software
Testing Avira Internet Security's spam protection
To ensure security programs offer good basic protection from day-to-day threats, we put together a horrible collection of old and new viruses and other malicious files, also known as malware. We expose these to each security software package and analyse how they deal with the attacks. Malware is delivered in a different way – through a zipped email file attachment, a USB stick, downloaded from a website and via a networked computer – to see how each program reacts and whether it catches malware early enough.
We also test the programs to ensure they don’t pass infected files to other people, even if they haven’t been opened by the original recipient.
Programs with spam filters are subjected to more than 100 spam emails, with a few genuine ones thrown in. And we click on links contained in phishing emails – hoax emails sent in the hope that the recipient will enter confidential information – to see whether the software stops us from proceeding.
We probe firewalls for weaknesses to see how the software’s first line of defence measures up against hackers after your personal information. We also test parental controls for ability to block adult and other inappropriate websites, while checking to see how much control they can actually offer to parents and whether interaction with the child is also suitable. We also examine identity protection, backup, tune-up and components of the software, for their effectiveness and usefulness.
Ease of use
It's all very well for a security software program to pick up every virus and spam email that's out there, but it still needs to be easy to use. We look for security software that not only identifies threats, but warns you clearly and talks to you in jargon-free language as you go through the removal process. Points are deducted for confusing or overbearing alerts and any ambiguous action that could compromise security.
Before that, we buy and install each security software package on a ‘fresh’ computer, and rate this process looking out for complications and attempts to sell you more products. We also carefully rate the initial software set-up process and first scan, looking to see if it's easy for a beginner to set up properly.
Ratings explained
All data sent back from our laboratories is carefully analysed by Which? staff so that the highs and lows of every software package are revealed. If anything does not add up with a product then we will retest it. If there's a fault we'll purchase a new product or contact the manufacturer to see whether it’s a one-off issue, or symptomatic of a larger problem. The security software score ignores price and is based on:
• Ease of use 60%
• Performance 40%
Ratings system:
