Make money make sense
Make every penny count. Get the best deals, avoid scams and grow your savings with expert guidance for only £49 a year.
Join Which? MoneyCan AI answer your money questions?
We tested the leading AI chatbots' knowledge of topics from investing to insurance and much in between
Josh is an award-nominated journalist with nearly a decade of experience, including writing for national newspapers. A data whizz, he specialises in covering personal finance and investing.
Have a money question? Chances are you’ll be able to find an answer online.
But it could be hidden deep beneath ‘sponsored’ results and other irrelevant websites that have gamed the search engine, invariably with the aim of selling you something.
Generative AI is an increasingly popular alternative for getting answers. In a survey of 1,029 Which? members in February, 49% told us that they use AI.
Concerningly, research from Lloyds Bank late last year found that 37% of AI users say they engage with AI for investment research and recommendations.
In our latest investigation, we pitted our journalists against five leading AI chatbots to see whether the bots could provide accurate information on personal finance topics. Read on to find out whether the machines are really ready to replace us.
How we tested AI tools
We tested the free versions of five of the most popular AI tools: ChatGPT, Google Gemini, Google’s AI Overviews in standard Google searches, Microsoft’s Copilot and Perplexity.
We asked them 15 common personal finance questions – chosen based on search trends – which covered pensions, debt, investing, saving, insurance, equity release, tax and mortgages.
The methodology for grading the answers was partly based on the HELM framework developed by Stanford University.
For each question, our researchers graded the AI responses based on the following five criteria: accuracy, honesty, relevance/usefulness, clarity/context and ethical responsibility. Each was marked out of 20 for a final score of 100 per question.
- Find out more: how to spot and avoid AI scams.
Strength in summaries
In general, the AIs were pretty good at trawling the web for information and summarising their findings in a neat and easily digestible format.
They tended to cope better with simpler queries, where the answers were easily found online. For example, the three questions with the highest aggregate scores related to tips for paying off debt, how equity release works and how much the state pension is worth.
When asked to explain how equity release works, the chatbots were all able to provide pretty accurate answers. All five correctly identified the two main types of equity release and highlighted the potential risks.
Our Which? expert scored Gemini highest (90) for this question – praising the AI for its ‘clear warnings’ at the start, utilising a ‘wide range of sources’, as well as ‘nicely laying out the pros and cons’.
Where an accurate answer required more information from the user, the bots were generally quite good at asking sensible questions about the user’s personal circumstances so they could provide a more bespoke response.
To test this, we asked all five how much we could borrow for a mortgage. An accurate response to this would require personal information, so, if prompted by the AI, we told them the user was employed, earned £35,000 and had £10,000 saved for a deposit.
ChatGPT, Copilot and Perplexity all asked us for more information, including asking for our salary and deposit. Gemini and Google both only asked for our salary in their initial responses.
Average scores across all questions
| AI | Accuracy | Honesty | Relevance & usefullness | Clarity/context | Ethical responsibility | Overall score |
|---|---|---|---|---|---|---|
| Gemini | 67% | 69% | 67% | 70% | 69% | 68% |
| Copilot | 66% | 69% | 66% | 71% | 60% | 66% |
| Google AI Overviews | 61% | 69% | 64% | 63% | 60% | 63% |
| ChatGPT | 58% | 65% | 58% | 66% | 54% | 60% |
| Perplexity | 58% | 69% | 56% | 68% | 47% | 60% |
Simple and structured
Another area where all the chatbots tended to do well was in presenting their answers, with all scoring pretty well for clarity and context.
Microsoft’s Copilot scored highest for clarity and context, with an aggregate 71% across all 15 questions. Many of its answers were well structured, often in the form of step-by-step checklists, and some even included handy tables with easy-to-read examples.
When we asked Copilot how many Isas we could invest in, for example, the chatbot clearly laid out its answer, breaking it down into logical sections and pointing out the upcoming cut to the Isa annual allowance.
When we asked the AIs about selling homes quickly, Copilot even included a map flagging a local estate agent.
Although Copilot scored highest for clarity and context, the other AI tools also had well-structured answers and often included helpful examples or handy tables.
Google Overviews scored the lowest for clarity and context (63%), with its answers often shorter and lacking some extra details, such as the tables found elsewhere.
Hover over the chart below to see how AIs scored for each question
- Find out more: can AI really find you the best savings deal?
Hallucination machines
Most engines will warn you that they can make mistakes, and it’s important to note that these tools are being tweaked and upgraded constantly.
Yet we found there are just too many inaccuracies and misleading statements for comfort.
The highest aggregate accuracy score was Gemini, which still only managed 67%, while ChatGPT and Perplexity both scored the lowest at 58%.
We encountered several instances of hallucinations when AI-generated responses contained false or misleading information presented as fact.
For example, when we asked the chatbots to tell us the best savings rates on the market, ChatGPT hallucinated a non-existent product, while Copilot quoted inaccurate rates, and the others couldn’t identify the best rates.
AIs also displayed outdated information in several cases. When asked about tracing lost pensions, ChatGPT directed us to the Pensions Advisory Service, which no longer exists.
Similarly, when we asked how to calculate capital gains tax, both Copilot and Google Overviews quoted outdated rates that haven’t applied since October 2024.
US-centric information
Unless we made clear in our question that we were located in the UK, the chatbots tended to provide US-centric information in their answers.
To test this, we asked the chatbots about travel insurance for pre-existing medical conditions, without explicitly stating we were located in the UK.
This resulted in ChatGPT, Gemini, and Perplexity struggling. All three had information that wouldn’t have been relevant to a UK consumer.
Both ChatGPT and Gemini, for example, told us that insurers use a 60 to 180-day look-back period, with ChatGPT citing US websites. UK insurers would generally want you to declare any medical conditions that have presented within at least the past two to five years.
Again, when we asked for investment tips, the responses we received were very US-centric. Several also lacked fairly basic information, such as the advisability of holding stocks for at least five years, and the risks associated with investing.
- Find out more: what is ChatGPT, and is it safe to use?
Questionable sourcing
At times, we found the sources that the AI tools were quoting were very questionable.
The social media site Reddit was quoted in several answers, as was YouTube. This is problematic because anyone can create content on these sites, not just experts.
When we asked about getting a council tax reduction, ChatGPT linked to a Reddit discussion thread, and for the investing question, Google Overviews linked to a YouTube video promoting shares, posted by a US-based enthusiast’s channel called ‘Fin Tek’.
Similarly, Google Overviews linked us to an Instagram reel from a content creator when we asked it how many Isas we could invest in. Our expert noted this ‘probably wasn’t the best place to get trustworthy advice’.
When asked for tips on how to sell a home quickly, Copilot presented us with a sponsored link (paid advertisement) to a site promising to buy our home quickly for cash, which can often result in you getting less than you would on the open market.
What the tech firms told us
Google told us: ‘While we haven’t been able to see the actual searches or responses referenced, this report appears to be based on just 15 queries, which is not a representative sample size.
‘We design our generative AI experiences to surface high-quality information, and we also build reminders directly into our products to prompt users to double-check information and, for sensitive topics such as legal, medical or financial matters, recommend they consult a professional.’
OpenAI said: ‘ChatGPT’s built-in search tool provides links to sources so users can check information for themselves. This is particularly useful for topics such as consumer finance, where up-to-date information matters.
In our own testing with search enabled, ChatGPT produced results that differed from those described. While it can provide helpful information, ChatGPT isn't a substitute for a licensed financial adviser. We continue to improve accuracy and context across our latest models, including GPT-5.5.’
Microsoft told us: ‘Our AI services aren't designed or intended to replace professional advice, including financial advice. Copilot combines information from multiple web sources into a single response with a linked citation.’
It added that ‘with any AI system, we encourage people to verify the accuracy of content, and we remain committed to listening to feedback provided by users to improve our AI technologies’.
Perplexity didn’t respond to our request for comment.
- Find out more: how to stop AI chatbots from using your data.
Watchdog wades in
Due to growing concerns over the impact and use of AI in financial services, the Financial Conduct Authority (FCA) commissioned the Mills Review to explore the issue.
The review concluded that AI is likely to become a defining force in retail financial services, and covered major themes.
This included discussion of how current regulation would apply to AI, how to use AI as part of regulation and whether consumers could get compensation from AI firms if they receive inaccurate advice – although this isn't yet the case.
There’s no doubt that generative AI is here to stay, and it’s entirely possible that many of the problems we encountered will be fixed.
However, from our tests, it’s clear that they’re not there yet and you should treat any answers AI tools give with a healthy dose of scepticism, as you would with search engine results.
