Consider the following scenario. Suzy is 63 and recently retired and is considering when to collect Social Security and how to manage her retirement savings to minimize her tax burden.
She opens an AI chatbot, enters the details and receives a calm, well-organized and confident response: Now claim, convert so much, here’s why.
The chatbot sounds authoritative and even shows its work. That’s why Suzy follows his lead and never calls a financial planner. Maybe the advice was fine. But perhaps it tacitly ignored the fact that Suzy’s spouse is younger and in poor health, which can upend the Social Security bill. It may also have been overlooked that the proposed transition to a retirement plan would cause Suzy to pay higher Medicare premiums two years later.
Suzy will only find out whether this instruction was right for her in a very long time, if at all. And the AI will never call back and say it’s unsafe.
Suzy is no exception. AI chatbots have entered everyday life with remarkable speed: A 2025 Pew Research Center survey found that 34% of U.S. adults and 58% of those under 30 have used ChatGPT, about double the share two years ago.
More and more people are asking AI for money, and some are becoming desperate. According to a 2025 survey of 2,000 U.S. adults by Pearl.com, a professional services platform, 19% said they lost more than $100 because they followed financial advice from an AI chatbot. This figure rose to 27% among Generation Z investors.
These are not hypothetical risks. People are already paying for answers about their money that are confident – and wrong.
As a finance professor who closely monitors the spread of AI in personal finance, this is the part of the AI story that worries me the most. And it’s not the part you usually hear about.
Table of Contents
We are arguing about AI in the wrong way
There are two seemingly opposing complaints about AI. One is that people trust it too much and treat a chatbot like an oracle, a tendency researchers call algorithm bias. The other is that people don’t trust it enough and reject its useful tools, a tendency known as algorithm aversion.
I argue that these are actually two sides of the same coin, and what determines which side you see is whether you can tell when the AI is wrong.
When an AI fails in an obvious way, you notice it and lose trust. Therefore, you are more likely to see a professional or someone you trust more quickly than you otherwise would. That is the sure mistake.
Dangerous failure is the opposite. The answer is fluid, certain – and wrong. You have no way of knowing, so you end up taking care of the problem yourself, long after you should have asked for help.
The problem is that when it comes to money, the second type of failure is the common one.
Typical users of financial advice chatbots tend to be younger, with men predominating.
Tim Gouw on Upslash, CC BY
If you confuse fluency with accuracy
Three things make AI financial advice particularly tricky.
First, fluency does not equal accuracy. People naturally interpret a confident and well-articulated answer as competent. But no matter how sophisticated an answer sounds, it tells you almost nothing about whether it fits your situation or how accurate the proposed solution is. A chatbot can be literal and still be wrong about your taxes because your taxes depend on details it never asked for.
Second, AI is least reliable precisely where the stakes are highest. AI tools are good for routine and general topics: what is a Roth IRA, how does compound interest work, the difference between a stock and a bond.
But financial life is full of rare, complicated and unique decisions: exercising stock options, understanding the alternative minimum tax, making required minimum 401(k) distributions, deciding on a Social Security strategy as a couple, drafting a divorce agreement.
I made a similar argument about AI trading on Wall Street three years ago. Because market crashes are rare, there is little data for the AI to learn from, so it can be safest where it is least informed.
That concern has not faded. Market watchers are now warning that AI trading bots are creating new financial risks, and that the same blind spot also applies to your personal finances. Researchers call this uneven competence a “jagged boundary”—reliable in common cases, but unreliable in unusual ones. And in finance, the unusual cases tend to be the expensive ones.
Thirdly, you often cannot check the work. Financial advice is what economists call “credibility goods,” like a mechanic’s diagnosis or a doctor’s recommendation. It is often not possible to tell whether the advice was good, sometimes for years. An incorrect tax decision may only come to light during an audit. A bad 401(k) drawdown plan may not make an impact until the stock market crashes. Without quick feedback, the wrong but safe answer will never be corrected.
For this reason, the Pearl numbers above are likely an undercount, as they only capture losses that people noticed.
Silent failure is what you should keep in mind
Note that the real damage in Suzy’s story is not a single dramatic mistake. Because of a confident response, Suzy didn’t feel the need to call a professional, so the call never happened.
The danger is not so much that you respond to bad advice, but that you never seek good advice. The smoother and more reassuring the tool is, the easier it is to stay in do-it-yourself mode even when you need outside help.
Who is most at risk? In a study of a large robo-advising platform in India, co-author Vishaal Baulkaran and I found that users tended to be young, predominantly male, and tended to be smaller retail investors and professionals. And during times of high market volatility, new account sign-ups increase.
In other words, the number of people who rely most heavily on automated advice is equivalent to the 27% of Gen Zers who lost more than $100 while using a chatbot for financial advice. They resort to it precisely when markets become turbulent and a wrong move is most costly.
There is also a notable incentive. In my new analysis, I argue that a tool that makes its profit by capturing your attention has a reason to sound confident and helpful: trust keeps you on the platform. The catch is that sometimes the user it keeps in this way is the one that should have been given to a human.
A system designed to motivate you is not the same as one designed to protect your financial future, and both can point in different directions. The disruption is already underway as asset managers face what Bloomberg calls a chatbot settlement. A single, new AI tax tool recently sent wealth management stocks lower as investors bet automated advice will hurt business.
How to use AI intelligently
These findings don’t mean people should avoid AI for money advice. When used correctly, these tools are a valuable and free financial education.
That doesn’t mean that a financial advisor always has the right answers. As with finding a specialist, it’s important to do your research first and make sure they meet the criteria set by the Consumer Financial Protection Bureau. Fee transparency is also crucial.
However, when you turn to AI, the skill lies in knowing where to draw the line.
Think of AI as a starting point, not a judgment. It’s great for learning concepts, writing questions, and orienting yourself before a meeting. It can teach people the vocabulary to have a smarter conversation with an expert.
But watch out for the signals that you’ve stepped out of your comfort zone and into the territory where AI is weakest and a sure answer is least trustworthy. The warning signs are large dollar amounts, tax consequences, anything irreversible, and anything that relates to the specifics of your situation rather than a general rule.
This category includes questions about inheritance, retirement savings, social security benefit utilization strategies, corporate structure, and larger one-time transactions. These are the decisions that require the involvement of a human, such as a certified financial planner.
And remember: trust does not equal competence. If the answer to your money sounds most sophisticated and safe, that’s no reason to relax. When faced with the toughest questions, that gentle confidence is exactly the signal that you should pick up the phone and speak to an expert.
https://theconversation.com/when-managing-your-money-take-a-chatbots-confidence-with-a-grain-of-salt-286106
