To get useful answers from AI-generating chatbots, advisors need to know what questions to ask and how to ask them.
Eric Ludwig, director of the American College of Retirement Income, and Chet Bennetts, assistant professor of financial planning and program director at the American College of Financial Services, made this point during their session “The Role of AI in Retirement and Longevity Planning” at Wealth. EDGE Management at Diplomat Beach Resort in Hollywood Beach, Fla.
The pair reviewed some of the most popular chatbots, including OpenAI's ChatGPT, Anthropic's Claude, Microsoft's Copilot, and Google's Gemini. Ludwig said advisers who don't currently use these services may already have similar ones without realizing it, including Apple's Siri, Amazon Alexa, Netflix and autocomplete in Microsoft Word and Outlook. These AI models generate results based on what they learned from the data that was trained on user interactions.
Bennetts said advisers should think of these models as an extended version of the Monte Carlo method, which finds a probability distribution of returns given a range of circumstances.
“If we are tossing a coin, what is the probability that this flip is heads versus tails? If you do it enough, you're going to get this distribution that's not exactly 50-50, but there's going to be an average,” he said. “Now, imagine a 30 billion page coin where it's all words.”
Ludwig said advisers should only consider using a service like ChatGPT in their practices if they have the expertise to verify the result is accurate and are willing to take full responsibility for inaccuracies.
“It's a know-it-all that's constantly being learned,” he said. Based on its training, generative AI is still sensitive to pre-existing social biases. Perhaps it could not replicate the qualitative analysis.
Using the “rapid engineering” technique means that the more well-structured the questions, the better the answers.
To highlight the importance of this emerging skill set, they showed a job posting for a fast-track engineering manager at Anthropic, with a salary range of $320,000 to $520,000.
“It's part art and part science,” Bennetts said. “This involves creating or adjusting the data to improve the model's response.”
They created an acronym to explain the steps to creating better requests: RATE, which stands for role, question or task, tone, and add-ons.
Ludwig said that the role means who you are and who you are writing for. After choosing the question or task, the tone should be specified. (Should it be funny? Seriously?) Then come the extras. This includes additional details specific to that question that will narrow the answer.
“I always try to say 'please' and 'thank you' at every opportunity, so when it takes over the world, ChatGPT remembers me as a friend,” Bennetts said.
The important part, Ludwig said, is to be as specific as possible and leave nothing to interpretation.
“Garbage in and out,” he said.