Researchers at the University of Chicago’s Booth School of Business have demonstrated that OpenAI’s GPT-4 can perform as well as or even better than human experts in financial statement interpretations.
Using a method called chain-of-thought, the researchers trained GPT-4 to simulate the mental processes of a human financial analyst. This allowed the robot to analyse and forecast future market movements.
The team taught the model to produce precise predictions by teaching it to recognise patterns, calculate ratios, and synthesise data. The study claimed that GPT-4 could forecast future profit direction with 60% accuracy, outperforming the majority of human financial analysts who averaged between 53% and 57% accuracy.
The researchers concluded, “LLM prediction does not stem from its training memory.” Instead, we discover that the LLM produces insightful narratives regarding a business’s potential performance.
Skeptics Aren’t Convinced
However, it’s crucial to exercise caution and not draw excessive conclusions from these findings.
ChatGPT uses the data it was trained on to answer user questions and works on their prompts. To be accurate, you must ask the correct question.
On Hacker News, a user pointed out that the researchers’ artificial neural network model, which they used as a benchmark, is from 1989 and cannot be compared to most advanced models utilised by financial analysts today.
On X, AI researcher Matt Holden questioned the researchers’ assertions, stating that it is improbable for GPT-4 to select equities that can outperform a more general index like S&P 500. These concerns reflect the ongoing debate about the effectiveness of AI in stock market analysis.
In one experiment, ChatGPT reported a 26.9% net return for the benchmark stock index the previous year, even though the index had dropped 20%. This was in response to a question on the performance of S&P 500.
These examples demonstrate both the potential and the limitations of AI in stock market analysis.
Researchers from Virginia Tech, Queen’s University, and JPMorgan AI Research have looked at how ChatGPT and GPT-4 performed on simulated Chartered Financial Analyst (CFA) exams. The outcomes are not that impressive. ‘The researcher found that, in tested situations, ChatGPT probably wouldn’t be able to pass CFA levels I and II.’
This suggests that while ChatGPT shows promise, it still has a long way to go before it can match the expertise of human financial analysts.
For now, the market seems to have chosen to stick with more traditional approaches, emphasising the continued importance of human discretion in financial analysis.
