A Journey with GPT-4 to Glean Pension Data (Part I)
Focus on Manual GPT Use
The evolution of artificial intelligence has led to Chat-GPT4 emerging as a transformative force in the space of financial analysis. In particular, the analysis of complex financial annual reports.
With the additional capability for GPT-4 to process PDF documents, a feature incorporated in mid-2023, and the continuous improvement of GPT’s performance, the technology is primed for widespread integration in various financial applications.
So, can GPT-4 take an annual report for a company and glean specific information about their pension funds from it? The short answer… Not exactly.
Using GPT-4 to Glean Pension Data (Basic)
Task: Seeing how ChatGPT-4 and new AI approaches can be used for Pension Analysis at Financial Canvas
The majority of FTSE 250 companies are obligated to submit comprehensive annual reports, in which data regarding pensions is detailed.
These reports vary in length, with reports ranging from 60 pages on the low end to over 300 pages in some cases. This presents an obvious challenge as locating specific data can be time-intensive when done manually.
For Financial Canvas, where raw data serves as the foundation of many models, the significance of acquiring and analysing such data becomes very apparent.
GPT-4 Response to PDFs
The 88-page Shell 2022 Annual Report provided an ideal foundation for evaluating GPT-4 capabilities in data extraction and analysis from PDFs.
Upon creating a simple prompt and pairing it with the corresponding PDF, it was clear that GPT-4 was going to require more guidance in analysing these documents if accurate retrieval of data was to be achieved.
ChatGPT’s initial response is vague and peers towards providing a general overview rather than the essence of the prompt and providing the relevant financial tables.
Further investigation suggests that the length of the PDF uses up a significant number of the available tokens, limiting the tokens available for a more comprehensive response with the required information.
After manually trimming the PDF to the relevant pension section, a role that should be delegated to AI itself, and pairing it with a similar prompt, the results were an instant improvement.
Further prompting of ChatGPT led to the generation of some informative graphics illustrating the data previously output by ChatGPT.
ChatGPT evidently plays a useful role in the extraction and analysis of financial data from PDF form. But can these capabilities be taken further?
Custom GPT’s for Efficient Analysis
Although effective at extracting tables and displaying data from smaller PDFs, the process required a significant amount of user intervention and assistance, something counteractive to the automation envisioned with AI.
The introduction of “Custom GPTs” allows for a slight reduction in user input and time. Pre-set learning prompts allow for more consistent GPT outputs reducing the need for a user to re-enter prompts to push GPT in the correct direction.
This increased accuracy allows for the process to push further towards the goal of automation. All custom GPTs developed by Sciurus Analytics Ltd are available on the GPT marketplace.
The development of PensionsGPT allowed for the accurate and now consistent removal of target financial data from trimmed annual report PDFs. This custom GPT focuses on displaying output data in an easy-to-use table format and provides graphical representations of output data when queried.
To further assist in making the process efficient, a custom Link Finder GPT was developed. With a focus on grabbing annual report links from the web, using ChatGPT's in-built search with Bing functionality, it greatly streamlines the process of getting target data for a company without having to navigate through their websites.
Utilising a combination of these customs GPTs along with a manual intervention to trim the PDFs down to the target sections, allowed for very comprehensive responses with far greater efficiency than before. This seamless integration automates PDF trimming and simplifies the processing of large datasets, representing a substantial leap towards the full automation of financial analysis discussed in part II of the blog.