Investor's Last Exam: Can AI Agents Do Equity Investment Without Seeing the Future?
Abstract
Investor's Last Exam evaluates financial agents on end-to-end equity research using frozen evidence, expert criteria, and dependency graphs, revealing that top models excel at research but fail at forecasting and investment judgments.
Financial agents must form forecasts and investment judgments from decision-time evidence, with predictions evaluated against subsequent outcomes. Existing benchmarks assess financial knowledge, research workflows, or forecasting, but evaluating the full research process requires a strictly frozen information set and professional evaluation criteria. We introduce Investor's Last Exam (ILE), an equity-research benchmark led by experienced buy-side investors and fund managers. Its three components are a versioned financial database spanning 3.43 million document versions and nine source categories, supplemented by structured market data, with validated timestamps governing retrieval and computation; a task taxonomy organized around professional analyst deliverables; and an expert-led framework linking scoring criteria to sources, calculations, and assumptions through financial dependency graphs. Evaluation spans 86 assignments and jointly assesses research quality, forecast accuracy, and simulated investment performance. GPT-6 Astra and Claude Opus 5.5 lead overall at 68.1 and 66.9 out of 100, yet even Astra scores only 52.5 on Forecast & Model. Our results show that even state-of-the-art models fail to translate overall research capability into predictive performance. ILE shifts the evaluation unit from isolated capabilities to the evidence--assumption--judgment chain, allowing researchers to locate where analytical advantages fail to translate into forecasts and investment judgments, and providing a traceable, diagnostic foundation for training and evaluating financial agents.
Community
Evaluating forecasting capability usually means waiting for outcomes to materialize — months, sometimes years — during which market conditions, fundamentals, and the information environment shift enough that scores can no longer be cleanly attributed to the model; moreover, most work in LLM4Finance has focused on the research stage rather than the investment stage, which is where the field's core objective lies. ILE asks whether an agent, using only what was visible at decision time, can independently complete the full chain from evidence to assumptions to forecast. To answer it, we built a versioned corpus (3.43M document versions, 1.52TB, 9 source classes) that enforces the cutoff across the entire toolchain — specify a timestamp and you get exactly the versions visible at that moment — and organized 86 assignments around buy-side analyst deliverables, with rubrics anchored to sources and reproducible calculations through financial dependency graphs. Among current SOTA systems, Astra scores 78.9 on Research & Evidence but only 52.5 on Forecast & Model, and ordering by realized outcomes puts DeepSeek V4.1 Flash ahead (40.3 vs 37.7): today's models are very good at organizing evidence of what has already happened into coherent analysis, and not yet good at turning that analysis into judgment about what has not. For me, the longer-term significance is on the training side — once the time boundary is a mechanism, historical timestamps become a re-samplable environment for large-scale domain RL, with evaluation and training sharing one environment.
The strict time-cutoff across the whole toolchain is the most useful design choice here. The main leak risk I would test for is an agent reaching information through a side channel, such as a tool that returns the latest version of a document, a cached web fetch, or model memory of post-cutoff events. I would add canary checks that plant a document dated after the cutoff and assert no tool call ever returns it, plus a probe that asks the agent for facts known only after the decision date. With 86 assignments and a gap like 78.9 versus 52.5, I would also report variance over repeated runs of the same assignment, since one forecast sample can swing the investment score. Releasing the per-step tool trace would let others audit retrieval timestamps.
Froze a document corpus for a backtest last year and still got burned — the embedding index has no concept of a date, so a query about Q3 earnings happily returned a chunk from the 10-K filed in February. Pinning the corpus isn't enough; you have to replay every question through the actual retrieval stack and count how often the top-k contains a version timestamped after the decision point. That leak rate belongs next to the headline score, because a non-zero value means the benchmark is grading hindsight and calling it skill. The 3.43M versioned documents are the real contribution — that's the substrate every point-in-time eval needs and almost nobody bothers to build. I'd also want grader agreement published, since a rubric written by a handful of buy-side PMs is only as stable as their consensus, and an agent tuned against it learns their taste, not the market's.
Get this paper in your agent:
hf papers read 2610.38342 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper