MediPFN
MediPFN estimates the population natural indirect effect (NIE) and its uncertainty from observed tabular data. The StatsArena checkpoint takes one 500-row table and combines frozen TabPFN-v3 nuisance predictions with a cross-fitted mediation score, a fine-tuned error head, and independent calibration.
Paper · Code · Arena leaderboard · Benchmark data and submissions · Contributor template
Benchmark data card · Run one example · Checkpoint quick start
Benchmark data card
MediPFN V2 Arena, 500-row release. How much would the mean outcome under treatment change if the mediator vector moved from the values it would take under control to those it would take under treatment? The NIE above is a population contrast in outcome units. Each row is one observation with baseline covariates $X$, binary treatment $A$, mediator(s) $M$ and outcome $Y$. At test time each method sees one complete observed table. Only the evaluator uses the mechanism and its true NIE.
Training data
| Stage | Distinct tables | How they were used |
|---|---|---|
| Adapter | 8,000 | Mediation-specific adaptation of frozen TabPFN-v3 backbones |
| Error head | 10,000 | Learn quantiles of the point estimator's standardized error |
| Supervised fine-tuning | 2,000 new | Each batch mixed fresh tables and replay in equal proportions |
| Total used for fitting | 20,000 | Replayed tables counted once |
| Development | 1,552 | 16 for the adapter, 1,024 for the error head, 512 for fine-tuning |
| Independent calibration | 1,024 | Fresh draws from the training families to calibrate intervals |
All tables in this release have 500 observations. These counts describe MediPFN adaptation. The upstream TabPFN pretraining data exposure is unknown.
Training used simulated mediation studies with linear and nonlinear relationships, single and multiple mediators, and null and nonzero effects. For each study the generator selected a design, drew its parameters and sampled 500 observations. The published fitting tables and their population NIE labels can be used to train another method. The independent starter kit documents two practice designs, with equations, sampling rules and original references. Training data and public examples
Official evaluation
The official test contains 2,000 observed studies from 1,000 configurations, with two independent study draws per configuration. Each study receives an opaque ID. Participants receive the observed variables and five treatment-stratified outer folds. The organizers hold each study's generator assignment, parameter values, simulation seed and true effect. Fix the method before processing the official studies and return one NIE estimate and one 95% interval per ID. Evaluation and submission instructions
MediPFN and the random-forest baseline used the same studies and outer folds. The published checkpoint runner uses two folds. The five-fold Arena comparison used the existing interval calibration and weighted all 2,000 studies equally.
| Method | Bias | RMSE | 95% coverage | Mean interval length |
|---|---|---|---|---|
| MediPFN | 0.0219 | 0.5891 | 97.30% | 1.2654 |
| Random forest with bootstrap | -0.1098 | 2.0533 | 94.90% | 4.3238 |
Run one example
This independent He–Song–Xu-inspired table has 500 rows and seed 1729. Draw $X_1$ from a standard normal distribution and draw $X_2$ and $A$ independently from Bernoulli$(0.5)$. With independent standard-normal errors, the equations are
An intervention on $A$ shifts the mediator mean by 0.2. Each additional unit of $M$ shifts the outcome mean by 0.2 at fixed treatment. The population NIE is therefore $0.2\times0.2=0.04$.
| $X_1$ | $X_2$ | $A$ | $M$ | $Y$ |
|---|---|---|---|---|
| 2.0785 | 1 | 1 | 4.2548 | 5.7718 |
| −0.5509 | 1 | 1 | 1.8781 | 2.5719 |
| 0.2774 | 0 | 1 | 1.2213 | 2.1576 |
Download the observed CSV · Generator · Example runner
For a quick check, run the adjusted linear product estimator on the example. This fits an OLS regression for each path and uses 1,000 row-bootstrap draws for its percentile interval. From a Python 3.10 environment:
python -m pip install numpy "huggingface_hub==2.4.0"
hf download shibo769/MediPFN run_example.py generate_example.py arena_rf_reference.py arena_n500/requirements-research.txt --local-dir medipfn_example
cd medipfn_example
python run_example.py --method linear
To run the released MediPFN predictor and the leaderboard's random-forest estimator on these same observations:
python -m pip install -r arena_n500/requirements-research.txt
python run_example.py --method medipfn
python run_example.py --method arena-rf
MediPFN downloads the pinned backbones on first use. The RF analysis uses 200 trees, five outer folds and 50 bootstrap replicates. Each command reports the estimate, 95% interval and error against the same known NIE. The five-fold leaderboard aggregates a separate collection of 2,000 tables.
| Method on this example | NIE estimate | 95% interval | Absolute error from 0.04 |
|---|---|---|---|
| MediPFN, released two-fold predictor | 0.0373 | [0.0001, 0.0745] | 0.0027 |
| Arena random forest, five folds | 0.0015 | [−0.1231, 0.2601] | 0.0385 |
| Adjusted linear product estimator | 0.0348 | [0.0121, 0.0651] | 0.0052 |
To document another data design, copy the submission template. To enter your estimator, follow the mediation submission instructions.
Checkpoints
| File | Contents |
|---|---|
| Arena method ZIP | Complete MediPFN inference bundle with the files below, code, and a synthetic example |
| Mediation adapter | Trained adaptation for nuisance prediction |
| Error head | Fine-tuned model for quantiles of estimation error |
| Calibration | Independently fitted interval adjustments |
The Quick Start also downloads the official classifier and regressor. Prior Labs TabPFN-v3 weights
Quick Start
Use Python 3.10. The first run downloads about 446 MB of official TabPFN-v3 weights from Prior Labs. Both Hugging Face repositories support anonymous downloads.
python -m pip install "huggingface_hub==2.4.0"
hf download shibo769/MediPFN arena_n500/requirements-research.txt --revision 315046cf9838ebcf2b920dad0f87e95bbb5bbc04 --local-dir medipfn_hf
python -m pip install -r medipfn_hf/arena_n500/requirements-research.txt
Save the following as quickstart.py and run python quickstart.py. The example
downloads a fixed MediPFN release and the two matching TabPFN-v3 checkpoints,
then scores the included synthetic table on CPU.
import json
from pathlib import Path
import sys
from huggingface_hub import hf_hub_download, snapshot_download
model_dir = Path(snapshot_download(
repo_id="shibo769/MediPFN",
revision="315046cf9838ebcf2b920dad0f87e95bbb5bbc04",
allow_patterns=["arena_n500/**"],
ignore_patterns=["arena_n500/MODEL_CARD.md"],
local_dir="medipfn_hf",
token=False,
)) / "arena_n500"
backbone_repo = "Prior-Labs/tabpfn_3"
backbone_revision = "e923ba9be85784206c9e2f43b0035c84d5fd5747"
classifier, regressor = [
hf_hub_download(
repo_id=backbone_repo,
filename=f"tabpfn-v3-{role}-v3_default.ckpt",
revision=backbone_revision,
token=False,
)
for role in ("classifier", "regressor")
]
sys.path.insert(0, str(model_dir.resolve()))
from predict import Predictor, read_observed
predictor = Predictor(classifier, regressor, device="cpu", threads=4)
observed = read_observed(model_dir / "example" / "observed_n500.npz")
result = predictor(observed, fold_seed=18031)
print(json.dumps({
"estimate": result["estimate"],
"standard_error": result["standard_error"],
"intervals": result["intervals"],
}, indent=2))
With the example seed, the NIE is approximately 0.2457, the score standard
error is 0.0634, and the 95% confidence interval is [0.064, 0.427].
For Arena, initialize Predictor once and reuse it across tables. Pass the
evaluator-assigned fold_seed on each call. Supply an observed 500-row table
with x, a, m, y, x_types, and mediator_types.
See the input specification for the table format, model details for identification assumptions, and license and attribution for usage terms. NOTICE.
Model tree for shibo769/MediPFN
Base model
Prior-Labs/tabpfn_3