MediPFN

MediPFN estimates the population natural indirect effect (NIE) and its uncertainty from observed tabular data. The StatsArena checkpoint takes one 500-row table and combines frozen TabPFN-v3 nuisance predictions with a cross-fitted mediation score, a fine-tuned error head, and independent calibration.

NIE=E ⁣[Y(1,M(1))−Y(1,M(0))]. \mathrm{NIE} = \mathbb{E}\!\left[Y^{(1,M^{(1)})} - Y^{(1,M^{(0)})}\right].

Paper · Code · Arena leaderboard · Benchmark data and submissions · Contributor template

Benchmark data card · Run one example · Checkpoint quick start

Benchmark data card

MediPFN V2 Arena, 500-row release. How much would the mean outcome under treatment change if the mediator vector moved from the values it would take under control to those it would take under treatment? The NIE above is a population contrast in outcome units. Each row is one observation with baseline covariates $X$, binary treatment $A$, mediator(s) $M$ and outcome $Y$. At test time each method sees one complete observed table. Only the evaluator uses the mechanism and its true NIE.

Training data

Stage Distinct tables How they were used
Adapter 8,000 Mediation-specific adaptation of frozen TabPFN-v3 backbones
Error head 10,000 Learn quantiles of the point estimator's standardized error
Supervised fine-tuning 2,000 new Each batch mixed fresh tables and replay in equal proportions
Total used for fitting 20,000 Replayed tables counted once
Development 1,552 16 for the adapter, 1,024 for the error head, 512 for fine-tuning
Independent calibration 1,024 Fresh draws from the training families to calibrate intervals

All tables in this release have 500 observations. These counts describe MediPFN adaptation. The upstream TabPFN pretraining data exposure is unknown.

Training used simulated mediation studies with linear and nonlinear relationships, single and multiple mediators, and null and nonzero effects. For each study the generator selected a design, drew its parameters and sampled 500 observations. The published fitting tables and their population NIE labels can be used to train another method. The independent starter kit documents two practice designs, with equations, sampling rules and original references. Training data and public examples

Official evaluation

The official test contains 2,000 observed studies from 1,000 configurations, with two independent study draws per configuration. Each study receives an opaque ID. Participants receive the observed variables and five treatment-stratified outer folds. The organizers hold each study's generator assignment, parameter values, simulation seed and true effect. Fix the method before processing the official studies and return one NIE estimate and one 95% interval per ID. Evaluation and submission instructions

MediPFN and the random-forest baseline used the same studies and outer folds. The published checkpoint runner uses two folds. The five-fold Arena comparison used the existing interval calibration and weighted all 2,000 studies equally.

Method Bias RMSE 95% coverage Mean interval length
MediPFN 0.0219 0.5891 97.30% 1.2654
Random forest with bootstrap -0.1098 2.0533 94.90% 4.3238

Run one example

This independent He–Song–Xu-inspired table has 500 rows and seed 1729. Draw $X_1$ from a standard normal distribution and draw $X_2$ and $A$ independently from Bernoulli$(0.5)$. With independent standard-normal errors, the equations are

M=1+0.2A+X1+X2+0.5ϵM,Y=1+A+0.2M+X1+X2+0.5ϵY. M=1+0.2A+X_1+X_2+0.5\epsilon_M,\qquad Y=1+A+0.2M+X_1+X_2+0.5\epsilon_Y.

An intervention on $A$ shifts the mediator mean by 0.2. Each additional unit of $M$ shifts the outcome mean by 0.2 at fixed treatment. The population NIE is therefore $0.2\times0.2=0.04$.

$X_1$ $X_2$ $A$ $M$ $Y$
2.0785 1 1 4.2548 5.7718
−0.5509 1 1 1.8781 2.5719
0.2774 0 1 1.2213 2.1576

Download the observed CSV · Generator · Example runner

For a quick check, run the adjusted linear product estimator on the example. This fits an OLS regression for each path and uses 1,000 row-bootstrap draws for its percentile interval. From a Python 3.10 environment:

python -m pip install numpy "huggingface_hub==2.4.0"
hf download shibo769/MediPFN run_example.py generate_example.py arena_rf_reference.py arena_n500/requirements-research.txt --local-dir medipfn_example
cd medipfn_example
python run_example.py --method linear

To run the released MediPFN predictor and the leaderboard's random-forest estimator on these same observations:

python -m pip install -r arena_n500/requirements-research.txt
python run_example.py --method medipfn
python run_example.py --method arena-rf

MediPFN downloads the pinned backbones on first use. The RF analysis uses 200 trees, five outer folds and 50 bootstrap replicates. Each command reports the estimate, 95% interval and error against the same known NIE. The five-fold leaderboard aggregates a separate collection of 2,000 tables.

Method on this example NIE estimate 95% interval Absolute error from 0.04
MediPFN, released two-fold predictor 0.0373 [0.0001, 0.0745] 0.0027
Arena random forest, five folds 0.0015 [−0.1231, 0.2601] 0.0385
Adjusted linear product estimator 0.0348 [0.0121, 0.0651] 0.0052

To document another data design, copy the submission template. To enter your estimator, follow the mediation submission instructions.

Checkpoints

File Contents
Arena method ZIP Complete MediPFN inference bundle with the files below, code, and a synthetic example
Mediation adapter Trained adaptation for nuisance prediction
Error head Fine-tuned model for quantiles of estimation error
Calibration Independently fitted interval adjustments

The Quick Start also downloads the official classifier and regressor. Prior Labs TabPFN-v3 weights

Quick Start

Use Python 3.10. The first run downloads about 446 MB of official TabPFN-v3 weights from Prior Labs. Both Hugging Face repositories support anonymous downloads.

python -m pip install "huggingface_hub==2.4.0"
hf download shibo769/MediPFN arena_n500/requirements-research.txt --revision 315046cf9838ebcf2b920dad0f87e95bbb5bbc04 --local-dir medipfn_hf
python -m pip install -r medipfn_hf/arena_n500/requirements-research.txt

Save the following as quickstart.py and run python quickstart.py. The example downloads a fixed MediPFN release and the two matching TabPFN-v3 checkpoints, then scores the included synthetic table on CPU.

import json
from pathlib import Path
import sys

from huggingface_hub import hf_hub_download, snapshot_download

model_dir = Path(snapshot_download(
    repo_id="shibo769/MediPFN",
    revision="315046cf9838ebcf2b920dad0f87e95bbb5bbc04",
    allow_patterns=["arena_n500/**"],
    ignore_patterns=["arena_n500/MODEL_CARD.md"],
    local_dir="medipfn_hf",
    token=False,
)) / "arena_n500"

backbone_repo = "Prior-Labs/tabpfn_3"
backbone_revision = "e923ba9be85784206c9e2f43b0035c84d5fd5747"
classifier, regressor = [
    hf_hub_download(
        repo_id=backbone_repo,
        filename=f"tabpfn-v3-{role}-v3_default.ckpt",
        revision=backbone_revision,
        token=False,
    )
    for role in ("classifier", "regressor")
]

sys.path.insert(0, str(model_dir.resolve()))
from predict import Predictor, read_observed

predictor = Predictor(classifier, regressor, device="cpu", threads=4)
observed = read_observed(model_dir / "example" / "observed_n500.npz")
result = predictor(observed, fold_seed=18031)
print(json.dumps({
    "estimate": result["estimate"],
    "standard_error": result["standard_error"],
    "intervals": result["intervals"],
}, indent=2))

With the example seed, the NIE is approximately 0.2457, the score standard error is 0.0634, and the 95% confidence interval is [0.064, 0.427].

For Arena, initialize Predictor once and reuse it across tables. Pass the evaluator-assigned fold_seed on each call. Supply an observed 500-row table with x, a, m, y, x_types, and mediator_types.

See the input specification for the table format, model details for identification assumptions, and license and attribution for usage terms. NOTICE.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for shibo769/MediPFN

Adapter
(1)
this model

Paper for shibo769/MediPFN