Post
2579
New dataset: 24,922 real Python dependency compatibility experiments🧪
Every experiment installed a pair of Python packages into a fresh environment and tested three stages: resolution → import → runtime smoke test.
The finding that motivated this: resolvers only read published metadata. In this data, ~3,000 environments resolved successfully and still failed at import or smoke-test time. That gap is invisible to pip and uv, and this dataset makes it measurable.
What's inside:
- 21,490 development + 3,432 sealed validation experiments
- 41 unique packages, Python 3.8–3.14, Linux x86_64
- The 6 validation packages (boto3, botocore, s3transfer, celery, kombu, billiard) never appear in training — validation measures generalization to entirely unseen packages
- Full per-stage logs: exact commands, exit codes, stdout/stderr
- Normalized error text + Python exception types for every failure
- Wheel metadata with Python/ABI/platform tag compatibility decisions
- PEP 751 lockfile hashes
- 142 engineered features per experiment, ready for training
Outcome labels: pass, resolution_failure, import_failure, smoke_test_failure.
I used it to train two models for DepLab (a structured model + a fine-tuned ModernBERT on release notes). On the sealed validation set (entirely unseen packages) the structured model reached 90.0% accuracy and 100% precision when predicting failures.
CC BY 4.0. Load it directly:
Dataset: Abhisek987/deplab-dependency-compatibility
Zenodo DOI: https://doi.org/10.5281/zenodo.21729353
Code: https://github.com/Abhisek12378/DepLab
If you're working on ML for developer tools or software engineering research, I'd love to hear what you build with it.
Every experiment installed a pair of Python packages into a fresh environment and tested three stages: resolution → import → runtime smoke test.
The finding that motivated this: resolvers only read published metadata. In this data, ~3,000 environments resolved successfully and still failed at import or smoke-test time. That gap is invisible to pip and uv, and this dataset makes it measurable.
What's inside:
- 21,490 development + 3,432 sealed validation experiments
- 41 unique packages, Python 3.8–3.14, Linux x86_64
- The 6 validation packages (boto3, botocore, s3transfer, celery, kombu, billiard) never appear in training — validation measures generalization to entirely unseen packages
- Full per-stage logs: exact commands, exit codes, stdout/stderr
- Normalized error text + Python exception types for every failure
- Wheel metadata with Python/ABI/platform tag compatibility decisions
- PEP 751 lockfile hashes
- 142 engineered features per experiment, ready for training
Outcome labels: pass, resolution_failure, import_failure, smoke_test_failure.
I used it to train two models for DepLab (a structured model + a fine-tuned ModernBERT on release notes). On the sealed validation set (entirely unseen packages) the structured model reached 90.0% accuracy and 100% precision when predicting failures.
CC BY 4.0. Load it directly:
from datasets import load_dataset
ds = load_dataset("Abhisek987/deplab-dependency-compatibility", "experiments")Dataset: Abhisek987/deplab-dependency-compatibility
Zenodo DOI: https://doi.org/10.5281/zenodo.21729353
Code: https://github.com/Abhisek12378/DepLab
If you're working on ML for developer tools or software engineering research, I'd love to hear what you build with it.
- 7 replies
