Magnexis/lexiconerror-router

LexiconError Router

LexiconError Router is a compact, CPU-friendly diagnostic-routing model trained on Magnexis/lexiconerror-diagnostics. Given an error message, stack trace, compiler diagnostic, or nearby trigger snippet, it predicts the likely programming language, diagnostic category, and severity. It does not generate fixes or execute supplied code.

Evaluation

The model uses a deterministic 80/20 split grouped within each language. Singleton and otherwise unseen labels stay in the training split. Inputs exclude explicit language, category, severity, tool, and source fields to avoid direct metadata leakage.

Head	Accuracy	Macro F1	Weighted F1	Top-3 accuracy
Language	0.998	0.935	0.998	0.999
Category	0.969	0.842	0.969	0.999
Severity	0.988	0.792	0.987	1.000

• Training records: 13,189
• Evaluation records: 3,285
• Dataset SHA-256: 5adb619e064b389f4e81adac9ce5ebb7156dfb63540ea9ae83d7d652c09fb897
• Random seed: 42
• Runtime: scikit-learn 1.9.0, Python 3.13.14

Full machine-readable results are in metrics.json.

Usage

import joblib

bundle = joblib.load("lexiconerror-router.joblib")
text = "error[E0382]: borrow of moved value: `value`"
matrix = bundle["vectorizer"].transform([text])
for head, classifier in bundle["classifiers"].items():
    print(head, classifier.predict(matrix)[0])

Loading a joblib/pickle artifact can execute code. Only load this file from the official Magnexis repository or after verifying SHA256SUMS.txt.

Training data and approval status

All 16,474 structured records were eligible for training. Verified records receive a modest 1.5x sample weight; Needs Review records remain explicitly unapproved and are not misrepresented as editorially verified. The target labels are catalog-routing metadata, not proof that every explanation or remediation is correct.

Limitations

• Registry-derived templates and repeated diagnostic families can make held-out scores optimistic.
• Rare labels may have too little evaluation support for reliable per-class conclusions.
• Confidence values are classifier probabilities, not guarantees of diagnostic correctness.
• The model should route a query into LexiconError; it should not replace official compiler or runtime documentation, security review, or human debugging.
• Upstream data terms vary, so the model uses license: other; review the dataset NOTICE before redistribution.

Reproduction

python modeling\train_router.py --dataset hf\lexiconerror-diagnostics\data\diagnostics.jsonl --output artifacts\model\lexiconerror-router
python modeling\test_router.py