gliner_datause_extended

Fine-tune of urchade/gliner_large-v2.1 for data-use mention extraction (dataset / survey / census / registry mentions in economics research papers).

Labels

• NAMED_DATA — a proper name, title, or acronym of a specific data source
• DESCRIPTIVE_DATA — a source described in words but not named
• VAGUE_DATA — generic data wording with no identifiable source

Training

• base model: urchade/gliner_large-v2.1
• dataset: rafmacalaba/data-use-mentions-extended (gliner config)
• epochs: 5
• learning rate: 5e-06
• batch size: 16
• precision: bf16

Evaluation (holdout)

thr	tp	fp	fn	precision	recall	f0.5	f1
0.10	12283	8280	281	0.5973	0.9776	0.6477	0.7416
0.20	12192	6065	372	0.6678	0.9704	0.7122	0.7911
0.30	12062	4883	502	0.7118	0.9600	0.7506	0.8175
0.40	11845	3857	719	0.7544	0.9428	0.7858	0.8381
0.50	11498	2837	1066	0.8021	0.9152	0.8224	0.8549
0.60	10519	1798	2045	0.8540	0.8372	0.8506	0.8455
0.70	8328	892	4236	0.9033	0.6628	0.8422	0.7646

Best F0.5: 0.8506 (thr=0.6) Best F1: 0.8549 (thr=0.5)

NER holdout comparison

device: NVIDIA H100 NVL

rafmacalaba/data-use-mentions-extended (n=9249)

model	backend	best F0.5	thr	best F1	thr	wall-clock (s)	texts/s
rafmacalaba/gliner_datause_extended	gliner	0.8506	0.6	0.8549	0.5	202.7	45.6
ai4data/gliner2_datause	gliner2	0.8634	0.7	0.8624	0.6	180.8	51.1

F0.5 by threshold (sweet spots side-by-side):

thr	rafmacalaba/gliner_datause_extended	ai4data/gliner2_datause
0.1	0.6476	0.7321
0.2	0.7121	0.7712
0.3	0.7505	0.7979
0.4	0.7858	0.8201
0.5	0.8224	0.8363
0.6	0.8506	0.8523
0.7	0.8422	0.8634

FCV cross-corpus validation (precision)

Validated on the World Bank Fragility, Conflict & Violence (FCV) document corpus with the gpt-5.6-luna judge (Luna validation). Precision-only: there is no gold set of all true mentions, so recall / F1 are not reported here.

Precision by config × label

config	NAMED_DATA	DESCRIPTIVE_DATA	VAGUE_DATA	overall
fcv_pads_east_asia	80.1%	81.6%	91.8%	81.9%
jdc_operational	86.8%	91.3%	91.5%	89.1%
refugee_pads	83.8%	92.8%	91.9%	89.1%
reliefweb	83.7%	85.4%	95.5%	85.6%

Specificity agreement (extractor label == judge specificity)

config	NAMED_DATA	DESCRIPTIVE_DATA	VAGUE_DATA
fcv_pads_east_asia	82.8%	88.3%	82.1%
jdc_operational	90.2%	83.2%	71.7%
refugee_pads	89.4%	86.5%	77.5%
reliefweb	92.6%	82.5%	73.4%

Notes:

• VAGUE_DATA is the safest label (91–95% precision); NAMED_DATA is the weakest (80–87%).
• The extractor under-specifies: VAGUE_DATA is often judged descriptive, and DESCRIPTIVE_DATA → named leaks occur.
• reliefweb carries more bad-JSON judge errors (OCR/HTML artifacts) than the other configs.