AdarshSingh7647/TabRankMultiTableCoTCond

TabRank — Single + Multi Table (our method, flagship)

Part of the TabRank family: six Qwen3-8B checkpoints for single-call generative listwise table reranking. Given a question and a list of candidate tables, the model reads them all in one prompt and returns the full ranking in a single generation — no pairwise scoring, no cross-encoder passes.

This is the flagship TabRank checkpoint — our reasoning-conditioned method, trained on the largest data mix (NQ Tables + MultiTabQA). It has the best mean score in the family, both in-distribution and across 7 out-of-distribution benchmarks it never saw in training, beating Base Qwen3-8B on 5 of 7. If you only try one model from this collection, this is the one to start with.

Related checkpoints: TabRankMultiTableCoTGen (Standard SFT, same data mix) and TabRankSingleTableCoTCond (same method, single-table data only, smaller/faster to train on). Full family: TabRankSingleTableNaive · TabRankSingleTableCoTGen · TabRankSingleTableCoTCond · TabRankMultiTableNaive · TabRankMultiTableCoTGen · TabRankMultiTableCoTCond (this model).

How it works

TabRank is trained on 6,728 chain-of-thought reasoning traces distilled from a teacher model reasoning about table relevance. Rather than forcing the student to imitate the teacher's exact reasoning text — which tends to overfit to the teacher's phrasing and generalize poorly — this checkpoint conditions on the reasoning as context during training and learns its own, shorter internal reasoning at inference time. Combined with the larger NQ Tables + MultiTabQA training mix, this is the configuration that generalized best in our experiments, including to table-retrieval benchmarks and domains never seen during training. Full method details, ablations, and the reasoning-trace dataset construction are in the paper.

Input / output format

Input — a chat message with the question followed by each candidate table, labeled ### Table 1, ### Table 2, ...:

Question: Which table shows 2022 quarterly revenue by region?

### Table 1
| Region | Q1 2022 | Q2 2022 | Q3 2022 | Q4 2022 |
|---|---|---|---|---|
| North America | 120 | 134 | 128 | 145 |
| Europe | 88 | 91 | 95 | 102 |

### Table 2
| Product | Units Sold | Year |
|---|---|---|
| Widget A | 4200 | 2021 |

### Table 3
| Region | Headcount |
|---|---|
| North America | 340 |

Output — a <think> block with the model's reasoning, followed by a single JSON object with the ranked, one-indexed candidate positions, best first:

<think>
Table 1 has quarterly revenue by region for 2022, which is exactly what the question asks
for. Table 3 has region data but no revenue. Table 2 has neither region nor 2022 data.
</think>
{"ranked_tables": [1, 3, 2]}

Map the numbers back to your own table ids to get the reranked list — position 1 in the output is ### Table 1 from the input, etc.

Evaluation — this checkpoint's results

Scored as a listwise reranker reordering a first-stage top-25 candidate list on 5 in-distribution benchmarks (SQA, TAT-QA, HybridQA, TabFact, and NQ-Tables — the actual LoRA training split) and 7 out-of-distribution benchmarks from the IBM table-text-ir-evaluation suite (OpenWikiTables, OTT-QA, MultiHiertt, AIT-QA, FeTaQA, StatCanDialogue, WatsonxDocsQA) that this model never saw during training.

This checkpoint (TabRank), ndcg@10:

	SQA	TAT-QA	HybridQA	TabFact	NQ-Tables	OpenWikiTables	OTT-QA	MultiHiertt	AIT-QA	FeTaQA	StatCanDialogue	WatsonxDocsQA	Mean
TabRank	0.741	0.519	0.783	0.688	0.747	0.938	0.903	0.599	0.536	0.919	0.580	0.690	0.720

Compared against Base Qwen3-8B and Standard SFT on the same data mix:

Model	SQA	TAT-QA	HybridQA	TabFact	NQ-Tables	OpenWikiTables	OTT-QA	MultiHiertt	AIT-QA	FeTaQA	StatCanDialogue	WatsonxDocsQA	Mean
Base Qwen3-8B	—	—	0.735	0.656	0.723	0.887	0.813	0.521	0.495	0.896	0.615	0.756	0.710
Standard SFT	0.736	0.540	0.791	0.670	0.735	0.903	0.832	0.537	0.506	0.881	0.585	0.679	0.700
TabRank (this model)	0.741	0.519	0.783	0.688	0.747	0.938	0.903	0.599	0.536	0.919	0.580	0.690	0.720

The first 5 columns (SQA through NQ-Tables) are in-distribution; the remaining 7 are out-of-distribution. TabRank has the best overall mean and wins 5 of 7 out-of-distribution benchmarks; Base Qwen3-8B edges it out on StatCanDialogue and WatsonxDocsQA specifically. Margins here are modest by design — this is a fair, matched comparison run on the same eval harness with normal output-parsing success rates for all three models (no failure-rate caveat needed on these numbers).

On acc@10 (strictest metric — every gold table must land in the top 10) against the 4 in-distribution benchmarks, this checkpoint improves over base Qwen3-8B by +30.5% on HybridQA, +15.2% on SQA, +52.9% on TabFact, and +13.1% on TAT-QA (see the paper, Table 2).

Source eval code and logs: GitHub repo.

Usage with vLLM

from vllm import LLM, SamplingParams
from transformers import AutoTokenizer

repo = "AdarshSingh7647/TabRankMultiTableCoTCond"
tok = AutoTokenizer.from_pretrained(repo)
llm = LLM(model=repo, dtype="bfloat16", max_model_len=32768)

system = ("You are a table relevance expert. Given a question and a set of candidate tables "
          "rank them from most to least useful for answering the question. Reason in a "
          "<think>...</think> block then output exactly JSON with key ranked_tables.")
user = "Question: ...\n\n### Table 1\n...\n\n### Table 2\n...\n"

msgs = [{"role": "system", "content": system}, {"role": "user", "content": user}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
# the model writes a <think> block then the ranking json
out = llm.generate([text], SamplingParams(temperature=0.6, top_p=0.95, max_tokens=8192))
print(out[0].outputs[0].text)

Usage with Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "AdarshSingh7647/TabRankMultiTableCoTCond"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16, device_map="auto")

system = ("You are a table relevance expert. Given a question and a set of candidate tables "
          "rank them from most to least useful for answering the question. Reason in a "
          "<think>...</think> block then output exactly JSON with key ranked_tables.")
user = "Question: ...\n\n### Table 1\n...\n\n### Table 2\n...\n"

msgs = [{"role": "system", "content": system}, {"role": "user", "content": user}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=8192, temperature=0.6, top_p=0.95, do_sample=True)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

For full training and evaluation code, dataset builders, and the reasoning-trace dataset, see the TabRanker GitHub repo.

Model details

• Base model: Qwen3-8B
• Method: LoRA rank 16 fine-tuning, merged into the base weights so it loads directly
• Precision: bfloat16, single-file safetensors, ~16 GB
• Training data: NQ Tables + MultiTabQA (single- and multi-table retrieval)
• Family: six TabRank checkpoints span three objectives (Answer-Only, Standard SFT, TabRank) across two training mixes (Single Table, Single + Multi Table)

Citation

If you use these models, please cite the TabRank paper:

@misc{singh2026tabrank,
      title={TabRank: Chain-of-Thought Distillation for Table Re-Rankers},
      author={Adarsh Singh and Kushal Raj Bhandari and Jianxi Gao and Soham Dan and Vivek Gupta},
      year={2026},
      eprint={2607.25182},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2607.25182}
}

The MultiTabQA data in this checkpoint's training mix comes from RAG over Tables. If your usage relies specifically on the multi-table data, please also cite:

@misc{zou2025ragtableshierarchicalmemory,
      title={RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarking},
      author={Jiaru Zou and Dongqi Fu and Sirui Chen and Xinrui He and Zihao Li and Yada Zhu and Jiawei Han and Jingrui He},
      year={2025},
      eprint={2504.01346},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2504.01346}
}