kunal jadhav

SNIPER-SQL ranks #1 globally on the BIRD-CRITIC-1.0-Open leaderboard (570 multi-dialect tasks) with a database-free validate-and-repair pipeline: dialect-strict chain-of-thought generation, adaptive decoding, and a static schema-aware SQL validator.

It generalises across six large language models from five model families, spanning 17B to 235B parameters, and raises SQL repair success for both frontier and open-weight models, with paired significance testing.

Further details and the paper will be released once the paper is accepted.

Literature-grounded question answering requires a system to retrieve the right paper, locate the right evidence within it, and generate an answer faithful to that evidence. These are three separate stages, each with its own failure mode. We present a stage-wise diagnostic study of this pipeline on LitTraceQA, a shared task requiring paper retrieval, coarse evidence localization (table/figure/page), and structured answer generation. Our pipeline combines cross-model hypothesis diversity for retrieval, a filter-then-rank evidence localizer (caption-pattern match, embedding similarity, evidence-type gate), and cross-model semantic entropy for generation-time answer aggregation and confidence estimation. We use an oracle-substitution ablation ladder, successively handing the pipeline the gold paper and then the gold evidence value, and reading off the resulting change in each stage’s own metric. These are sequential oracle-intervention gains, not an additive error decomposition: the stages are measured with different quantities and are not independent. Read that way, the ladder points at evidence localization as the dominant remediable bottleneck — multiple-choice accuracy rises from 0.42 with predicted evidence to 0.76 with the gold evidence value, and end-to-end accuracy stays near 0.29 even on the questions where retrieval already returned every gold paper. We also test whether cheap uncertainty signals computed at retrieval time predict which stage will fail before it does. Along the way we surface two infrastructure-level findings with direct implications for anyone building literature-grounded QA systems. First, evidence page numbers in this and similar benchmarks are keyed to a paper’s canonical published version rather than its arXiv preprint: a real failure mode when a paper has been substantively revised, though a systematic paired ablation across 45 questions finds it is an occasional rather than pervasive hazard in this pool. Second, a substantial fraction of papers hosted on OpenReview (ICLR/ICML/NeurIPS) are unreachable by ordinary scripted HTTP requests due to a JS bot-challenge, a venue-correlated failure mode unrelated to any modeling choice. We report our diagnostic protocol, quantitative results with bootstrap confidence intervals given the benchmark’s small public dev set, and a qualitative failure taxonomy, framing this as a resource and lessons-learned contribution rather than a leaderboard claim. All multiple-choice numbers are a best-case diagnostic condition: they assume access to the gold option text, which a genuine blind submission would not have.

Dense retrieval embeds each query as a single vector, inheriting the failure modes of whichever language model generates the representation. HyDE partially addresses this by embedding a generated hypothetical answer rather than the raw question, but the single-generator design is brittle: a confident hallucination deflects the query vector away from relevant passages, and on multi-hop benchmarks four of the five individual HyDE variants we tested degrade retrieval below the direct-query baseline (the fifth is roughly at parity).

We propose Cross-Model Hypothesis Aggregation (CMHA), which queries N diverse LLMs from distinct model families, embeds each generated hypothesis, and uses the normalized centroid as the final query vector. Per-model errors are geometrically uncorrelated across families, so centroid averaging cancels idiosyncratic mistakes while reinforcing shared factual content. On BEIR benchmarks with BAAI/bge-large-en-v1.5, CMHA with N=4 achieves Recall@10 of 0.8242 on Natural Questions (+8.9pp over direct query) and 0.8166 on HotPotQA, recovering a 7.7pp single-model HyDE degradation; pairing cross-model diversity with K=3 temperature samples per model further improves peak Recall@10 to 0.8313 on NQ and 0.8328 on HotPotQA. All headline gains are statistically robust (bootstrap 95% CIs excluding zero) and hold, with a larger effect size, under a stricter joint-recall metric. We also find, to our knowledge for the first time, that embedding the raw output of chain-of-thought thinking models catastrophically breaks HyDE-style retrieval by up to 68.2pp: unparsed reasoning traces are not document-like text and embed far from the relevant passage region. A diversity-based adaptive routing policy achieves 99.9% of full CMHA recall at only 3.8 average LLM calls per query, making the approach practical for latency-sensitive vector database deployments.

Energy poverty is nearly absent from NLP-for-social-good, and the little existing work is either static retrieval/QA or relies on carbon-intensive cloud LLMs, a self-defeating "computational irony" for a humanitarian setting. We present EqGrid, a closed-loop simulation in which a low-frequency, open-weight LLM policy agent sets price and carbon bounds and targeted subsidies over a community of empirically-grounded household personas, while high-frequency multi-agent RL traders clear a continuous double auction constrained by a physical distribution grid (IEEE-33-bus with Dynamic Operating Envelopes). Our contribution is threefold and directly addresses how to measure the social impact of AI: (i) grounded personas (region-matched socio-demographics) whose load curves are checked for shape and level realism against real smart-meter data; (ii) formal energy-poverty equity metrics (Energy Burden, Gini of EB, LIHC) showing the intervention reduces burden inequality without raising net grid cost; and (iii) a compute-efficiency frontier that measures how much equity performance survives compressing the policy agent from a 235B teacher down to a sub-1B model deployable on a laptop, in estimated energy/carbon per decision. A decoupled-safety design (the LLM sets bounds; a validate-and-project grid gate executes) yields zero grid-constraint violations versus 55 under direct LLM control. On energy-poverty equity, the LLM policy lowers the Gini of energy burden to 0.305 (from 0.351) and mean burden by 28% while cutting cost (outperforming a tuned rule baseline), and a 3B-active model retains 95% of the benefit at roughly 9x lower inference energy than the teacher, with even a 0.8B on-device model retaining 92% at roughly 24x lower energy. We will release code and configs.