Abhimanyu Pallavi Sudhir

Formal education

Experience

Research

Markets and AI (PhD work)

My primary work focuses on topics at the intersection of markets and AI, especially AI alignment.

Consistency checks and forecasting (2024)

Developed a Consistency Benchmark for LLM forecasters, along with a principled arbitrage-based metric for inconsistency and a novel consistency calibration method similar to Platt scaling.

Scalable Oversight Benchmark (2024–25)

Created a benchmark for evaluating human feedback mechanisms for how effective they are at incentivizing alignment [code].

General mathematics (Undergraduate work and prior)

Awards, academic service, etc.

Other projects

Pramāṇa (2025) – to Zotero what pass is to Bitwarden.

Wrote pramaana, a minimalist command-line reference and full text manager. Works like Zotero + BetterBibTeX + Zotmoov/Zotfile, but you can just operate on your references with your terminal or file system instead of clicking everywhere in an app.

Project page: github.com/abhimanyupallavisudhir/pramaana

Install: pipx install pramaana

Costly (2024) – the best/only LLM cost estimation library.

Wrote the Python package costly for estimating costs and running times of complex LLM workflows/experiments/pipelines in advance before spending money, via simulations.

Project page: github.com/abhimanyupallavisudhir/costly

Install: pip install costly

Equivariant learning (2021-22)

Final-year MSci project with Professor Jeroen Lamb at Imperial College London exploring equivariant learning and causal DAGs.

Report: abhimanyu.io/legacy_writing/Imperial_reports/m4r.pdf

Lie theory (2019)

Undergraduate research project with Professor Richard Thomas at Imperial College London on Lie groups and algebras.

Report: abhimanyu.io/legacy_writing/Imperial_reports/urop.pdf

Presentation: abhimanyu.io/legacy_writing/Imperial_presentations/lie_theory.pdf

Lean (2018-19)

Computerized formal proving in Lean with Professor Kevin Buzzard at Imperial College London.

PhysicsOverflow (2014-15)

Co-founded PhysicsOverflow, a postgraduate-level physics Q&A site and open peer review system. See en.wikipedia.org/wiki/PhysicsOverflow for more details.