Abhimanyu Pallavi Sudhir

Formal education

Experience

Research

Markets and AI (PhD work)

My primary work focuses on topics at the intersection of markets and AI, especially AI alignment.

Related write-ups and talks.

Consistency checks and forecasting (2024)

Developed a Consistency Benchmark for LLM forecasters, along with a principled arbitrage-based metric for inconsistency and a novel consistency calibration method similar to Platt scaling.

Scalable Oversight Benchmark (2024)

Ongoing collaboration with a team supervised by Arjun Panickssery and Nina Rimsky, to develop a comprehensive benchmark for Scalable Oversight protocols.

General mathematics (Undergraduate work and prior)

Academic service

Courses and workshops attended

Other projects

Pramāṇa (2025) – the best reference manager

Wrote pramaana, a minimalist command-line reference and full text manager. Works like Zotero + BetterBibTeX + Zotmoov/Zotfile, but you can just operate on your references with your terminal or file system instead of clicking everywhere in an app.

Project page: github.com/abhimanyupallavisudhir/pramaana

Install: pipx install pramaana

Costly (2024) – the best/only LLM cost estimation library

Wrote the Python package costly for estimating costs and running times of complex LLM workflows/experiments/pipelines in advance before spending money, via simulations.

Project page: github.com/abhimanyupallavisudhir/costly

Install: pip install costly

Equivariant learning (2021-22)

Final-year MSci project with Professor Jeroen Lamb at Imperial College London exploring equivariant learning and causal DAGs.

Report: abhimanyu.io/legacy_writing/Imperial_reports/m4r.pdf

Lie theory (2019)

Undergraduate research project with Professor Richard Thomas at Imperial College London on Lie groups and algebras.

Report: abhimanyu.io/legacy_writing/Imperial_reports/urop.pdf

Presentation: abhimanyu.io/legacy_writing/Imperial_presentations/lie_theory.pdf

Lean (2018-19)

Computerized formal proving in Lean with Professor Kevin Buzzard at Imperial College London.

PhysicsOverflow (2014-15)

Co-founded PhysicsOverflow, a postgraduate-level physics Q&A site and open peer review system. See en.wikipedia.org/wiki/PhysicsOverflow for more details.