Abhimanyu Pallavi Sudhir

Formal education

Internships

Research

Markets and AI (PhD work)

My primary work focuses on topics at the intersection of markets and AI, especially AI alignment.

Related write-ups and talks.

Consistency checks and forecasting (2024)

Developed a Consistency Benchmark for LLM forecasters, along with a principled arbitrage-based metric for inconsistency and a novel consistency calibration method similar to Platt scaling.

Scalable Oversight Benchmark (2024)

Ongoing collaboration with a team supervised by Arjun Panickssery and Nina Rimsky, to develop a comprehensive benchmark for Scalable Oversight protocols.

General mathematics (Undergraduate work and prior)

Academic service

Courses and workshops attended

Other projects

Costly (2024)

Wrote the Python package costly for estimating costs and running times of complex LLM workflows/experiments/pipelines in advance before spending money, via simulations.

Project page: github.com/abhimanyupallavisudhir/costly

Install: pip install costly

Equivariant learning (2021-22)

Final-year MSci project with Professor Jeroen Lamb at Imperial College London exploring equivariant learning and causal DAGs.

Report: abhimanyu.io/legacy_writing/Imperial_reports/m4r.pdf

Lie theory (2019)

Undergraduate research project with Professor Richard Thomas at Imperial College London on Lie groups and algebras.

Report: abhimanyu.io/legacy_writing/Imperial_reports/urop.pdf

Presentation: abhimanyu.io/legacy_writing/Imperial_presentations/lie_theory.pdf

Lean (2018-19)

Computerized formal proving in Lean with Professor Kevin Buzzard at Imperial College London.

PhysicsOverflow (2014-15)

Co-founded PhysicsOverflow, a postgraduate-level physics Q&A site and open peer review system. See en.wikipedia.org/wiki/PhysicsOverflow for more details.