• Markets and agents do the “same kinds of things” (i.e. have the same “type signature”): they compute decisions (in the form of goods allocation) and beliefs (via e.g. prediction markets).
• Beliefs may be inconsistent (arbitrage) or incomplete. In fact, they must be incomplete, due to a simple counting argument.
• “Rationality” and “Efficient Market Hypothesis”. Both markets and boundedly rational agents are only “rational given the available algorithmic information”[¹].
• Markets have self-awareness: they can contain prediction markets for the market’s own properties (e.g. “the market probability of XX prediction market will be >0.7“)¹ and also can self-improve, because the algorithmic functionality of markets are also provided by some goods (e.g. “the labour supply of arbitrageurs”) and this supply can be adjusted and optimized.
• Abstraction is like economies of scale.
• Transaction costs are like computational costs.
• Markets are general intelligence: they can be asked (paid) to accomplish any task. In particular, they are intelligent about managing this generality: about forming abstractions (factor goods) that will be useful for the distribution of tasks they expect to be asked to accomplish.

The basic point is that

1. Markets are boundedly rational agents
2. You can actually get intelligence as an emergent property of markets (rather than the intelligence being just from the intelligence of the individual participants in the market). So you could imagine just building a market out of really “dumb” / simple programs, and the resulting market would be an intelligent agent.

The first point is clearly true; the latter could be more controversial. But (1) the idea of a multi-agent basis of mind/intelligence is not original to me, and has precedent in the literature — see “Related work” for details (2) this is intuitively very plausible, right? Think e.g. the “I, Pencil” essay: no human is intelligent / capable enough to single-handedly produce a pencil, yet the market distributes this highly complex task amongst many dumb humans. Specifically, markets achieve this level of distributedness via two methods:

1. Modularity of action — instead of a single agent that completes the entire task \(\alpha:\Omega\to\Omega\), this action is decomposed into many smaller tasks \(\Omega\xrightarrow{\alpha_1}\Omega\xrightarrow{\alpha_2}\dots\xrightarrow{\alpha_n}\Omega\), with each little step optimized separately (by selecting the best agent for the subtask).
2. Modularity of state — instead of handing the entire state \(\omega\) to an agent to transform it, we factor the state into dimensions called “goods”: \(\Omega=\Omega_1\times\dots\Omega_m\). See “What are markets? The math” for details.

Markets serve as an example of “already-existing (aligned) AGI”, so they serve as a nice setting for us to play with concepts of intelligence and alignment.

So even if we don’t want to build market-based agents, markets serve as a valuable intuition pump for the AI agents we do build.

Some interesting properties of markets / concepts of intelligence in the context of markets —

Some excerpts from a paper I have in progress. This “definition of a market” is valid in a setting that subsumes any MDP (proof left as an exercise), and subsumes the existing market-based RL toys listed in “Related work” (proof left as an exercise).

Observe that there is no fixed market graph or structure — this graph is generated/learned on the fly. The class of agents is not specified, and should be chosen so that a universal approximation theorem exists.

We may also consider markets where instead of transacting the entire “state” through the economy, the state is modularized into “goods” that are transacted separately. This way, agents can specialize not only on activation-conditions and actions, but also on what parts of the state to observe and act on.

(For neatness we’re pretending that there is some “general equilibrium computation algorithm” Equ — in reality you could imagine other mechanisms, e.g. asking for independent demand schedules for each good and computing their equilibria independently, so that the job of marginalizing their demand schedules on their estimated prices of other goods are left to the agents. You might generally have other mechanisms, requiring perhaps different signals from the agents, and it’s not clear to me exactly how to “abstract this out”. Perhaps the mechanism itself should be a good, IDK.)

DO NOT INCLUDE

Markets are a real-life ASI. What does this mean?

…

they receive continuous signal—which avoids misgeneralization

corporations “know their place”

Much of this is based on the taxonomy in Distinguishing AI takeover scenarios

Corrigibility

Reward-hacking / wireheading

Goodhart’s law

Influence seeking behaviour

Paperclip maximizer

Inner misaligned paperclip maximizer

A brief bibliography of work in this area I’m aware of:

Classifier systems and citations thereof

• Holland, J (1986). Escaping brittleness: The possibilities of general purpose learning algorithms applied to parallel rule-based systems.
• Baum, E. B. (1999). Toward a Model of Intelligence as a Laissez-Faire Economy of Idiots.
• Kwee, I., Hutter, M., and Schmidhuber, J. (2001) Market-Based Reinforcement Learning in Partially Observable Worlds.
• Chang, M., Kaushik, S., Weinberg, S. M., Griffiths, T., and Levine, S. (2020) Decentralized Reinforcement Learning: Global Decision-Making via Local Economic Transactions

Garrabrant induction and citations thereof

• Garrabrant et al. (2016) Logical Induction.
• Oesterheld et al. (2023). A theory of bounded inductive rationality.

“Philosophical” commentary

• Minsky (1986), A society of mind.
• Oesterheld (2017), Futarchy implements evidential decision theory.
• Agoric Computation

So market-based agents are a promising idea for AGI, and we should develop AGI in the form of marketic agents. There are at least three schticks missing in this line of argumentation:

• So why haven’t marketic agents approaches, like those in “related work”, been a revolution already?
• Goods are an important part of real-world markets; how do you actually get “goods” in a general RL context?
• Do we really know that marketic agents — say with human “consumers” as the model of human feedback instead of RLHF or whatever — are really safe? Yes, real-life markets do great things, but we can’t just reason by analogy.

I think the main obstacle for getting marketic agents to work is that information markets are inefficient. They’re subject to buyer’s inspection/imperfect information and positive externalities (that’s the issue with prediction markets — sure, you can just subsidise one market with your “price for information” — but how can agents subsidize “subsidiary markets” that will help them in their forecast on your market, without creating positive externalities for someone else?). The Baum paper noticed this too — that you can’t have agents communicate useful information to other agents, like “in this scenario, don’t do X”.

This problem needs solving. Perhaps with an information bazaar (LLMs can act like a ]], so they can just inspect the information, make a purchase decision, then forget the information), or perhaps with "[[https://www.lesswrong.com/posts/ufW5LvcwDuL6qjdBT/latent-variables-for-prediction-markets-motivation-technical][latent variable prediction markets". Maybe combinatorial prediction markets are a relevant concept — is anyone familiar enough with the literature to tell me if they can do latent variable discovery?

A good starting to-do for this project will look something like this:

• [ ] Write a Python module for working with generic marketic agents experiments. Stuff like the classifier systems papers should be easily reproducible as example runs.
• [ ] Figure out where goods come from. I think the answer has to do with the “actions” in the RL tuple, assigning rights to them and contracts over those rights.
• [ ] Write a LessWrong post titled “Intuition pumping for alignment” exploring what common alignment failure modes would look like with markets, and why they do not show up with markets (or if they do, how we can prevent them — I suspect this will be a lot easier with markets than with other AI models — something like “land value tax + antitrust” would do it).
• [ ] Make some basic theoretical connections like:
  • [ ] How markets can subsume a POMDP setting — perhaps this will also shed light on questions about information, and about different market mechanisms which involve eliciting different signals from agents.
  • [ ] The Reward Hypothesis is false is a nice reading on multi-objective MDPs, btw.
  • [ ] Explain what a universal approximation theorem would look like, and some kind of optimality theorem though it’s unclear how to choose a natural “discount factor” (interest rate).
• [ ] Solve the information market issue
  • [ ] Explore information bazaar. It’s a novel idea with a lot of un-explored applications IMO. It can be applied to prediction markets subsidizing subsidiaries, to RLHF alternatives, to recommender systems, etc.
  • [ ] Explore latent variable prediction markets.
  • [ ] Think about whether the below definition of information markets makes any sense.

This will be a big project — I’m literally demanding that all AI work everywhere be reoriented towards marketic agents, and the TO-DO above is the bare minimum I need to do to produce valuable results to even convince anyone marketic agents is even a worthwhile research area.

If you think this is something you can meaningfully work on (if reading this post “struck a chord”, with related thoughts you’ve had before, if you have an economist’s intuitions), please reach out.

This project will be the greatest human achievement since the invention of agriculture. If we succeed, we will solve not only AI alignment, but also basically all of economics as well as philosophy.