Intelligence is Collective, Not Artificial: Prof. Michael I. Jordan's Perspective on AI and Economics

The Fallacy of AGI and the Anthropomorphizing of Intelligence

Artificial General Intelligence (AGI) is largely a PR term that creates a distortionary effect on research, business models, and the aspirations of young builders. The current narrative—that we are building a disembodied superintelligence that may either save or wipe out humanity—is science fiction that demoralizes young engineers by suggesting there is nothing left to build in their lifetime.

Intelligence should not be anthropomorphized. The goal of technical systems is not to "understand" in a human sense, but to provide predictability and control. For example, in industrial supply chain modeling, a system can make fantastic predictions about shipping delays without "understanding" logistics; what matters is that it reduces uncertainty and allows for effective engineering and planning.

AI as a Collective Economic System

Intelligence is not an individual property but a collective one. Human intelligence emerges from the aggregation of opinions, thoughts, and cultures that provide the necessary context for action. Consequently, AI should be viewed as a collective economic system rather than a standalone statistical box.

The Role of Economics and Game Theory

To move beyond simple pattern matching, AI must incorporate economic thinking, specifically game theory and mechanism design:

• Game Theory: A mathematical framework for predicting outcomes based on strategic interactions (similar to how $F=ma$ predicts physical motion).
• Mechanism Design: The "inverse" of game theory. Instead of predicting the outcome of a given game, it asks: "What game must I design to achieve a specific desired outcome (e.g., fairness, wealth distribution, or market efficiency)?"
• Contract Theory: A subset of mechanism design dealing with information asymmetry, where one party knows more than the other, requiring specific incentives to ensure truthful interaction.

Technical Limitations of Foundation Models

While Large Language Models (LLMs) exhibit impressive scaling, they are often detached from reality because they lack a framework for uncertainty and incentives.

The AlphaFold Case Study and Prediction-Powered Inference

AlphaFold is a powerful tool for protein prediction, but it can be misleading when used for hypothesis testing. In research regarding quantum fluctuations and phosphorylation, it was found that AlphaFold's predictions could produce extremely narrow confidence intervals that were far from the truth because the training set lacked sufficient examples of those specific fluctuations.

To solve this, Prediction-Powered Inference (PPI) was developed. PPI allows researchers to merge a small amount of ground-truth data with a large volume of biased model predictions to shift the error bars so they cover the truth, providing a scientifically trustable answer even when using a biased foundation model.

The Problem with LLM Confidence

LLMs generally cannot quantify their own confidence because they are trained solely to predict the next word. When an LLM expresses confidence, it is typically mimicking human assertions found in its training data rather than performing epistemic reasoning under uncertainty.

Designing Sustainable Data Markets

Data is not merely a resource for training models but a commodity with value and privacy implications. A three-layer data market model illustrates the tension between users, platforms, and data buyers:

1. Users: Provide data to platforms in exchange for services.
2. Platforms: Use data to improve services and sell it to third parties to remain viable.
3. Data Buyers: Purchase data for market research and behavioral studies.

In an optimized system, platforms would offer tunable levels of differential privacy. Users who value privacy would choose platforms with higher privacy budgets, while data buyers would pay less for that noisier data. The goal is to find a mathematical equilibrium that maximizes social welfare rather than simply optimizing a single objective function.

A New Liberal Arts Triangle for the AI Era

To build societally responsible technology, the next generation of researchers should move away from a pure focus on computational optimization and instead adopt a multidisciplinary "triangle" of thinking:

• Computational Thinking: Modularity, abstractions, and algorithmic efficiency.
• Inferential Thinking (Statistics): Managing uncertainty, controlling error rates, and designing optimal experiments.
• Economic Thinking: Understanding incentives, strategic interactions, and the design of mechanisms to coordinate human and machine agents.

By combining these three, AI can move from being a "secretary on your shoulder" to a system that improves human decision-making, mitigates uncertainty (like markets do for resources), and supports human creativity and democracy.