OpenAI Governance of Superintelligence Proposal

OpenAI has proposed a framework for the governance of superintelligence to ensure that the development of highly capable AI systems is safe and integrated smoothly into society. The proposal focuses on coordination among developers, the establishment of an international regulatory authority, and the pursuit of technical safety research.

International Coordination and Development Rates

To maintain safety and ensure societal integration, OpenAI suggests that leading AI development efforts must coordinate. This coordination could take several forms, including the integration of current efforts into a government-led project or a collective agreement to limit the rate of growth in AI capabilities at the frontier to a specific annual rate.

OpenAI emphasizes that individual companies must be held to an extremely high standard of responsible action during this process.

Establishment of a Global Regulatory Authority

OpenAI proposes the creation of an international authority, similar to the International Atomic Energy Agency (IAEA), to oversee superintelligence efforts. This agency would be triggered by systems exceeding a certain capability or compute threshold.

Key functions of this proposed agency would include:

• Inspecting systems and requiring audits.
• Testing for compliance with safety standards.
• Placing restrictions on deployment levels and and security levels.
• Tracking compute and energy usage to ensure implementability.

The proposal specifies that this agency should focus exclusively on reducing existential risk rather than addressing issues that individual countries should manage, such as content moderation policies (defining what an AI should be allowed to say).

Technical Safety and Public Oversight

Ensuring the safety of superintelligence is currently an open research question. OpenAI and other organizations are investing significant effort into solving this technical challenge to make superintelligence safe.

Beyond technical safety, the governance of powerful systems and their deployment decisions must involve strong public oversight. OpenAI advocates for a democratic process to determine the bounds and defaults for AI systems, while maintaining that individual users should retain significant control over AI behavior within those established bounds.

Rationale for Developing Superintelligence

OpenAI identifies two primary reasons for continuing the development of this technology despite the risks:

1. Societal Benefit: The technology is expected to lead to significant economic growth, increased quality of life, and solutions to complex global problems in areas such as education, creative work, and personal productivity.
2. Inevitability: OpenAI argues that stopping the creation of superintelligence would be "unintuitively risky and difficult." Due to the decreasing cost of development, the increasing number of actors involved, and the current technological trajectory, stopping it would require a global surveillance regime that might still fail. Therefore, the focus must be on getting the development right.