OpenAI Codex: A GPT-3 Descendant for Code Generation

OpenAI Codex is a generative AI model designed to translate natural language intent into executable code. By combining natural language understanding with a massive dataset of public source code, Codex allows users to issue commands in English to software with an API, reducing the barrier to entry for programming.

Technical Foundation and Language Support

OpenAI Codex is a descendant of GPT-3, trained on a combination of natural language and billions of lines of source code from publicly available sources, including public GitHub repositories. While it is most capable in Python, the model is proficient in over a dozen other languages, including JavaScript, Go, Perl, PHP, Ruby, Swift, TypeScript, and Shell.

Compared to GPT-3, Codex features a significantly expanded memory for Python code, with a 14KB memory capacity compared to GPT-3's 4KB. This allows Codex to process over three times as much contextual information when performing a task.

Capabilities and Programming Tasks

Codex is a general-purpose programming model capable of being applied to a wide range of programming tasks. Its primary strength lies in mapping simple problems to existing libraries, APIs, or functions, which OpenAI describes as the least enjoyable part of programming.

Key applications of the model include:

• Transpilation: Converting code from one language to another.
• Code Explanation: Translating code into natural language to explain its function.
• Code Refactoring: Restructuring existing code to the optimize it.

Accessibility and Deployment

OpenAI is releasing Codex in private beta via an API, offering it for free during the initial period. The deployment follows the safety groundwork established with GPT-3, involving a review of applications and incremental scaling to understand the real-world impact of the technology.