Hacker News Goalposts: Tracking AI Progress and the Moving Target of AGI

The pursuit of Artificial General Intelligence (AGI) is often characterized by a 'moving goalpost' phenomenon, where capabilities once deemed impossible become routine, leading observers to redefine success. A recent community project, "Goalposts," which aggregates historical challenges set by Hacker News users, highlights this evolution and the persistent gap between a model's ability to perform a task once and its ability to perform it reliably.

The Evolution of AI Benchmarks

AI benchmarks have shifted from simple digital tasks to complex, real-world interactions. Analysis of challenges from 2016 through 2026 shows a distinct transition in what the community considers a "hard" problem:

• Early Challenges (2016-2024): Focused primarily on the Turing Test, basic software engineering, and simple agentic actions (e.g., "order me a coffee").
• Recent Challenges (2026): Shifted toward physical world interaction (e.g., "open a physical door"), high-level cognitive empathy ("emulates human pettiness convincingly"), and fundamental scientific discovery ("makes novel scientific breakthroughs").

This shift suggests that while digital mimicry and code generation have been largely internalized as solved, the frontier of AI progress has moved toward world-modeling and autonomous physical agency.

Reliability vs. Stochastic Success

A critical distinction has emerged between a model's capacity to achieve a result through brute force and its ability to perform a task reliably.

"In many cases, I am convinced that a present-day LLM could accomplish the task at least once given an infinite compute budget and an infinite number of tries... But some of these are not routine occurrences, or the model cannot (at present) routinely and reliably complete the task in question."

This gap is evident in several specific failure modes:

• Spatial Reasoning: Despite high vote counts suggesting success, users report that frontier models (such as Opus 5.5 and 5.6 Luna) still struggle with ASCII art, often misidentifying simple shapes (e.g., identifying a foot as a locomotive).
• Domain-Specific Expertise: While LLMs excel in coding due to the vast amount of training data, they frequently fail in specialized physical trades (agriculture, construction, mechanics) where they may provide "catastrophic" advice or hallucinate sources.
• Contextual Nuance: AI tools still struggle with tasks requiring human-like intuition, such as correcting mistakes in a chess scoresheet by inferring the intended move based on the level of play.

The 'World Model' Requirement for Self-Driving

One of the most contentious goalposts is the complete solution of self-driving cars. The consensus among some technical observers is that self-driving cannot be solved through pattern recognition alone but requires a fundamental "world model" for prediction.

The argument posits that the ability to distinguish between a harmless object (an empty soda can) and a critical hazard (a brick) requires a level of general intelligence (the "G" in AGI) to predict outcomes and avoid failures. Without a comprehensive world model, AI cannot reliably navigate the unpredictability of the physical world.

Redefining the Turing Test

The community's view of the Turing Test has evolved from a binary pass/fail to a nuanced discussion on human perception. Some argue the test is fundamentally flawed because humans are often unable to distinguish between a sophisticated "parrot" and a sentient being, making the test a measure of human gullibility rather than AI intelligence.

Furthermore, a distinction is now drawn between a standard Turing test and an "adversarial" Turing test, where the model must maintain a human persona over an extended period under intense scrutiny, a benchmark that many believe remains unmet.