Beyond Human Measure: ASI Should Be Guided by Open-World Alignment
preprint
OA: closed
Abstract
ASI should be guided by open-world alignment. While human labels, preferences, and benchmarks remain indispensable, they are too narrow to serve as master objectives for frontier systems. The road to ASI is inherently a frontier supervision problem. As humans become structurally weak supervisors, the technical challenge shifts from imitating judgments to building evaluation loops answerable to reality, explicit constraints, and post-deployment evidence. We critique human-signal monism— the assumption that human-facing signals sufficiently proxy overall system quality—and diagnose its failure modes, including evaluator weakness and proxy over-optimization. As a constructive alternative, open-world alignment centers human intent while coupling it to verifier ecologies: layered task verifiers, hard constraints, uncertainty gates, and monitoring. This framework contributes an operational ASI definition, an actionable formalization, a coding agent case study, an analysis of open-world alignment’s own failure modes, and the Open-World Evaluation Card reporting artifact. Ultimately, ASI must be guided by objectives that remain answerable to the world.
My notes (saved in your browser only)
Citation neighborhood (no data yet)
We don't have any in-corpus citations linked to this paper yet. This is a recent paper (2026) — citers typically take a year or two to land, and the OpenAlex reference graph may still be filling in.
Source provenance
- europepmc
- last seen: 2026-05-20T01:45:00.602351+00:00