ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems
|
// A North Star To AGI The ARC-AGI-2 Technical Report is now published. ARC-AGI-2 incorporates a newly curated and expanded sets of tasks specifically designed to provide a granular signal for assessing the abstract reasoning and problem-solving capabilities of today's AI systems. These revamped sets of tasks target higher levels of fluid intelligence, demanding more sophisticated generalization, while continuing to target the intersection of what is feasible for humans but still out of reach for AI. Human Performance Testing One of the notable differences with ARC-AGI-2 is the first-party human performance data. Over 3 sessions, 400 people were tested on 1.8K candidate ARC-AGI-2 tasks. Only tasks that were solved by at least two people in two attempts or less were included in the final sets. 100% of ARC-AGI-2 tasks were solved by non-specialized testers. Alongside the paper, we have open-sourced the aggregated per-task performance data that was collected.
|