What are the main measures of AI progress?

Question

I&rsquo;m interested in how AI progress is currently evaluated and trying to build a list of the major approaches people actually use.I&rsquo;m aware that all of these measures have limitations and that many are controversial or imperfect. My goal is discovery and understanding, not to defend or attack any particular framework.I&rsquo;d love to hear:- What measures, benchmarks, or methodologies you think belong on this list- What you see as their key strengths and failure modes- How (or whether) you personally use them to interpret AI progress

johnnyfived · Accepted Answer

There'd first have to be an intense evaluation and standardization process for AI / measuring AGI now. All current benchmarks are tailored to one use case (e.g. SWE) or are evaluations that can be gamed and manipulated.I think this would take the form of something more abstract instead of concrete with raw numbers, like a revised Turing Test.