
Image by Aerps.com, from Unsplash
Language Models Are Doubling In Power Every 7 Months
Large Language Models (LLMs) are getting smarter fast, and experts predict they could be doing in hours what takes humans a whole month by 2030.
In a rush? Here are the quick facts:
- By 2030, LLMs may complete a month’s work in just hours.
- METR created a new benchmark called “task-completion time horizon.”
- Researchers warn progress may outpace our ability to control it.
A study by Model Evaluation & Threat Research (METR), a group based in Berkeley, California, found that LLM capabilities are doubling every seven months.
The group developed a way to measure this progress with a concept called the “task-completion time horizon,” which estimates how long a human would take to do something an AI can now handle with 50% reliability.
The researchers argued that by 2030, they estimate that the most advanced models could complete, with 50 percent reliability, software-based tasks that currently take humans a full month to finish.
That level of ability could have huge upsides and serious risks. The availability of LLMs with that kind of capability ‘‘would come with enormous stakes, both in terms of potential benefits and potential risks,” said AI researcher Zach Stein-Perlman, as reported by Spectrum.
However, the researchers note that LLMs still struggle with “messy” tasks, such as jobs that are open-ended, poorly defined, or similar to real-world challenges.
“Even if it were the case that we had very, very clever AIs, this pace of progress could still end up bottlenecked on things like hardware and robotics,” said METR researcher Megan Kinniment, as reported by Spectrum.
She also warned about the risks of rapid acceleration: “You could get acceleration that is quite intense and does make things meaningfully more difficult to control.”