AI agents quadruple their score on a real freelance-work benchmark

Overview

Updated Jul 9

Eight months ago, the best AI agent could finish 2.5% of real freelance jobs well enough for a paying client to accept. On July 6, the Center for AI Safety reported that a new model now clears 16.1%.

The number is small, but the slope is steep. The test uses 240 real paid projects, and human professionals earned about $144,000 doing them. AI agents are still failing most of the work. They are failing less of it, fast.

Why it matters

If AI agents keep closing this gap at the current pace, freelance work in design, video, and data analysis is the first labor market to feel it.

Questions about this story

Free account needed to ask — your question is kept and asked for you right after sign-up. Answers are public.

No questions yet — be the first to ask.

Key Indicators

16.1%

Top automation rate

Share of paid projects the best agent, Fable 5, completed at professional quality.

2.5%

Rate 8 months earlier

The best agent's score when the benchmark launched in October 2025.

240

Real projects tested

Paid freelance jobs across 23 categories, each judged against professional work.

$144K

Human earnings on the same work

Combined amount freelancers were paid for the 240 projects.

84%

Projects agents still fail

Even the top model falls short of client-acceptable quality on most work.

Voices

Curated perspectives — historical figures and your fellow readers.

Ever wondered what historical figures would say about today's headlines?

People Involved

Dan Hendrycks

Executive Director, Center for AI Safety

Leading the group that runs the benchmark

Bing Liu

Head of Research, Scale AI

Co-led the benchmark's design and evaluation

Organizations Involved

Center for AI Safety

Nonprofit Research Organization

Runs and publishes the Remote Labor Index

A San Francisco nonprofit that studies risks from advanced AI and builds tests to measure what AI systems can do.

Scale AI

AI Data and Evaluation Company

Partner that runs the human grading

A company that supplies data and evaluation services to AI developers, providing the human graders behind the benchmark.

Timeline

October 2025 July 2026

2 events Latest: July 6th, 2026 · 2 months ago

July 2026
Frontier score more than quadruples
Latest Research

New results show Fable 5 at 16.1%, Opus 4.8 at 8.3%, and GPT-5.5 at 6.3%. The top score has more than quadrupled from the previous leader's 4.2% in under eight months.
July 6th, 2026
October 2025
Benchmark launches with a 2.5% ceiling
Research

The Center for AI Safety and Scale AI release the Remote Labor Index. The best agent, Manus, completes 2.5% of the 240 paid projects at professional quality.
October 30th, 2025

Scenarios

AI clears 30% of the benchmark within a year

Possible Resolves by Jul 6, 2027

Discussed by: Center for AI Safety, tracking the frontier on remotelabor.ai

Model releases keep arriving every few months, and each recent jump has been large. If the next generation of frontier models extends the current slope, a top agent passes 30% on the Remote Labor Index. That would mean AI can finish roughly one in three real paid remote projects at a quality clients accept, a level that starts to reshape freelance pricing in the affected categories.

Progress stalls below 25% as easy gains run out

Possible Resolves by Jul 6, 2027

Discussed by: Skeptics who note agents still fail the large majority of projects

The early jumps may reflect models mastering the more structured tasks first, leaving the messy, judgment-heavy projects that resist automation. If gains flatten, the frontier settles in the high teens or low twenties for an extended stretch, and headlines about imminent freelance displacement cool. The benchmark's human graders keep rejecting most deliverables as below client standard.

A major AI lab cites the index in a model launch

Likely Resolves by Jan 6, 2027

Discussed by: AI industry watchers tracking how labs market capability

Benchmarks gain weight when labs use them in their own announcements. If a frontier lab reports its Remote Labor Index score in an official model release, the test becomes a standard yardstick for economic capability, the way earlier reasoning benchmarks did. That adoption would push rivals to publish their own scores and sharpen public attention on real-work automation.

Historical Context

3 moments from history that rhyme with this story — and how they unfolded.

September 2012

ImageNet and the deep-learning breakout (2012)

A neural network called AlexNet cut the error rate on the ImageNet image-recognition contest by roughly 10 percentage points in one year. The benchmark had moved slowly before. Then it moved fast.

Then

Research funding and talent poured into deep learning within months.

Now

The approach became the base for modern computer vision, from phone cameras to self-driving car perception.

Why this matters now

A single benchmark can flip from slow crawl to steep climb. The Remote Labor Index's jump from 2.5% to 16.1% has the same early shape, though the outcome is not yet settled.

2015 to present

Self-driving 90% problem (2015 onward)

Autonomous-driving demos quickly handled most road situations, and companies predicted robotaxis within a few years. The last fraction of hard cases proved far slower to solve.

Then

Billions in investment chased a near-term rollout that kept slipping.

Now

Deployment came, but narrowly, city by city, more than a decade after the early hype.

Why this matters now

Fast early gains do not guarantee a fast finish. The messy final projects on the Remote Labor Index could resist automation the way rare road cases did.

November 2020

DeepMind's protein-folding leap on CASP (2020)

AlphaFold scored high enough on the CASP protein-structure contest that organizers called the 50-year problem largely solved. Prior systems had plateaued for years.

Then

Structural biology labs began using predicted structures almost immediately.

Now

DeepMind released structures for most known proteins, speeding drug and disease research worldwide.

Why this matters now

A trusted, hard-to-game benchmark turned a research milestone into real-world use. The Remote Labor Index aims for the same credibility by grading against paid human work.