90% of US Work Hours Go to Tasks AI Benchmarks Barely Test, Mercor Says. Build Your Own Evals

Mercor mapped 29 public AI benchmarks onto the Labor Department's catalog of American jobs and found that about 90% of US working hours go to tasks with little or no benchmark coverage. Only 208 of

Mercor mapped 29 public AI benchmarks onto the Labor Department's catalog of American jobs and found that about 90% of US working hours go to tasks with little or no benchmark coverage. Only 208 of 923 occupations have even one task tested by at least five benchmark prompts. It became one of the most discussed posts on Hacker News on Friday, mostly as an argument about jobs. I read it as something more practical for builders: the leaderboard you used to pick your model measures a narrow slice of