AI Agent Evaluation Expert

Put your expertise to work at the frontier of AI. Bring the expertise you know best and help train AI agents that do real technical work, on your own schedule, from anywhere. You'll turn problems from your own research or professional practice into hands-on challenges an AI agent has to solve at the command line, find exactly where it breaks, and set the expert standard it's measured against.

Workada connects skilled contributors with meaningful AI work. Your expertise makes AI systems smarter, safer, and more useful for everyone. We're building a community of experts across science, engineering, business, and creative fields, both PhDs and seasoned practitioners, to test how far today's AI agents can go on real problems, and we want you in it.

What You'll Do

As a contributor, you'll take problems from your own work and turn each one into a self-contained Task that an AI agent has to solve in a Linux terminal. For every Task, you'll write a clear brief, set up the environment with the files and tools the agent needs, build a reference solution, and write automated checks on the final result.

The strongest Tasks are the ones today's best agents still get wrong. Your firsthand knowledge of where real work in your field goes wrong is what makes them valuable. That might be a numerical solver that quietly loses precision, a migration that silently drops rows, a data loader that leaks the test split, a stripped binary hiding its real logic, a clock-domain crossing that passes simulation but fails in silicon, a reconciliation that's off by a currency conversion, or a color profile that shifts on conversion.

Who We're Looking For

Put your expertise to work at the frontier of AI. Bring the expertise you know best and help train AI agents that do real technical work, on your own schedule, from anywhere. You'll turn problems from your own research or professional practice into hands-on challenges an AI agent has to solve at the command line, find exactly where it breaks, and set the expert standard it's measured against.

Workada connects skilled contributors with meaningful AI work. Your expertise makes AI systems smarter, safer, and more useful for everyone. We're building a community of experts across science, engineering, business, and creative fields, both PhDs and seasoned practitioners, to test how far today's AI agents can go on real problems, and we want you in it.

Requirements

• You hold, or are completing, a PhD in one of the fields above or a closely related field, or you have 5+ years of professional experience in one of them.
• You've done technical or computational work in your field and are comfortable in a Linux terminal: shell commands, scripting in Python or Bash, and installing packages.
• You've used Docker, or can pick it up quickly. We share templates and worked examples.
• You can draw on your own research, projects, code, or files as the source material for your Tasks.
• You have a precise, adversarial eye; you enjoy finding the exact point where a workflow breaks and explaining it cleanly.
• You're honest about uncertainty and write instructions that leave no room for misreading.
• You're based in the US or Canada.

Workada is an Equal Opportunity Employer. Workada welcomes applicants from all backgrounds.

_This is a 1099 contractor role. Workada doesn't sponsor visas, act as a petitioning employer, file immigration paperwork, or sign offer or support letters tied to a visa category. You must already have work authorization that doesn't depend on Workada's involvement._