Content Specialist III

The Content Specialist owns label and AI quality for a portfolio of Facebook Marketplace labeling and evaluation projects. Most of these projects measure how well Marketplace product models perform, such as search relevance, query intent, product categorization, attribute and concept accuracy, and listing image quality. As more labeling moves to AI labelers, this role decides whether those labels can be trusted. You will design and run evaluations, audit AI and human labels, own labeling guidelines with product teams, and turn label data into insights that product partners act on.
The ideal candidate has strong editorial judgment, writes precise guidelines, is comfortable with data, and enjoys working directly with product teams.

Responsibilities
Own guidelines with product. Maintain labeling guidelines and rubrics for assigned projects in partnership with product points of contact. Translate product requirements and edge cases into clear, scalable guidance, and drive updates when audits surface ambiguity.
Evaluate AI labels. Run recurring evaluations of AI labelers: random-sample rubric audits and adjudication of AI vs human disagreements. Log error categories and root causes, recommend guideline and prompt fixes, and give evidence-based input on whether a queue is ready for full automation.
Audit human labels. Audit vendor rater output, lead calibration sessions with vendor leads, and monitor pass rate and agreement against quality targets.
Build ground truth. Create and maintain reference sets of hard, correctly labeled examples used to check labeler and model quality.
Deliver product insights. Analyze label trends (by category, human vs AI) and produce concise monthly readouts with clear findings and recommended actions for product partners.
Stay close to product. Work in product team chats, answer guideline questions quickly, and adjust priorities as product needs change.
Work AI-first. Use LLM tools in daily work for analysis, drafting, and evaluation. Develop lightweight, repeatable processes and templates that keep the team fast.

Skills
5-7 years in content evaluation, search or ads quality rating, data annotation quality assurance, trust and safety, or LLM output evaluation.
Strong editorial judgment and a strong command of grammar, style, and tone. Able to write guidelines that leave little room for interpretation.
Comfortable with numbers: pass rates, agreement rates, sampling, and trend analysis in Google Sheets or Excel.
Able to turn data findings into short, clear, actionable summaries for product audiences.
Experience prompting LLMs or evaluating AI-generated output.
Self-starter who works independently and handles shifting priorities well.

Preferred
SQL experience.
Familiarity with e-commerce, marketplaces, or search relevance.

Education
Bachelor's degree required.

This posting is for a contract assignment with Tundra Technical Solutions to provide services to Meta

The pay range that Tundra in good faith reasonably expects to pay for this position is $31.72/hour - $39.66/hour.

Tundra’s benefits offering includes optional medical, dental, vision, retirement benefits, up to 15 Days PTO per annum and a New Child Benefit.

We would like to thank you for your interest in this opportunity. However, only candidates under consideration will be contacted.

Tundra Technical Solutions is a global workforce and technology delivery firm, ranked by Staffing Industry Analysts as one of the largest in North America. At Tundra, we aren't just hiring top talent at the world's most recognizable brands; we are pioneers of social recruitment.

This post is for a contract assignment. Assignment duties and duration are subject to business needs and requirements. We are an equal opportunity employer, and we do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, age, veteran status, disability, genetic information, or other legally protected characteristics.

California Notice: Pursuant to the California Fair Chance Act, Los Angeles County Fair Chance Ordinance for Employers, Los Angeles Fair Chance Initiative for Hiring Ordinance, and San Francisco Fair Chance Ordinance, qualified applicants will be considered for assignment with arrest and conviction records. Criminal history may have a direct, adverse, and negative relationship with some of the material job duties of this position. These include the duties and responsibilities listed above, as well as the abilities to adhere to company policies, exercise sound judgment, effectively manage stress and work safely and respectfully with others, exhibit trustworthiness, meet client expectations, standards, and accompanying requirements, and safeguard business operations and company reputation.

New York Notice: Notice Regarding Automated Employment Decision Tools in New York City[LD1]

Meta and Tundra Technical Solutions, its recruiting services provider, use software with artificial intelligence and automated processing features to assist with recruiting for this role, including sourcing and reviewing applications. A recruiter or hiring manager decides which candidates advance.

The Meta CWX Program is enabled by a cutting-edge software platform called TalentNet that leads the contingent labor world for technology innovation. The software platform leverages Machine Learning and Artificial Intelligence to make sure the right people end up in the right job.

At Meta, we are constantly iterating, solving problems, and working together to connect people all over the world. That’s why it’s important that our workforce reflects the diversity of the people we serve. Hiring people with different backgrounds and points of view helps us make better decisions, build better products, and create better experiences for everyone.

We give people the power to build community and bring the world closer together. Our products empower more than 3 billion people around the world to share ideas, offer support, and make a difference