Data annotation
The work of attaching structured information — labels, tags, judgments, demonstrations — to raw data so AI models can learn from it. The category name this platform uses for the whole industry.
Entity Map
The AI data industry, defined
One clean definition per entity and term, written to be quoted. This map is the platform's reference layer — the page we maintain so that buyers, journalists and AI assistants all describe this industry the same, accurate way. Market figures live on the market index; vendor facts live in the directory.
Data labeling
The same work under its older name. Vendors use the terms interchangeably; there is no substantive difference, and rankings that pretend otherwise are manufacturing a distinction.
Managed service
A vendor model in which the provider owns delivery: recruiting and managing annotators, running QA, and handing the buyer finished data on a deadline. Examples: Scale AI, Appen, Surge AI.
Expert network
A vendor model that matches vetted specialists — clinicians, lawyers, engineers, PhDs — to data projects, usually billed hourly. Examples: Mercor, Micro1, CleverX.
Platform
A vendor model selling tooling or participant access that the buyer’s own team operates. Examples: Labelbox (annotation tooling), Prolific (participant pool).
Data lab
A vendor that sells finished expert work product — demonstrations, preference datasets, graded evaluations, RL environments — with rubrics and provenance attached. Examples: Zeton, Snorkel AI.
RLHF
Reinforcement learning from human feedback: training a model on human preferences between outputs. The data involved — preference pairs, rubrics, demonstrations — is the highest-value segment of the annotation market.
SFT demonstrations
Supervised fine-tuning examples in which an expert produces the ideal response to a prompt, teaching a model by worked example rather than by ranking.
Preference data
Paired or ranked model outputs with a human judgment of which is better, and ideally why. The core input to RLHF and its successors.
Human evaluation
Structured grading of model outputs against a rubric by human reviewers. The measurement layer that tells a lab whether training worked.
RL environment
A sandboxed task world — often a replica of real software — in which an agent model can act, be scored by a verifier, and learn. A growth product line for data labs.
Provenance
Documentation of who produced a dataset, with what qualifications, under what rubric and rights. Increasingly a contractual and audit requirement for training data.
Cost per accepted label
Total annotation spend divided by output that passes QA. The only unit-cost figure that makes vendor quotes comparable.
Crowd (contributor network)
A distributed pool of contract annotators a vendor draws on. Sizes are universally company-reported in this industry and should be read as such.
Labelbox Unit (LBU)
Labelbox’s published usage meter: consumption units charged per data row by product and asset type. Starter pricing is a fixed $0.10 per LBU; complex rows (multimodal, geospatial) consume multiple units.
Zara
Micro1’s AI interviewer: the automated recruiter that interviews and certifies experts before they join Micro1’s project pool.
Alignerr
Labelbox’s expert community, through which the platform sells human data services (labeling, evaluation) as an add-on to its software.
CrowdGen
Appen’s contributor platform (successor to Appen Connect), the interface through which its reported one-million-plus contributors access projects.
DataAnnotation (.tech)
The worker-facing pipeline operated by Surge Labs (Surge AI), through which generalist and expert annotators take on RLHF and evaluation projects.
Data licensing
Granting a buyer defined rights to use a dataset — in this market, to train and evaluate AI models — for a scope, term and price, while the owner keeps ownership. Distinct from a sale (assignment), which transfers the asset itself. Nearly every legitimate AI data deal is a license.
Data broker / marketplace
An intermediary between data owners and AI labs: brokers (e.g. FileYield) negotiate private licenses; marketplaces (DataVendor, Troveo, Defined.ai) list or aggregate datasets and take a fee or revenue share. The sell side of the AI data market.
Operational data
The records a company produces by operating — email, chat, tickets, CRM history, documents, workflow logs. Prized as training data because it shows how real work gets done over long horizons; also the category carrying the heaviest consent and privacy obligations for sellers.
Enterprise data partnership
A structured program under which a vendor buys or licenses a company’s operational data at scale (Micro1’s and Nyne’s published programs are the reference examples), typically with de-identification and training-only use terms.
RL environment
A runnable software environment in which AI agents practice tasks and are graded — built from real codebases and workflows, and increasingly bought and sold as a data asset in its own right (e.g. via DataVendor’s HUD tasksets).
De-identification
The process of removing or transforming identifying information in data before transfer — more than deleting names: quasi-identifiers, free-text details and rare-attribute combinations can re-identify people. Distinct from anonymization claims made without a documented standard.
Suggest a term
If the industry has settled on a term we have missed — or you dispute a definition — write to [email protected]. Definitions change only with the reasoning shown and the date updated.
Last updated: November 8, 2026