Data labeling and classification

Label and classify text, images, audio, video, and PDFs.

Agents can:

• Turn text, images, audio, video, and PDFs into structured records.
• Assign labels, label sets, taxonomy paths, and labeled spans.
• Score and rank model outputs for evals and preference data.
• Add a reviewer and an adjudicator when label quality matters.

Each task follows a similar pattern: an agent with an output_schema.

Agent.run() accepts text and media inputs. Set output_schema to validate the response against a Pydantic model.

Example

from typing import Literal

from agno.agent import Agent
from agno.models.google import Gemini
from pydantic import BaseModel, Field


class Classification(BaseModel):
    label: Literal["positive", "negative", "neutral"] = Field(
        ..., description="The assigned sentiment label"
    )


agent = Agent(
    model=Gemini(id="gemini-3.5-flash"),
    instructions="You classify product reviews by sentiment.",
    output_schema=Classification,
)

result = agent.run("Broken on arrival, total waste of money.").content
# Classification(label='negative')

Swap the schema and instructions and the same pattern covers data extraction, span labeling, scoring, and preference ranking.

If you’re looking to jump straight into code, the data labeling cookbook contains 40+ runnable recipes across 18 data labeling patterns.

Data labeling workflows

Pick the page that matches what you need.

Workload	Input	Output	Page
Data extraction	Any modality	Typed Pydantic object	Data extraction
Classification	Any modality	One label, label set, or spans	Classification
Scoring / evaluation	Prompt + response	Rubric scores	LLM as judge
Preference ranking	Prompt + two responses	Winner + rationale	Preference data
Non-text input	Image, audio, video, PDF	Any of the above	Multimodal inputs
Reviewed labels	Any input	Adjudicated label + audit trail	Quality pipeline

Model choice

The cookbooks use gemini-3.5-flash for text, image, audio, video, and PDF examples. A replacement model must support the input modality and structured output used by the recipe.

To run the examples, install the Google provider and set your API key:

pip install -U "agno[google]"
export GOOGLE_API_KEY=***

The Quality pipeline runs its second labeler, reviewer, and adjudicator on Anthropic for provider diversity. That page needs a second provider and key:

pip install -U "agno[anthropic]"
export ANTHROPIC_API_KEY=***

Explore

Data extraction

Turn any modality into a typed object, with optional per-field confidence.

Classification

Single-label, multi-label, hierarchical, and span labeling.

LLM as judge

Score outputs against a rubric. The same machinery, used for evals.

Preference data

Rank A vs B for RLHF and DPO datasets.

Multimodal inputs

Feed images, audio, video, and PDFs into any labeler.

Quality pipeline

Two labelers, a reviewer, and an adjudicator with an audit trail.

Developer Resources

• Data labeling cookbook
• Structured output
• Multimodal agents