Welcome to the Nucleus API Reference!

Aggregate metrics in ML are not good enough. To improve production ML, you need to understand qualitative failure modes, fix them by gathering more data, and curate diverse scenarios.

Scale Nucleus helps you:

  • Visualize your data

  • Curate interesting slices within your dataset

  • Review and manage annotations

  • Measure and debug your model performance

Nucleus is a new way—the right way—to develop ML models, helping us move away from the concept of one dataset and towards a paradigm of collections of scenarios.

Evaluations V2

Evaluation V2 measures how well a model run matches ground-truth annotations. Create a run with NucleusClient.create_evaluation_v2(), wait with nucleus.evaluation_v2.EvaluationV2.wait_for_completion(), then read summary metrics with nucleus.evaluation_v2.EvaluationV2.charts() or individual matches with nucleus.evaluation_v2.EvaluationV2.examples().

import nucleus

client = nucleus.NucleusClient(api_key="YOUR_API_KEY")
evaluation = client.create_evaluation_v2(
    model_run_id="run_xxx",
    name="my-eval",
    allowed_label_matches=[
        nucleus.AllowedLabelMatch(
            ground_truth_label="car",
            model_prediction_label="vehicle",
        ),
    ],
)
evaluation.wait_for_completion()
charts = evaluation.charts(iou_threshold=0.5)
fps = evaluation.examples(match_type="FP", limit=20)

Installation

To use Nucleus, first install it using pip:

(venv) $ pip install scale-nucleus

Sections

Index