nucleus.annotation#

`Annotation`	Internal base class, not to be used directly.
`AnnotationList`	Wrapper class separating a list of annotations by type.
`AnnotationTypes`	Generic enumeration.
`BoxAnnotation`	A bounding box annotation.
`CategoryAnnotation`	A category annotation.
`CuboidAnnotation`	A 3D Cuboid annotation.
`Keypoint`	A 2D point that has an additional visibility flag.
`KeypointsAnnotation`	A keypoints annotation containing a list of keypoints and the structure
`LidarPoint`	A Lidar point in 3D space and intensity.
`LineAnnotation`	A polyline annotation consisting of an ordered list of 2D points.
`MultiCategoryAnnotation`	This class is not yet supported: MultiCategory annotation support coming soon!
`Point`	A point in 2D space.
`Point3D`	A point in 3D space.
`PolygonAnnotation`	A polygon annotation consisting of an ordered list of 2D points.
`SceneCategoryAnnotation`	A scene category annotation.
`Segment`	Segment represents either a class or an instance depending on the task type.
`SegmentationAnnotation`	A segmentation mask on a 2D image.

class nucleus.annotation.Annotation#

Internal base class, not to be used directly.

classmethod from_json(payload)#

Instantiates annotation object from schematized JSON dict payload.

Parameters:: payload (dict) –

has_local_files_to_upload()#

Returns True if annotation has local files that need to be uploaded.

Nearly all subclasses have no local files, so we default this to just return false. If the subclass has local files, it should override this method (but that is not the only thing required to get local upload of files to work.)

Return type:: bool

to_json()#

Serializes annotation object to schematized JSON string.

Return type:: str

abstract to_payload()#

Serializes annotation object to schematized JSON dict.

Return type:: dict

class nucleus.annotation.AnnotationList#: Wrapper class separating a list of annotations by type.

class nucleus.annotation.AnnotationTypes#

Generic enumeration.

Derive from this class to define new enumerations.

name()#: The name of the Enum member.

value()#: The value of the Enum member.

class nucleus.annotation.BoxAnnotation#

A bounding box annotation.

from nucleus import BoxAnnotation

box = BoxAnnotation(
    label="car",
    x=0,
    y=0,
    width=10,
    height=10,
    reference_id="image_1",
    annotation_id="image_1_car_box_1",
    metadata={"vehicle_color": "red"},
    embedding_vector=[0.1423, 1.432, ..., 3.829],
    track_reference_id="car_a",
)

Parameters:

label (str) – The label for this annotation.
x (Union[float, int]) – The distance, in pixels, between the left border of the bounding box and the left border of the image.
y (Union[float, int]) – The distance, in pixels, between the top border of the bounding box and the top border of the image.
width (Union[float, int]) – The width in pixels of the annotation.
height (Union[float, int]) – The height in pixels of the annotation.
reference_id (str) – User-defined ID of the image to which to apply this annotation.
annotation_id (Optional[str]) – The annotation ID that uniquely identifies this annotation within its target dataset item. Upon ingest, a matching annotation id will be ignored by default, and overwritten if update=True for dataset.annotate. If no annotation ID is passed, one will be automatically generated using the label, x, y, width, and height, so that you can make inserts idempotently as identical boxes will be ignored.
metadata (Optional[Dict]) –
Arbitrary key/value dictionary of info to attach to this annotation. Strings, floats and ints are supported best by querying and insights features within Nucleus. For more details see our metadata guide.

Coordinate metadata may be provided to enable the Map Chart in the Nucleus Dataset charts page. These values can be specified as { “lat”: 52.5, “lon”: 13.3, … }.
embedding_vector – Custom embedding vector for this object annotation. If any custom object embeddings have been uploaded previously to this dataset, this vector must match the dimensions of the previously ingested vectors.
track_reference_id – A unique string to identify the annotation as part of a group. For instance, multiple “car” annotations across several dataset items may have the same track_reference_id such as “red_car”.

classmethod from_json(payload)#

Instantiates annotation object from schematized JSON dict payload.

Parameters:: payload (dict) –

has_local_files_to_upload()#

Returns True if annotation has local files that need to be uploaded.

Return type:: bool

to_json()#

Serializes annotation object to schematized JSON string.

Return type:: str

to_payload()#

Serializes annotation object to schematized JSON dict.

Return type:: dict

class nucleus.annotation.CategoryAnnotation#

A category annotation.

from nucleus import CategoryAnnotation

category = CategoryAnnotation(
    label="dress",
    reference_id="image_1",
    taxonomy_name="clothing_type",
    metadata={"dress_color": "navy"},
    track_reference_id="blue_and_black_dress",
)

Parameters:

label (str) – The label for this annotation.
reference_id (str) – User-defined ID of the image to which to apply this annotation.
taxonomy_name (Optional[str]) – The name of the taxonomy this annotation conforms to. See Dataset.add_taxonomy().
metadata (Optional[Dict]) –
Arbitrary key/value dictionary of info to attach to this annotation. Strings, floats and ints are supported best by querying and insights features within Nucleus. For more details see our metadata guide.
track_reference_id – A unique string to identify the annotation as part of a group. For instance, multiple “car” annotations across several dataset items may have the same track_reference_id such as “red_car”.

classmethod from_json(payload)#

Instantiates annotation object from schematized JSON dict payload.

Parameters:: payload (dict) –

has_local_files_to_upload()#

Returns True if annotation has local files that need to be uploaded.

Return type:: bool

to_json()#

Serializes annotation object to schematized JSON string.

Return type:: str

to_payload()#

Serializes annotation object to schematized JSON dict.

Return type:: dict

class nucleus.annotation.CuboidAnnotation#

A 3D Cuboid annotation.

from nucleus import CuboidAnnotation

cuboid = CuboidAnnotation(
    label="car",
    position=Point3D(100, 100, 10),
    dimensions=Point3D(5, 10, 5),
    yaw=0,
    reference_id="pointcloud_1",
    annotation_id="pointcloud_1_car_cuboid_1",
    metadata={"vehicle_color": "green"},
    track_reference_id="red_car",
)

Parameters:

label (str) – The label for this annotation.
position (Point3D) – The point at the center of the cuboid
dimensions (Point3D) – The length (x), width (y), and height (z) of the cuboid
yaw (float) – The rotation, in radians, about the Z axis of the cuboid
reference_id (str) – User-defined ID of the image to which to apply this annotation.
annotation_id (Optional[str]) – The annotation ID that uniquely identifies this annotation within its target dataset item. Upon ingest, a matching annotation id will be ignored by default, and updated if update=True for dataset.annotate.
metadata (Optional[str]) –
Arbitrary key/value dictionary of info to attach to this annotation. Strings, floats and ints are supported best by querying and insights features within Nucleus. For more details see our metadata guide.
track_reference_id – A unique string to identify the annotation as part of a group. For instance, multiple “car” annotations across several dataset items may have the same track_reference_id such as “red_car”.

classmethod from_json(payload)#

Instantiates annotation object from schematized JSON dict payload.

Parameters:: payload (dict) –

has_local_files_to_upload()#

Returns True if annotation has local files that need to be uploaded.

Return type:: bool

to_json()#

Serializes annotation object to schematized JSON string.

Return type:: str

to_payload()#

Serializes annotation object to schematized JSON dict.

Return type:: dict

class nucleus.annotation.Keypoint#

A 2D point that has an additional visibility flag.

Keypoints are intended to be part of a larger collection, and connected via a pre-defined skeleton. A keypoint in this skeleton may be visible or not-visible, and may be unlabeled and not visible. Because of this, the x, y coordinates may be optional, assuming that the keypoint is not visible, and would not be shown as part of the combined label.

Parameters:

x (Optional[float]) – The x coordinate of the point.
y (Optional[float]) – The y coordinate of the point.
visible (bool) – The visibility of the point.

class nucleus.annotation.KeypointsAnnotation#

A keypoints annotation containing a list of keypoints and the structure of those keypoints: the naming of each point and the skeleton that connects those keypoints.

from nucleus import KeypointsAnnotation

keypoints = KeypointsAnnotation(
    label="face",
    keypoints=[Keypoint(100, 100), Keypoint(120, 120), Keypoint(visible=False), Keypoint(0, 0)],
    names=["point1", "point2", "point3", "point4"],
    skeleton=[[0, 1], [1, 2], [1, 3], [2, 3]],
    reference_id="image_2",
    annotation_id="image_2_face_keypoints_1",
    metadata={"face_direction": "forward"},
    track_reference_id="face_1",
)

Parameters:

label (str) – The label for this annotation.
keypoints (List[Keypoint]) – The list of keypoints objects.
names (List[str]) – A list that corresponds to the names of each keypoint.
skeleton (Optional[List[List[int]]]) – A list of 2-length lists indicating a beginning and ending index for each line segment in the skeleton of this keypoint label.
reference_id (str) – User-defined ID of the image to which to apply this annotation.
annotation_id (Optional[str]) – The annotation ID that uniquely identifies this annotation within its target dataset item. Upon ingest, a matching annotation id will be ignored by default, and updated if update=True for dataset.annotate.
metadata (Optional[Dict]) –
Arbitrary key/value dictionary of info to attach to this annotation. Strings, floats and ints are supported best by querying and insights features within Nucleus. For more details see our metadata guide.
track_reference_id – A unique string to identify the annotation as part of a group. For instance, multiple “car” annotations across several dataset items may have the same track_reference_id such as “red_car”.

classmethod from_json(payload)#

Instantiates annotation object from schematized JSON dict payload.

Parameters:: payload (dict) –

has_local_files_to_upload()#

Returns True if annotation has local files that need to be uploaded.

Return type:: bool

to_json()#

Serializes annotation object to schematized JSON string.

Return type:: str

to_payload()#

Serializes annotation object to schematized JSON dict.

Return type:: dict

class nucleus.annotation.LidarPoint#

A Lidar point in 3D space and intensity.

Parameters:

x (float) – The x coordinate of the point.
y (float) – The y coordinate of the point.
z (float) – The z coordinate of the point.
i (float) – The intensity value returned by the lidar scan point.

class nucleus.annotation.LineAnnotation#

A polyline annotation consisting of an ordered list of 2D points. A LineAnnotation differs from a PolygonAnnotation by not forming a closed loop, and by having zero area.

from nucleus import LineAnnotation

line = LineAnnotation(
    label="face",
    vertices=[Point(100, 100), Point(200, 300), Point(300, 200)],
    reference_id="person_image_1",
    annotation_id="person_image_1_line_1",
    metadata={"camera_mode": "portrait"},
    track_reference_id="face_human",
)

Parameters:

label (str) – The label for this annotation.
vertices (List[Point]) – The list of points making up the line.
reference_id (str) – User-defined ID of the image to which to apply this annotation.
annotation_id (Optional[str]) – The annotation ID that uniquely identifies this annotation within its target dataset item. Upon ingest, a matching annotation id will be ignored by default, and updated if update=True for dataset.annotate.
metadata (Optional[Dict]) –
Arbitrary key/value dictionary of info to attach to this annotation. Strings, floats and ints are supported best by querying and insights features within Nucleus. For more details see our metadata guide.
track_reference_id – A unique string to identify the annotation as part of a group. For instance, multiple “car” annotations across several dataset items may have the same track_reference_id such as “red_car”.

classmethod from_json(payload)#

Instantiates annotation object from schematized JSON dict payload.

Parameters:: payload (dict) –

has_local_files_to_upload()#

Returns True if annotation has local files that need to be uploaded.

Return type:: bool

to_json()#

Serializes annotation object to schematized JSON string.

Return type:: str

to_payload()#

Serializes annotation object to schematized JSON dict.

Return type:: dict

class nucleus.annotation.MultiCategoryAnnotation#

This class is not yet supported: MultiCategory annotation support coming soon!

classmethod from_json(payload)#

Instantiates annotation object from schematized JSON dict payload.

Parameters:: payload (dict) –

has_local_files_to_upload()#

Returns True if annotation has local files that need to be uploaded.

Return type:: bool

to_json()#

Serializes annotation object to schematized JSON string.

Return type:: str

to_payload()#

Serializes annotation object to schematized JSON dict.

Return type:: dict

class nucleus.annotation.Point#

A point in 2D space.

Parameters:

x (float) – The x coordinate of the point.
y (float) – The y coordinate of the point.

class nucleus.annotation.Point3D#

A point in 3D space.

Parameters:

x (float) – The x coordinate of the point.
y (float) – The y coordinate of the point.
z (float) – The z coordinate of the point.

class nucleus.annotation.PolygonAnnotation#

A polygon annotation consisting of an ordered list of 2D points.

from nucleus import PolygonAnnotation

polygon = PolygonAnnotation(
    label="bus",
    vertices=[Point(100, 100), Point(150, 200), Point(200, 100)],
    reference_id="image_2",
    annotation_id="image_2_bus_polygon_1",
    metadata={"vehicle_color": "yellow"},
    embedding_vector=[0.1423, 1.432, ..., 3.829],
    track_reference_id="school_bus",
)

Parameters:

label (str) – The label for this annotation.
vertices (List[Point]) – The list of points making up the polygon.
reference_id (str) – User-defined ID of the image to which to apply this annotation.
annotation_id (Optional[str]) – The annotation ID that uniquely identifies this annotation within its target dataset item. Upon ingest, a matching annotation id will be ignored by default, and updated if update=True for dataset.annotate.
metadata (Optional[Dict]) –
Arbitrary key/value dictionary of info to attach to this annotation. Strings, floats and ints are supported best by querying and insights features within Nucleus. For more details see our metadata guide.
embedding_vector – Custom embedding vector for this object annotation. If any custom object embeddings have been uploaded previously to this dataset, this vector must match the dimensions of the previously ingested vectors.
track_reference_id – A unique string to identify the annotation as part of a group. For instance, multiple “car” annotations across several dataset items may have the same track_reference_id such as “red_car”.

classmethod from_json(payload)#

Instantiates annotation object from schematized JSON dict payload.

Parameters:: payload (dict) –

has_local_files_to_upload()#

Returns True if annotation has local files that need to be uploaded.

Return type:: bool

to_json()#

Serializes annotation object to schematized JSON string.

Return type:: str

to_payload()#

Serializes annotation object to schematized JSON dict.

Return type:: dict

class nucleus.annotation.SceneCategoryAnnotation#

A scene category annotation.

from nucleus import SceneCategoryAnnotation

category = SceneCategoryAnnotation(
    label="running",
    reference_id="scene_1",
    taxonomy_name="action",
    metadata={
        "weather": "clear",
    },
)

Parameters:

label (str) – The label for this annotation.
reference_id (str) – User-defined ID of the scene to which to apply this annotation.
taxonomy_name (Optional[str]) – The name of the taxonomy this annotation conforms to. See Dataset.add_taxonomy().
metadata –
Arbitrary key/value dictionary of info to attach to this annotation. Strings, floats and ints are supported best by querying and insights features within Nucleus. For more details see our metadata guide.

classmethod from_json(payload)#

Instantiates annotation object from schematized JSON dict payload.

Parameters:: payload (dict) –

has_local_files_to_upload()#

Returns True if annotation has local files that need to be uploaded.

Return type:: bool

to_json()#

Serializes annotation object to schematized JSON string.

Return type:: str

to_payload()#

Serializes annotation object to schematized JSON dict.

Return type:: dict

class nucleus.annotation.Segment#

Segment represents either a class or an instance depending on the task type.

For semantic segmentation, this object should store the mapping between a single class index and the string label.

For instance segmentation, you can use this class to store the label of a single instance, whose extent in the image is represented by the value of index.

In both cases, additional metadata can be attached to the segment.

Parameters:

label (str) – The label name of the class for the class or instance represented by index in the associated mask.
index (int) – The integer pixel value in the mask this mapping refers to.
metadata (Optional[Dict]) –
Arbitrary key/value dictionary of info to attach to this segment. Strings, floats and ints are supported best by querying and insights features within Nucleus. For more details see our metadata guide.

class nucleus.annotation.SegmentationAnnotation#

A segmentation mask on a 2D image.

When uploading a mask annotation, Nucleus expects the mask file to be in PNG format with each pixel being a 0-255 uint8. Currently, Nucleus only supports uploading masks from URL.

Nucleus automatically enforces the constraint that each DatasetItem can have at most one ground truth segmentation mask. As a consequence, if during upload a duplicate mask is detected for a given image, by default it will be ignored. You can change this behavior by setting update = True, which will replace the existing segmentation mask with the new mask.

from nucleus import SegmentationAnnotation

segmentation = SegmentationAnnotation(
    mask_url="s3://your-bucket-name/segmentation-masks/image_2_mask_id_1.png",
    annotations=[
        Segment(label="grass", index="1"),
        Segment(label="road", index="2"),
        Segment(label="bus", index="3", metadata={"vehicle_color": "yellow"}),
        Segment(label="tree", index="4")
    ],
    reference_id="image_2",
    annotation_id="image_2_mask_1",
)

Parameters:

mask_url (str) –
A URL pointing to the segmentation prediction mask which is accessible to Scale. This “URL” can also be a path to a local file. The mask is an HxW int8 array saved in PNG format, with each pixel value ranging from [0, N), where N is the number of possible classes (for semantic segmentation) or instances (for instance segmentation).

The height and width of the mask must be the same as the original image. One example for semantic segmentation: the mask is 0 for pixels where there is background, 1 where there is a car, and 2 where there is a pedestrian.

Another example for instance segmentation: the mask is 0 for one car, 1 for another car, 2 for a motorcycle and 3 for another motorcycle. The class name for each value in the mask is stored in the list of Segment objects passed for “annotations”
annotations (List[Segment]) – The list of mappings between the integer values contained in mask_url and string class labels. In the semantic segmentation example above these would map that 0 to background, 1 to car and 2 to pedestrian. In the instance segmentation example above, 0 and 1 would both be mapped to car, 2 and 3 would both be mapped to motorcycle
reference_id (str) – User-defined ID of the image to which to apply this annotation.
annotation_id (Optional[str]) – For segmentation annotations, this value is ignored because there can only be one segmentation annotation per dataset item. Therefore regardless of annotation ID, if there is an existing segmentation on a dataset item, it will be ignored unless update=True is passed to Dataset.annotate(), in which case it will be overwritten. Storing a custom ID here may be useful in order to tie this annotation to an external database, and its value will be returned for any export.

classmethod from_json(payload)#

Instantiates annotation object from schematized JSON dict payload.

Parameters:: payload (dict) –

has_local_files_to_upload()#

Check if the mask url is local and needs to be uploaded.

Return type:: bool

to_json()#

Serializes annotation object to schematized JSON string.

Return type:: str

to_payload()#

Serializes annotation object to schematized JSON dict.

Return type:: dict