nucleus.prediction ============= .. py:module:: nucleus.prediction .. autoapi-nested-parse:: All of the prediction types supported. In general, prediction types are the same as annotation types, but come with additional, optional data that can be attached such as confidence or probability distributions. .. autoapisummary:: nucleus.prediction.BoxPrediction nucleus.prediction.CategoryPrediction nucleus.prediction.CuboidPrediction nucleus.prediction.KeypointsPrediction nucleus.prediction.LinePrediction nucleus.prediction.PolygonPrediction nucleus.prediction.PredictionList nucleus.prediction.SceneCategoryPrediction nucleus.prediction.SegmentationPrediction .. py:class:: BoxPrediction(label, x, y, width, height, reference_id, confidence = None, annotation_id = None, metadata = None, class_pdf = None, embedding_vector = None, track_reference_id = None) Prediction of a bounding box. :param label: The label for this annotation (e.g. car, pedestrian, bicycle) :type label: str :param x: The distance, in pixels, between the left border of the bounding box and the left border of the image. :type x: Union[float, int] :param y: The distance, in pixels, between the top border of the bounding box and the top border of the image. :type y: Union[float, int] :param width: The width in pixels of the annotation. :type width: Union[float, int] :param height: The height in pixels of the annotation. :type height: Union[float, int] :param reference_id: User-defined ID of the image to which to apply this annotation. :type reference_id: str :param confidence: 0-1 indicating the confidence of the prediction. :param annotation_id: The annotation ID that uniquely identifies this annotation within its target dataset item. Upon ingest, a matching annotation id will be ignored by default, and updated if update=True for dataset.annotate. If no annotation ID is passed, one will be automatically generated using the label, x, y, width, and height, so that you can make inserts idempotently and identical boxes will be ignored. :type annotation_id: Optional[str] :param metadata: Arbitrary key/value dictionary of info to attach to this annotation. Strings, floats and ints are supported best by querying and insights features within Nucleus. For more details see our `metadata guide `_. Coordinate metadata may be provided to enable the Map Chart in the Nucleus Dataset charts page. These values can be specified as `{ "lat": 52.5, "lon": 13.3, ... }`. :type metadata: Optional[Dict] :param class_pdf: An optional complete class probability distribution on this annotation. Each value should be between 0 and 1 (inclusive), and sum up to 1 as a complete distribution. This can be useful for computing entropy to surface places where the model is most uncertain. :param embedding_vector: Custom embedding vector for this object annotation. If any custom object embeddings have been uploaded previously to this dataset, this vector must match the dimensions of the previously ingested vectors. :type embedding_vector: Optional[List] :param track_reference_id: A unique string to identify the prediction as part of a group. For instance, multiple "car" predictions across several dataset items may have the same `track_reference_id` such as "red_car". .. py:method:: from_json(payload) :classmethod: Instantiates annotation object from schematized JSON dict payload. .. py:method:: has_local_files_to_upload() Returns True if annotation has local files that need to be uploaded. Nearly all subclasses have no local files, so we default this to just return false. If the subclass has local files, it should override this method (but that is not the only thing required to get local upload of files to work.) .. py:method:: to_json() Serializes annotation object to schematized JSON string. .. py:method:: to_payload() Serializes annotation object to schematized JSON dict. .. py:class:: CategoryPrediction(label, reference_id, taxonomy_name = None, confidence = None, metadata = None, class_pdf = None, track_reference_id = None) A prediction of a category. :param label: The label for this annotation (e.g. car, pedestrian, bicycle). :param reference_id: The reference ID of the image you wish to apply this annotation to. :param taxonomy_name: The name of the taxonomy this annotation conforms to. See :meth:`Dataset.add_taxonomy`. :param confidence: 0-1 indicating the confidence of the prediction. :param class_pdf: An optional complete class probability distribution on this prediction. Each value should be between 0 and 1 (inclusive), and sum up to 1 as a complete distribution. This can be useful for computing entropy to surface places where the model is most uncertain. :param metadata: Arbitrary key/value dictionary of info to attach to this annotation. Strings, floats and ints are supported best by querying and insights features within Nucleus. For more details see our `metadata guide `_. :param track_reference_id: A unique string to identify the prediction as part of a group. For instance, multiple "car" predictions across several dataset items may have the same `track_reference_id` such as "red_car". .. py:method:: from_json(payload) :classmethod: Instantiates annotation object from schematized JSON dict payload. .. py:method:: has_local_files_to_upload() Returns True if annotation has local files that need to be uploaded. Nearly all subclasses have no local files, so we default this to just return false. If the subclass has local files, it should override this method (but that is not the only thing required to get local upload of files to work.) .. py:method:: to_json() Serializes annotation object to schematized JSON string. .. py:method:: to_payload() Serializes annotation object to schematized JSON dict. .. py:class:: CuboidPrediction(label, position, dimensions, yaw, reference_id, confidence = None, annotation_id = None, metadata = None, class_pdf = None, track_reference_id = None) A prediction of 3D cuboid. :param label: The label for this annotation (e.g. car, pedestrian, bicycle) :type label: str :param position: The point at the center of the cuboid :type position: :class:`Point3D` :param dimensions: The length (x), width (y), and height (z) of the cuboid :type dimensions: :class:`Point3D` :param yaw: The rotation, in radians, about the Z axis of the cuboid :type yaw: float :param reference_id: User-defined ID of the image to which to apply this annotation. :type reference_id: str :param confidence: 0-1 indicating the confidence of the prediction. :param annotation_id: The annotation ID that uniquely identifies this annotation within its target dataset item. Upon ingest, a matching annotation id will be ignored by default, and updated if update=True for dataset.annotate. :type annotation_id: Optional[str] :param metadata: Arbitrary key/value dictionary of info to attach to this annotation. Strings, floats and ints are supported best by querying and insights features within Nucleus. For more details see our `metadata guide `_. :type metadata: Optional[str] :param class_pdf: An optional complete class probability distribution on this annotation. Each value should be between 0 and 1 (inclusive), and sum up to 1 as a complete distribution. This can be useful for computing entropy to surface places where the model is most uncertain. :param track_reference_id: A unique string to identify the prediction as part of a group. For instance, multiple "car" predictions across several dataset items may have the same `track_reference_id` such as "red_car". .. py:method:: from_json(payload) :classmethod: Instantiates annotation object from schematized JSON dict payload. .. py:method:: has_local_files_to_upload() Returns True if annotation has local files that need to be uploaded. Nearly all subclasses have no local files, so we default this to just return false. If the subclass has local files, it should override this method (but that is not the only thing required to get local upload of files to work.) .. py:method:: to_json() Serializes annotation object to schematized JSON string. .. py:method:: to_payload() Serializes annotation object to schematized JSON dict. .. py:class:: KeypointsPrediction(label, keypoints, names, skeleton, reference_id, confidence = None, annotation_id = None, metadata = None, class_pdf = None, track_reference_id = None) Prediction of keypoints. :param label: The label for this annotation (e.g. car, pedestrian, bicycle). :type label: str :param keypoints: The list of keypoints objects. :type keypoints: List[:class:`Keypoint`] :param names: A list that corresponds to the names of each keypoint. :type names: List[str] :param skeleton: A list of 2-length lists indicating a beginning and ending index for each line segment in the skeleton of this keypoint label. :type skeleton: List[List[int]] :param reference_id: User-defined ID of the image to which to apply this annotation. :type reference_id: str :param confidence: 0-1 indicating the confidence of the prediction. :param annotation_id: The annotation ID that uniquely identifies this annotation within its target dataset item. Upon ingest, a matching annotation id will be ignored by default, and updated if update=True for dataset.annotate. :type annotation_id: Optional[str] :param metadata: Arbitrary key/value dictionary of info to attach to this annotation. Strings, floats and ints are supported best by querying and insights features within Nucleus. For more details see our `metadata guide `_. :type metadata: Optional[Dict] :param class_pdf: An optional complete class probability distribution on this annotation. Each value should be between 0 and 1 (inclusive), and sum up to 1 as a complete distribution. This can be useful for computing entropy to surface places where the model is most uncertain. :param track_reference_id: A unique string to identify the prediction as part of a group. For instance, multiple "car" predictions across several dataset items may have the same `track_reference_id` such as "red_car". .. py:method:: from_json(payload) :classmethod: Instantiates annotation object from schematized JSON dict payload. .. py:method:: has_local_files_to_upload() Returns True if annotation has local files that need to be uploaded. Nearly all subclasses have no local files, so we default this to just return false. If the subclass has local files, it should override this method (but that is not the only thing required to get local upload of files to work.) .. py:method:: to_json() Serializes annotation object to schematized JSON string. .. py:method:: to_payload() Serializes annotation object to schematized JSON dict. .. py:class:: LinePrediction(label, vertices, reference_id, confidence = None, annotation_id = None, metadata = None, class_pdf = None, track_reference_id = None) Prediction of a line. :param label: The label for this prediction (e.g. car, pedestrian, bicycle). :type label: str :param vertices: The list of points making up the line. :type vertices: List[:class:`Point`] :param reference_id: User-defined ID of the image to which to apply this annotation. :type reference_id: str :param confidence: 0-1 indicating the confidence of the prediction. :param annotation_id: The annotation ID that uniquely identifies this annotation within its target dataset item. Upon ingest, a matching annotation id will be ignored by default, and updated if update=True for dataset.annotate. :type annotation_id: Optional[str] :param metadata: Arbitrary key/value dictionary of info to attach to this prediction. Strings, floats and ints are supported best by querying and insights features within Nucleus. For more details see our `metadata guide `_. :type metadata: Optional[Dict] :param class_pdf: An optional complete class probability distribution on this annotation. Each value should be between 0 and 1 (inclusive), and sum up to 1 as a complete distribution. This can be useful for computing entropy to surface places where the model is most uncertain. :param track_reference_id: A unique string to identify the prediction as part of a group. For instance, multiple "car" predictions across several dataset items may have the same `track_reference_id` such as "red_car". .. py:method:: from_json(payload) :classmethod: Instantiates annotation object from schematized JSON dict payload. .. py:method:: has_local_files_to_upload() Returns True if annotation has local files that need to be uploaded. Nearly all subclasses have no local files, so we default this to just return false. If the subclass has local files, it should override this method (but that is not the only thing required to get local upload of files to work.) .. py:method:: to_json() Serializes annotation object to schematized JSON string. .. py:method:: to_payload() Serializes annotation object to schematized JSON dict. .. py:class:: PolygonPrediction(label, vertices, reference_id, confidence = None, annotation_id = None, metadata = None, class_pdf = None, embedding_vector = None, track_reference_id = None) Prediction of a polygon. :param label: The label for this annotation (e.g. car, pedestrian, bicycle). :type label: str :param vertices List[:class:`Point`]: The list of points making up the polygon. :param reference_id: User-defined ID of the image to which to apply this annotation. :type reference_id: str :param confidence: 0-1 indicating the confidence of the prediction. :param annotation_id: The annotation ID that uniquely identifies this annotation within its target dataset item. Upon ingest, a matching annotation id will be ignored by default, and updated if update=True for dataset.annotate. :type annotation_id: Optional[str] :param metadata: Arbitrary key/value dictionary of info to attach to this annotation. Strings, floats and ints are supported best by querying and insights features within Nucleus. For more details see our `metadata guide `_. :type metadata: Optional[Dict] :param class_pdf: An optional complete class probability distribution on this annotation. Each value should be between 0 and 1 (inclusive), and sum up to 1 as a complete distribution. This can be useful for computing entropy to surface places where the model is most uncertain. :param embedding_vector: Custom embedding vector for this object annotation. If any custom object embeddings have been uploaded previously to this dataset, this vector must match the dimensions of the previously ingested vectors. :param track_reference_id: A unique string to identify the prediction as part of a group. For instance, multiple "car" predictions across several dataset items may have the same `track_reference_id` such as "red_car". .. py:method:: from_json(payload) :classmethod: Instantiates annotation object from schematized JSON dict payload. .. py:method:: has_local_files_to_upload() Returns True if annotation has local files that need to be uploaded. Nearly all subclasses have no local files, so we default this to just return false. If the subclass has local files, it should override this method (but that is not the only thing required to get local upload of files to work.) .. py:method:: to_json() Serializes annotation object to schematized JSON string. .. py:method:: to_payload() Serializes annotation object to schematized JSON dict. .. py:class:: PredictionList Wrapper class separating a list of predictions by type. .. py:class:: SceneCategoryPrediction(label, reference_id, taxonomy_name = None, confidence = None, metadata = None) A prediction of a category for a scene. :: from nucleus import SceneCategoryPrediction category = SceneCategoryPrediction( label="running", reference_id="scene_1", taxonomy_name="action", confidence=0.83, metadata={ "weather": "clear", }, ) :param label: The label for this annotation (e.g. action, subject, scenario). :param reference_id: The reference ID of the scene you wish to apply this annotation to. :param taxonomy_name: The name of the taxonomy this annotation conforms to. See :meth:`Dataset.add_taxonomy`. :param confidence: 0-1 indicating the confidence of the prediction. :param metadata: Arbitrary key/value dictionary of info to attach to this annotation. Strings, floats and ints are supported best by querying and insights features within Nucleus. For more details see our `metadata guide `_. .. py:method:: from_json(payload) :classmethod: Instantiates annotation object from schematized JSON dict payload. .. py:method:: has_local_files_to_upload() Returns True if annotation has local files that need to be uploaded. Nearly all subclasses have no local files, so we default this to just return false. If the subclass has local files, it should override this method (but that is not the only thing required to get local upload of files to work.) .. py:method:: to_json() Serializes annotation object to schematized JSON string. .. py:method:: to_payload() Serializes annotation object to schematized JSON dict. .. py:class:: SegmentationPrediction Predicted segmentation mask on a 2D image. :: from nucleus import SegmentationPrediction segmentation = SegmentationPrediction( mask_url="s3://your-bucket-name/pred-seg-masks/image_2_pred_mask_id_1.png", annotations=[ Segment(label="grass", index="1"), Segment(label="road", index="2"), Segment(label="bus", index="3", metadata={"vehicle_color": "yellow"}), Segment(label="tree", index="4") ], reference_id="image_2", annotation_id="image_2_pred_mask_1", ) :param mask_url: A URL pointing to the segmentation prediction mask which is accessible to Scale. This "URL" can also be a path to a local file. The mask is an HxW int8 array saved in PNG format, with each pixel value ranging from [0, N), where N is the number of possible classes (for semantic segmentation) or instances (for instance segmentation). The height and width of the mask must be the same as the original image. One example for semantic segmentation: the mask is 0 for pixels where there is background, 1 where there is a car, and 2 where there is a pedestrian. Another example for instance segmentation: the mask is 0 for one car, 1 for another car, 2 for a motorcycle and 3 for another motorcycle. The class name for each value in the mask is stored in the list of Segment objects passed for "annotations" :type mask_url: str :param annotations: The list of mappings between the integer values contained in mask_url and string class labels. In the semantic segmentation example above these would map that 0 to background, 1 to car and 2 to pedestrian. In the instance segmentation example above, 0 and 1 would both be mapped to car, 2 and 3 would both be mapped to motorcycle :type annotations: List[:class:`Segment`] :param reference_id: User-defined ID of the image to which to apply this annotation. :type reference_id: str :param annotation_id: For segmentation predictions, this value is ignored because there can only be one segmentation prediction per dataset item. Therefore regardless of annotation ID, if there is an existing segmentation on a dataset item, it will be ignored unless update=True is passed to :meth:`Dataset.annotate`, in which case it will be overwritten. Storing a custom ID here may be useful in order to tie this annotation to an external database, and its value will be returned for any export. :type annotation_id: Optional[str] .. py:method:: from_json(payload) :classmethod: Instantiates annotation object from schematized JSON dict payload. .. py:method:: has_local_files_to_upload() Check if the mask url is local and needs to be uploaded. .. py:method:: to_json() Serializes annotation object to schematized JSON string. .. py:method:: to_payload() Serializes annotation object to schematized JSON dict.