nucleus.scene

Frame	Collection of sensor data pertaining to a single time step.
LidarScene	Sequence of lidar pointcloud and camera images over time.
Scene	Helper class that provides a standard way to create an ABC using
VideoScene	Video or sequence of images over time.

class nucleus.scene.Frame(**kwargs)

Collection of sensor data pertaining to a single time step.

For 3D data, each Frame houses a sensor-to-data mapping and must have exactly one pointcloud with any number of camera images.

Parameters:

**kwargs (Dict[str, DatasetItem]) – Mappings from sensor name to dataset item. Each frame of a lidar scene must contain exactly one pointcloud and any number of images (e.g. from different angles).

Refer to our guide to uploading 3D data for more info!

add_item(item, sensor_name)

Adds DatasetItem object to frame as sensor data.

Parameters:

• item (DatasetItem) – Pointcloud or camera image item to add.

• sensor_name (str) – Name of the sensor, e.g. “lidar” or “front_cam.”

Return type:

None

classmethod from_json(payload)

Instantiates frame object from schematized JSON dict payload.

Parameters:

payload (dict) –

get_item(sensor_name)

Fetches the DatasetItem object associated with the given sensor.

Parameters:

sensor_name (str) – Name of the sensor, e.g. “lidar” or “front_cam.”

Returns:

DatasetItem object pertaining to the sensor.

Return type:

DatasetItem

get_items()

Fetches all items in the frame.

Returns:

List of all DatasetItem objects in the frame.

Return type:

List[DatasetItem]

get_sensors()

Fetches all sensor names of the frame.

Returns:

List of all sensor names of the frame.

Return type:

List[str]

to_payload()

Serializes frame object to schematized JSON dict.

Return type:

dict

class nucleus.scene.LidarScene

Sequence of lidar pointcloud and camera images over time.

Nucleus 3D datasets are comprised of LidarScenes, which are sequences of lidar pointclouds and camera images over time. These sequences are in turn comprised of Frames.

By organizing data across multiple sensors over time, LidarScenes make it easier to interpret pointclouds, allowing you to see objects move over time by clicking through frames and providing context in the form of corresponding images.

You can think of scenes and frames as nested groupings of sensor data across time:

• LidarScene for a given location

  • Frame at timestep 0

    • DatasetItem of pointcloud

    • DatasetItem of front camera image

    • DatasetItem of rear camera image

  • Frame at timestep 1

    • …

  • …

• LidarScene for another location

  • …

LidarScenes are uploaded to a Dataset with any accompanying metadata. Frames do not accept metadata, but each of its constituent DatasetItems does.

Note: Uploads with a different number of frames/items will error out (only on scenes that now differ). Existing scenes are expected to retain the same structure, i.e. the same number of frames, and same items per frame. If a scene definition is changed (for example, additional frames added) the update operation will be ignored. If you would like to alter the structure of a scene, please delete the scene and re-upload.

Parameters:

• reference_id (str) – User-specified identifier to reference the scene.

• frames (Optional[List[Frame]]) – List of frames to be a part of the scene. A scene can be created before frames or items have been added to it, but must be non-empty when uploading to a Dataset.

• metadata (Optional[Dict]) –

  Optional metadata to include with the scene.

  Coordinate metadata may be provided to enable the Map Chart in the Nucleus Dataset charts page. These values can be specified as { “lat”: 52.5, “lon”: 13.3, … }.

  Context Attachments may be provided to display the attachments side by side with the dataset item in the Detail View by specifying { “context_attachments”: [ { “attachment”: ‘https://example.com/1’ }, { “attachment”: ‘https://example.com/2’ }, … ] }.

Refer to our guide to uploading 3D data for more info!

add_frame(frame, index, update=False)

Adds frame to scene at the specified index.

Parameters:

• frame (Frame) – Frame object to add.

• index (int) – Serial index at which to add the frame.

• update (bool) – Whether to overwrite the frame at the specified index, if it exists. Default is False.

Return type:

None

add_item(index, sensor_name, item)

Adds DatasetItem to the specified frame as sensor data.

Parameters:

• index (int) – Serial index of the frame to which to add the item.

• item (DatasetItem) – Pointcloud or camera image item to add.

• sensor_name (str) – Name of the sensor, e.g. “lidar” or “front_cam.”

Return type:

None

classmethod from_json(payload, client=None, skip_validate=False)

Instantiates scene object from schematized JSON dict payload.

Parameters:

• payload (dict) –

• client (Optional[nucleus.NucleusClient]) –

• skip_validate (Optional[bool]) –

get_frame(index)

Fetches the Frame object at the specified index.

Parameters:

index (int) – Serial index for which to retrieve the Frame.

Returns:

Frame object at the specified index.

Return type:

Frame

get_frames()

Fetches a sorted list of Frames of the scene.

Returns:

List of Frames, sorted by index ascending.

Return type:

List[Frame]

get_item(index, sensor_name)

Fetches the DatasetItem object of the given frame and sensor.

Parameters:

• index (int) – Serial index of the frame from which to fetch the item.

• sensor_name (str) – Name of the sensor, e.g. “lidar” or “front_cam.”

Returns:

DatasetItem object of the frame and sensor.

Return type:

DatasetItem

get_items()

Fetches all items in the scene.

Returns:

Unordered list of all DatasetItem objects in the scene.

Return type:

List[DatasetItem]

get_items_from_sensor(sensor_name)

Fetches all DatasetItem objects of the given sensor.

Parameters:

sensor_name (str) – Name of the sensor, e.g. “lidar” or “front_cam.”

Returns:

List of DatasetItem objects associated with the specified sensor.

Return type:

List[DatasetItem]

get_sensors()

Fetches all sensor names of the scene.

Returns:

List of all sensor names associated with frames in the scene.

Return type:

List[str]

info()

Fetches information about the scene.

Returns:

Payload containing:

{
    "reference_id": str,
    "length": int,
    "num_sensors": int
}

to_json()

Serializes scene object to schematized JSON string.

Return type:

str

to_payload()

Serializes scene object to schematized JSON dict.

Return type:

dict

class nucleus.scene.Scene

Helper class that provides a standard way to create an ABC using inheritance.

add_frame(frame, index, update=False)

Adds frame to scene at the specified index.

Parameters:

• frame (Frame) – Frame object to add.

• index (int) – Serial index at which to add the frame.

• update (bool) – Whether to overwrite the frame at the specified index, if it exists. Default is False.

Return type:

None

add_item(index, sensor_name, item)

Adds DatasetItem to the specified frame as sensor data.

Parameters:

• index (int) – Serial index of the frame to which to add the item.

• item (DatasetItem) – Pointcloud or camera image item to add.

• sensor_name (str) – Name of the sensor, e.g. “lidar” or “front_cam.”

Return type:

None

classmethod from_json(payload, client=None, skip_validate=False)

Instantiates scene object from schematized JSON dict payload.

Parameters:

• payload (dict) –

• client (Optional[nucleus.NucleusClient]) –

• skip_validate (Optional[bool]) –

get_frame(index)

Fetches the Frame object at the specified index.

Parameters:

index (int) – Serial index for which to retrieve the Frame.

Returns:

Frame object at the specified index.

Return type:

Frame

get_frames()

Fetches a sorted list of Frames of the scene.

Returns:

List of Frames, sorted by index ascending.

Return type:

List[Frame]

get_item(index, sensor_name)

Fetches the DatasetItem object of the given frame and sensor.

Parameters:

• index (int) – Serial index of the frame from which to fetch the item.

• sensor_name (str) – Name of the sensor, e.g. “lidar” or “front_cam.”

Returns:

DatasetItem object of the frame and sensor.

Return type:

DatasetItem

get_items()

Fetches all items in the scene.

Returns:

Unordered list of all DatasetItem objects in the scene.

Return type:

List[DatasetItem]

get_items_from_sensor(sensor_name)

Fetches all DatasetItem objects of the given sensor.

Parameters:

sensor_name (str) – Name of the sensor, e.g. “lidar” or “front_cam.”

Returns:

List of DatasetItem objects associated with the specified sensor.

Return type:

List[DatasetItem]

get_sensors()

Fetches all sensor names of the scene.

Returns:

List of all sensor names associated with frames in the scene.

Return type:

List[str]

info()

Fetches information about the scene.

Returns:

Payload containing:

{
    "reference_id": str,
    "length": int,
    "num_sensors": int
}

to_json()

Serializes scene object to schematized JSON string.

Return type:

str

to_payload()

Serializes scene object to schematized JSON dict.

Return type:

dict

class nucleus.scene.VideoScene

Video or sequence of images over time.

Nucleus video datasets are comprised of VideoScenes. These can be comprised of a single video, or a sequence of DatasetItems which are equivalent to frames.

VideoScenes are uploaded to a Dataset with any accompanying metadata. Each of DatasetItems representing a frame also accepts metadata.

Note: Updates with different items will error out (only on scenes that now differ). Existing video are expected to retain the same frames, and only metadata can be updated. If a video definition is changed (for example, additional frames added) the update operation will be ignored. If you would like to alter the structure of a video scene, please delete the scene and re-upload.

Parameters:

• reference_id (str) – User-specified identifier to reference the scene.

• frame_rate (Optional[int]) – Required if uploading items. Frame rate of the video.

• video_location (Optional[str]) – Required if not uploading items. The remote URL containing the video MP4. Remote formats supported include any URL (http:// or https://) or URIs for AWS S3, Azure, or GCS (i.e. s3://, gcs://).

• items (Optional[List[DatasetItem]]) – Required if not uploading video_location. List of items representing frames, to be a part of the scene. A scene can be created before items have been added to it, but must be non-empty when uploading to a Dataset. A video scene can contain a maximum of 3000 items.

• metadata (Optional[Dict]) –

  Optional metadata to include with the scene.

  Coordinate metadata may be provided to enable the Map Chart in the Nucleus Dataset charts page. These values can be specified as { “lat”: 52.5, “lon”: 13.3, … }.

  Context Attachments may be provided to display the attachments side by side with the dataset item in the Detail View by specifying { “context_attachments”: [ { “attachment”: ‘https://example.com/1’ }, { “attachment”: ‘https://example.com/2’ }, … ] }.

Refer to our guide to uploading video data for more info!

add_item(item, index=None, update=False)

Adds DatasetItem to the specified index for videos uploaded as an array of images.

Parameters:

• item (DatasetItem) – Video item to add.

• index (Optional[int]) – Serial index at which to add the item.

• update (bool) – Whether to overwrite the item at the specified index, if it exists. Default is False.

Return type:

None

classmethod from_json(payload, client=None)

Instantiates scene object from schematized JSON dict payload.

Parameters:

• payload (dict) –

• client (Optional[nucleus.NucleusClient]) –

get_item(index)

Fetches the DatasetItem at the specified index for videos uploaded as an array of images.

Parameters:

index (int) – Serial index for which to retrieve the DatasetItem.

Returns:

DatasetItem at the specified index.

Return type:

DatasetItem

get_items()

Fetches a sorted list of DatasetItems of the scene for videos uploaded as an array of images.

Returns:

List of DatasetItems, sorted by index ascending.

Return type:

List[DatasetItem]

info()

Fetches information about the video scene.

Returns:

Payload containing:

{
    "reference_id": str,
    "length": Optional[int],
    "frame_rate": int,
    "video_url": Optional[str],
}

to_json()

Serializes scene object to schematized JSON string.

Return type:

str

to_payload()

Serializes scene object to schematized JSON dict.

Return type:

dict