d3d.abstraction

This module contains definition of basic data structures.

class d3d.abstraction.CameraMetadata(int width, int height, ndarray distort_coeffs, ndarray intri_matrix, float mirror_coeff)

Bases: object

This class represents intrinsic parameters of a camera

Parameters

  • width – Width of the image

  • height – Height of the image

  • distort_coeffs – Distortion coefficients

  • intri_matrix – Intrisic matrix of the camera

  • mirror_coeff – Mirror coefficient for stereo setup

distort_coeffs

numpy.ndarray

Coefficients of camera distortion model, follow OpenCV format

Type

distort_coeffs

height

‘int’

Height of the camera image

Type

height

intri_matrix

numpy.ndarray

Original intrinsic matrix used for cv2.undistortPoints

Type

intri_matrix

mirror_coeff

‘float’

Coefficient of mirror equation (as used in MEI camera model)

Type

mirror_coeff

width

‘int’

Width of the camera image

Type

width

class d3d.abstraction.EgoPose(position, orientation, position_var=None, orientation_var=None)

Bases: object

This object is used to store dynamic state of ego vehicle. All value is represented in earth-fixed coordinate (absolute coordinate).

Parameters

  • position – position of ego sensor, [x, y, z]

  • orientation – orientation of ego sensor, in format of [x, y, z, w] quaternion of scipy Rotation object

  • position_var – positional variance of ego sensor, [var_x, var_y, var_z]

  • orientation_var – orientation variance of ego sensor

homo(self)

Convert the pose to a homogeneous matrix representation

orientation

The orientation of the ego sensor

orientation_var

numpy.ndarray

Variance of the estimation of the sensor orientation

Type

orientation_var

position

numpy.ndarray

The position of the ego sensor

Type

position

position_var

numpy.ndarray

Variance of the estimation of the sensor position

Type

position_var

class d3d.abstraction.LidarMetadata

Bases: object

This class represents intrinsic parameters of a lidar

class d3d.abstraction.ObjectTag(labels, mapping=None, scores=None)

Bases: object

This class stands for label tags associate with object target. This class can contains multiple estimated classes with separate confidence scores.

Parameters

  • labels – A label or list of labels as enum object, enum name or enum id.

  • mapping – The enum object defining the label classes.

  • scores – Scores corresponding to the input labels.

deserialize(type cls, data)

Deserialize data from python primitives

labels

‘vector[int]’

The ids of the labels, sorted by score in descending order

Type

labels

mapping

object

The Enum class defining the label classes

Type

mapping

scores

‘vector[float]’

The ids of the labels, sorted by score in descending order

Type

scores

serialize(self)

Serialize this object to primitives

class d3d.abstraction.ObjectTarget3D(position, orientation, dimension, tag, tid=0, position_var=None, orientation_var=None, dimension_var=None, aux=None)

Bases: object

This class stands for a target in cartesian coordinate. The body coordinate is FLU (front-left-up).

Parameters

  • position – Position of object center (x,y,z)

  • orientation – Object heading (direction of x-axis attached on body) with regard to x-axis of the world at the object center.

  • dimension – Length of the object in 3 dimensions (lx,ly,lz)

  • tag – Classification information of the object

  • tid – ID of the object used for tracking (optional), 0 means no tracking id assigned

  • position_var – The uncertainty of target position

  • orientation_var – The uncertainty of target orientation

  • dimension_var – The uncertainty of target dimension

aux

dict

Auxiliary data attached to the object

Type

aux

box_iou(self, ObjectTarget3D other)
corners

Convert the bounding box representation to coorindate of 8 corner points

crop_points(self, ndarray cloud)
deserialize(type cls, data)

Deserialize data from python primitives

dimension

Dimension of the target

dimension_var

Variance of dimension estimation of the target

orientation

Orientation of the target

orientation_var

‘float’

Variance of orientation of the target. This API may be changed in future

Type

orientation_var

points_distance(self, ndarray cloud)
position

Position of the (center of) target

position_var

Positional variance of the (center of) target

serialize(self)

Serialize this object to python primitives

tag

d3d.abstraction.ObjectTag

The tag attached to the target

Type

tag

tag_top

Return the object of the target’s top tag

tag_top_score

Return the score of the target’s top tag

tid

‘unsigned long long’

The unique id of the target (across frames). tid = 0 means no id assigned, so valid tid should be greater than 1.

Type

tid

tid64

Return base64 represented tracking id

to_numpy(self, unicode box_type=u'ground') → ndarray

Convert the object to numpy array representation

Parameters

box_type – The type of box representation * ground: use the representation of bird’s eye view 2D projection

yaw

Return the rotation angle around z-axis (ignoring rotations in other two directions)

class d3d.abstraction.PinMetadata(float lon, float lat)

Bases: object

This class represents a ground-fixed coordinate. The coordinate can be in WGS-84 or local UTM coordinate system.

Parameters

  • lon – Longitude coordinate value

  • lat – Latitude coordinate value

lat

‘float’

Latitude coordinate of the pin

Type

lat

lon

‘float’

Longitude coordinate of the pin

Type

lon

class d3d.abstraction.RadarMetadata

Bases: object

This class represents intrinsic parameters of a radar

class d3d.abstraction.Target3DArray(iterable=[], frame=None, timestamp=0)

Bases: list

Target3DArray stores an array of ObjectTarget3D or TrackingTarget3D represented in the frame of certain senser at certain time.

Parameters

  • iterable – List of targets

  • frame – Sensor frame that the box parameters used. None means base frame (consistent with TransformSet)

  • timestamp – The timestamp of the target properties

crop_points(self, ndarray cloud)
deserialize(type cls, data)

Deserialize data from python primitives

dump(self, output)

Serialize the array and dump it into file

Parameters

output – output file-like object or file path

filter(self, predicate)

Filter the list of objects by predicate

filter_position(self, float x_min=float(u'nan'), float x_max=float(u'nan'), float y_min=float(u'nan'), float y_max=float(u'nan'), float z_min=float(u'nan'), float z_max=float(u'nan'))

Filter the list of objects by the center position

Parameters

  • x_min – Minimum x coordinate

  • x_max – Maximum x coordinate

  • y_min – Minimum y coordinate

  • y_max – Maximum y coordinate

  • z_min – Minimum z coordinate

  • z_max – Maximum z coordinate

filter_score(self, score)

Filter the list by select only objects higher than certain score

Parameters

score – The minimun score for tag_top_score field

filter_tag(self, tags)

Filter the list by select only objects with given tags

Parameters

tags – None means no filter, otherwise str/enum or list of str/enum

frame

unicode

The transform frame which the targets lie in

Type

frame

load(type cls, file)

Load the array form a binary file created by dump()

Parameters

file – path of input file or file-like object to be loaded

paint_label(self, ndarray cloud, ndarray semantics)
serialize(self)

Serialize this object to python primitives

sort_by_score(self, reverse=False)

Sort the box list (in place) by the score

Parameters

reverse – sorting is done ascendingly by default, reverse means descending

timestamp

‘unsigned long long’

The timestamp of when the targets are annotated or reported. It’s represented by unix timestamp in milliseconds

Type

timestamp

to_numpy(self, unicode box_type=u'ground') → ndarray

Convert the object array to numpy array representation

  • ground: use the representation of bird’s eye view 2D projection

to_torch(self, box_type='ground')

Convert the object array to PyTorch Tensor representation

class d3d.abstraction.TrackingTarget3D(position, orientation, dimension, velocity, angular_velocity, tag, tid=0, position_var=None, orientation_var=None, dimension_var=None, velocity_var=None, angular_velocity_var=None, history=None, aux=None)

Bases: d3d.abstraction.ObjectTarget3D

This class stands for a tracked target in cartesian coordinate. The body coordinate is FLU (front-left-up).

Parameters

  • velocity – Velocity of the object (vx,vy,vz)

  • angular_velocity – Angular velocity of the object (wx,wy,wz)

  • velocity_var – The uncertainty of the target velocity

  • angular_velocity – The uncertainty of the target angular velocity

  • history – The time of the object being tracked

angular_velocity

Angular velocity of the target

angular_velocity_var

Variance of angular velocity estimation of the target

deserialize(type cls, data)
history

‘float’

Tracked time of this target in seconds

Type

history

serialize(self)
to_numpy(self, unicode box_type=u'ground') → ndarray
velocity

Velocity of the (center of) target

velocity_var

Variance of velocity estimation of the target

class d3d.abstraction.TransformSet(unicode base_frame)

Bases: object

This object load a collection of intrinsic and extrinsic parameters All extrinsic parameters are stored as transform from base frame to its frame In this class, we require all frames to use FLU coordinate including camera frame

Parameters

base_frame – name of base frame used by extrinsics

base_frame

unicode

The default frame for the transform set

Type

base_frame

dump(self, output)

Serialize the transform collection and dump it into file

Parameters

output – output file-like object or file path

extrinsics

dict

Type

extrinsics

frames

Report registered frame names (excluding base_frame)

get_extrinsic(self, unicode frame_to=None, unicode frame_from=None) → ndarray
Parameters

frame_from – If set to None, then the source frame is base frame

intrinsics

dict

This dictionary defines all the valid sensor frames and store their projection matrix (for camera)

Type

intrinsics

intrinsics_meta

dict

This dictionary defines the related metadata for sensors

Type

intrinsics_meta

load(type cls, file)

Load the transform collection form a binary file created by dump()

Parameters

file – path of input file or file-like object to be loaded

project_points_to_camera(self, ndarray points, unicode frame_to, unicode frame_from=None, bool remove_outlier=True, bool return_dmask=False) → tuple
Parameters

  • remove_outlier – If set to True, the mask will be applied, i.e. only points that fall into image view will be returned

  • return_dmask – also return the mask for z > 0 only

Returns

return points, mask and dmask if required. The masks are array of indices

set_extrinsic(self, transform, unicode frame_to=None, unicode frame_from=None) → void

All extrinsics are stored as transform convert point from frame_from to frame_to :param frame_from: If set to None, then the source frame is base frame :param frame_to: If set to None, then the target frame is base frame

set_intrinsic_camera(self, unicode frame_id, ndarray transform, size, bool rotate=True, distort_coeffs=[], ndarray intri_matrix=None, float mirror_coeff=float(u'nan')) → void

Set camera intrinsics :param size: (width, height) :param rotate: if True, then transform will append an axis rotation (Front-Left-Up to Right-Down-Front) :param distort_coeffs: distortion coefficients, see [OpenCV](https://docs.opencv.org/3.4/d9/d0c/group__calib3d.html) for details :param intri_matrix: intrinsic matrix (in general camera model) :param mirror_coeff: the xi coefficient in MEI camera model. Reference: Single View Point OmnidirectionalCamera Calibration from Planar Grids

set_intrinsic_general(self, unicode frame_id, metadata=None) → void

Set intrinsic for a general sensor. This is used for marking existence of a frame

set_intrinsic_lidar(self, unicode frame_id) → void
set_intrinsic_map_pin(self, unicode frame_id, lon=float(u'nan'), lat=float(u'nan')) → void
set_intrinsic_pinhole(self, unicode frame_id, size, float cx, float cy, float fx, float fy, float s=0, distort_coeffs=[]) → void

Set camera intrinsics with pinhole model parameters :param s: skew coefficient

set_intrinsic_radar(self, unicode frame_id) → void
transform_objects(self, Target3DArray objects, unicode frame_to=None) → Target3DArray

Change the representing frame of a object array

transform_points(self, ndarray points, unicode frame_to, unicode frame_from=None) → ndarray

Convert point cloud from frame_from to frame_to