Camera extrinsics per frame (to get global frame coordinates)

[sanaebenjeddi]

Hi!

I’m working on converting image coordinates to real-world coordinates for each frame captured by the SDC’s camera, so image coordinates * intrinsics * extrinsics.

Since the extrinsics in CameraCalibration is a fixed matrix for each scene, transforming the camera frame to the vehicle frame, I’m wondering how the camera extrinsics per frame are stored or how they can be calculated to transform to the global frame.

Thanks! :)

[ctyfang]

Based on the code here:

waymo-open-dataset/tutorial/tutorial_camera_only.ipynb

Line 367 in 5f8a1cd

" calibration: Camera calibration details (including intrinsics/extrinsics).\n",

I believe we can use the extrinsics to transform from camera frame to vehicle frame, and then use the vehicle_to_world frame stored in the camera message to get to world frame.