Skip to main content

Wrist RGB Camera

The EgoSuite rig optionally includes cameras mounted on the left and right wrists. In the MCAP files, these cameras are represented by dedicated topics for:

  • Compressed video streams
  • Raw compressed video streams when source raw MP4 files are included
  • Per‑camera intrinsics (calibration)
  • Per‑camera extrinsics (pose per frame transforms)

Topic & Message Type​

The following topics are registered for wrist cameras. All channels use protobuf encoding:

  • Left Wrist Camera

    • Video topic: /sensor/camera/left_wrist/video
    • Message type: foxglove.CompressedVideo
    • Raw video topic: /sensor/camera/left_wrist/video_raw
    • Raw video type: foxglove.CompressedVideo
    • Intrinsic topic: /sensor/camera/left_wrist/intrinsic
    • Intrinsic type: foxglove.CameraCalibration
    • Extrinsic topic: /sensor/camera/left_wrist/extrinsic
    • Extrinsic type: foxglove.FrameTransforms

  • Right Wrist Camera

    • Video topic: /sensor/camera/right_wrist/video
    • Message type: foxglove.CompressedVideo
    • Raw video topic: /sensor/camera/right_wrist/video_raw
    • Raw video type: foxglove.CompressedVideo
    • Intrinsic topic: /sensor/camera/right_wrist/intrinsic
    • Intrinsic type: foxglove.CameraCalibration
    • Extrinsic topic: /sensor/camera/right_wrist/extrinsic
    • Extrinsic type: foxglove.FrameTransforms

Wrist camera video topics carry compressed video frames as foxglove.CompressedVideo messages. You can inspect, play back using LW-VIZ or export these streams with LW-EgoSuite Devkit.

Camera Intrinsics​

Camera publishes its calibration on a dedicated foxglove.CameraCalibration topic. For field definitions (including width, height, intrinsic matrix K, distortion model and parameters D, rectification matrix R, projection matrix P, and frame_id), see CameraCalibration documentation.

Camera Extrinsics​

Camera extrinsic info is expressed as foxglove.FrameTransforms message. For field definitions (e.g. parent_frame_id, child_frame_id, translation, rotation), see FrameTransform documentation.

In EgoSuite MCAP files, wrist camera extrinsics are generated from world-frame position and orientation in each wrist camera params file. The message uses parent_frame_id = "world" and child_frame_id set to the concrete camera frame (left_wrist_camera or right_wrist_camera). The converter additionally rotates the wrist camera orientation by 180 degrees around its local Z axis before writing the transform.

For details on computing the W2C (world-to-camera) extrinsic matrix from the C2W pose, refer to the Head Cameras page, which provides the same matrix inversion and projection formulas applicable to all EgoSuite cameras.

Coordinate Frames​

Wrist cameras use the same coordinate conventions as head cameras and the rest of EgoSuite. For detailed descriptions, refer to the Head Cameras - Coordinate frames section.

In summary:

  • World Frame: All pose data is expressed in a shared world frame.
  • Camera Frame (OpenCV convention):
    • zz axis points forward from the camera.
    • xx axis points to the right in the image.
    • yy axis points down in the image.

With intrinsics and extrinsics combined, you can project 3D hand or object points into wrist camera images or lift image measurements back to 3D in the world frame.

Typical Usage​

Common use cases for wrist camera data include:

  • Studying close‑range hand–object interactions from each wrist's egocentric perspective.
  • Aligning wrist camera images with 3D hand keypoints for grasp analysis.
  • Fusing multiple camera views (head and wrist) using the shared transform tree in foxglove.FrameTransforms.