Skip to main content

Wrist Camera

The EgoSuite rig optionally includes cameras mounted on the left and right wrists. In the MCAP files, these cameras are represented by dedicated topics for:

  • Compressed video streams
  • Per‑camera intrinsics (calibration)
  • Per‑camera extrinsics (pose per frame transforms)

Topic & Message Type​

The following topics are registered for wrist cameras. All channels use protobuf encoding:

  • Left Wrist Camera

    • Video topic: /sensor/camera/left_wrist/video
    • Message type: foxglove.CompressedVideo
    • Intrinsic topic: /sensor/camera/left_wrist/intrinsic
    • Intrinsic type: foxglove.CameraCalibration
    • Extrinsic topic: /sensor/camera/left_wrist/extrinsic
    • Extrinsic type: foxglove.FrameTransforms

  • Right Wrist Camera

    • Video topic: /sensor/camera/right_wrist/video
    • Message type: foxglove.CompressedVideo
    • Intrinsic topic: /sensor/camera/right_wrist/intrinsic
    • Intrinsic type: foxglove.CameraCalibration
    • Extrinsic topic: /sensor/camera/right_wrist/extrinsic
    • Extrinsic type: foxglove.FrameTransforms

Wrist camera video topics carry compressed video frames as foxglove.CompressedVideo messages. For details, please see here. You can inspect, play back using Foxglove Studio or export these streams with LW-Egosuite-Devkit.

Camera Intrinsics​

Camera publishes its calibration on a dedicated foxglove.CameraCalibration topic. For field definitions (including width, height, intrinsic matrix K, distortion model and parameters D, rectification matrix R, projection matrix P, and frame_id), see CameraCalibration documentation.

Camera Extrinsics​

Camera extrinsic info is expressed as foxglove.FrameTransforms message. For field definitions (e.g. parent_frame_id, child_frame_id, translation, rotation), see FrameTransform documentation.

In EgoSuite MCAP files, camera extrinsic message represents the position and orientation of the cameras in world frame. The message uses parent_frame_id = world and child_frame_id = camera. Wrist cameras are connected to the world frame through intermediate upper-body joints (shoulders, elbows, wrists) encoded in the transform tree.

For details on computing the W2C (world-to-camera) extrinsic matrix from the C2W pose, refer to the Head Cameras page, which provides the same matrix inversion and projection formulas applicable to all EgoSuite cameras.

Coordinate Frames​

Wrist cameras use the same coordinate conventions as head cameras and the rest of EgoSuite. For detailed descriptions, refer to the Head Cameras - Coordinate frames section.

In summary:

  • World Frame: All pose data is expressed in a shared world frame.
  • Camera Frame (OpenCV convention):
    • zz axis points forward from the camera.
    • xx axis points to the right in the image.
    • yy axis points down in the image.

With intrinsics and extrinsics combined, you can project 3D hand or object points into wrist camera images or lift image measurements back to 3D in the world frame.

Typical Usage​

Common use cases for wrist camera data include:

  • Studying close‑range hand–object interactions from each wrist's egocentric perspective.
  • Aligning wrist camera images with 3D hand keypoints for grasp analysis.
  • Fusing multiple camera views (head and wrist) using the shared transform tree in foxglove.FrameTransforms.