Overview

MCAP Data is the native EgoSuite recording format. Each .mcap file contains multiple time-synchronized topics: video streams, poses, annotations, session metadata, and optionally audio. The exact set of topics depends on the capture rig (e.g. head vs. wrist cameras).

All EgoSuite-native topics use Protobuf encoding and share a common Header (common.header.Header): count, ts (Unix time in nanoseconds), sender. Message types and proto definitions live in the lw-egosuite-msg package (protos/ and generated code).

Topic convention

• Some topics use third-party schemas (e.g. Foxglove foxglove.*, tf2_msgs.*) or are special-cased.
• Most EgoSuite messages carry a header field of type common.header.Header with:
  • count (uint64)
  • ts (int64, Unix time in nanoseconds)
  • sender (string)
    Third-party messages such as foxglove.* and tf2_msgs.* do not use this header.

Registered topics

Topic	Proto Message
/session/metadata	session.SessionMetadata
/annotation/semantic_segments	annotation.SemanticSegment
/pose/body	pose.BodyFrame
/pose/upper_body	pose.UpperBodyFrame
/pose/lower_body	pose.LowerBodyFrame
/pose/head	pose.HeadFrame
/pose/headcam	pose.HeadCamFrame
/pose/left_hand	pose.LeftHandFrame
/pose/right_hand	pose.RightHandFrame
/pose/right_eye_cam	pose.RightEyeCamFrame
/sensor/camera/head_left/video	foxglove.CompressedVideo
/sensor/camera/head_right/video	foxglove.CompressedVideo
/sensor/camera/head_left/intrinsic	foxglove.CameraCalibration
/sensor/camera/head_right/intrinsic	foxglove.CameraCalibration
/sensor/camera/head_left/extrinsic	foxglove.FrameTransforms
/sensor/camera/head_right/extrinsic	foxglove.FrameTransforms
— Optional —
/sensor/camera/left_wrist/video	foxglove.CompressedVideo
/sensor/camera/right_wrist/video	foxglove.CompressedVideo
/sensor/camera/left_wrist/intrinsic	foxglove.CameraCalibration
/sensor/camera/right_wrist/intrinsic	foxglove.CameraCalibration
/sensor/camera_raw/head_left/video	foxglove.CompressedVideo
/sensor/camera_raw/head_right/video	foxglove.CompressedVideo
/sensor/camera_raw/left_wrist/video	foxglove.CompressedVideo
/sensor/camera_raw/right_wrist/video	foxglove.CompressedVideo
/sensor/camera_raw/head_left/calibration	sensor.camera.RawCameraCalibration
/sensor/camera_raw/head_right/calibration	sensor.camera.RawCameraCalibration
/sensor/camera_raw/left_wrist/calibration	sensor.camera.RawCameraCalibration
/sensor/camera_raw/right_wrist/calibration	sensor.camera.RawCameraCalibration
/sensor/camera/head_depth/image	foxglove.CompressedImage
/sensor/camera/head_depth/intrinsic	foxglove.CameraCalibration
/sensor/camera/head_depth/extrinsic	foxglove.FrameTransforms
/sensor/audio	foxglove.RawAudio
/annotation/bad_frame/pose/hand	annotation.BadFrameHand
/annotation/bad_frame/pose/upper_body	annotation.BadFrameBody
/annotation/bad_frame/pose/lower_body	annotation.BadFrameBody
/annotation/bad_frame/camera/head_left	annotation.BadFrameImage
/annotation/bad_frame/camera/left_wrist	annotation.BadFrameImage
/annotation/bad_frame/camera/right_wrist	annotation.BadFrameImage

Navigation

Use the sidebar to open detailed docs:

Section	Topics / content
Metadata	/session/metadata — session/episode metadata (devices, global time, task info, operator)
Pose	Head, hand, body pose and related frames
Sensor	RGB camera (video, raw video, intrinsic, extrinsic), depth camera (depth image + intrinsic/extrinsic, liftable to 3D point cloud), Audio
Semantic Annotation	/annotation/semantic_segments — task description, segmented actions, skill names, and time bounds

For Devkit installation, CLI/API usage (e.g. lw-egosuite convert, lw-egosuite export-video), and visualization, see EgoSuite Devkit.