Overview

MCAP Data is the native EgoSuite recording format. Each .mcap file contains multiple time-synchronized topics: video streams, poses, annotations, session metadata, and optionally audio and point cloud. The exact set of topics depends on the capture rig (e.g. head vs. wrist cameras).

All EgoSuite-native topics use Protobuf encoding and share a common Header (common.header.Header): count, ts (Unix time in nanoseconds), sender. Message types and proto definitions live in the lw-egosuite-msg package (protos/ and generated code).

Topic convention

• Some topics use third-party schemas (e.g. Foxglove foxglove.*, tf2_msgs.*) or are special-cased.
• Most EgoSuite messages carry a header field of type common.header.Header with:
  • count (uint64)
  • ts (int64, Unix time in nanoseconds)
  • sender (string)
    Third-party messages such as foxglove.* and tf2_msgs.* do not use this header.

Registered topics

Topic	Proto Message
/session/metadata	session.metadata.SessionMetadata
/pose/body	pose.BodyFrame
/pose/head_pose	pose.HeadPoseFrame
/pose/headcam_pose	pose.HeadcamPoseFrame
/pose/left_hand	pose.LeftHandFrame
/pose/right_hand	pose.RightHandFrame
/pose/right_eye_cam	pose.RightEyeCamFrame
/annotation/segments	annotation.segments.AnnotationSegment
/sensor/camera/head_left/video	foxglove.CompressedVideo
/sensor/camera/head_right/video	foxglove.CompressedVideo
/sensor/camera/head_left/video_raw	foxglove.CompressedVideo
/sensor/camera/head_right/video_raw	foxglove.CompressedVideo
/sensor/camera/head_left/intrinsic	foxglove.CameraCalibration
/sensor/camera/head_right/intrinsic	foxglove.CameraCalibration
/sensor/camera/head_left/extrinsic	foxglove.FrameTransforms
/sensor/camera/head_right/extrinsic	foxglove.FrameTransforms
— Optional —
/sensor/camera/left_wrist/video	foxglove.CompressedVideo
/sensor/camera/right_wrist/video	foxglove.CompressedVideo
/sensor/camera/left_wrist/video_raw	foxglove.CompressedVideo
/sensor/camera/right_wrist/video_raw	foxglove.CompressedVideo
/sensor/camera/left_wrist/intrinsic	foxglove.CameraCalibration
/sensor/camera/right_wrist/intrinsic	foxglove.CameraCalibration
/sensor/camera/left_wrist/extrinsic	foxglove.FrameTransforms
/sensor/camera/right_wrist/extrinsic	foxglove.FrameTransforms
/sensor/camera/head_depth/image	foxglove.CompressedImage
/sensor/camera/head_depth/intrinsic	foxglove.CameraCalibration
/sensor/camera/head_depth/extrinsic	foxglove.FrameTransforms
/pointcloud	foxglove.PointCloud
/audio	foxglove.RawAudio

Navigation

Use the sidebar to open detailed docs:

Section	Topics / content
Metadata	/session/metadata — session/episode metadata (devices, global time, task info, operator)
Pose	Head, hand, body pose and related frames
Sensor	Camera (video, raw video, depth image, intrinsic, extrinsic), PointCloud, Audio
Semantic Annotation	/annotation/segments — task description, segmented actions, skill names, and time bounds

For Devkit installation, CLI/API usage (e.g. lw-egosuite convert, lw-egosuite export-video), and visualization, see EgoSuite Devkit.