Overview
EgoSuite episodes are stored as .mcap files. Each file contains multiple time-synchronized topics: video streams, poses, annotations, session metadata, and optionally audio and point cloud. The exact set of topics depends on the capture rig (e.g. head vs. wrist cameras).
All EgoSuite-native topics use Protobuf encoding and share a common Header (common.header.Header): count, ts (Unix time in nanoseconds), sender. Message types and proto definitions live in the lw-egosuite-msg package (protos/ and generated code).
Topic convention
- Some topics use third-party schemas (e.g. Foxglove
foxglove.*,tf2_msgs.*) or are special-cased. - Most EgoSuite messages carry a header field of type
common.header.Headerwith:count(uint64)ts(int64, Unix time in nanoseconds)sender(string)
Third-party messages such asfoxglove.*andtf2_msgs.*do not use this header.
Registered topics
| Topic | Proto message |
|---|---|
/session/metadata | SessionMetadata |
/annotation/segments | AnnotationSegment |
/pose/body | BodyFrame |
/pose/head_pose | HeadPoseFrame |
/pose/headcam_pose | HeadcamPoseFrame |
/pose/left_hand | LeftHandFrame |
/pose/right_hand | RightHandFrame |
/pose/right_eye_cam | RightEyeCamFrame |
/sensor/camera/head_left/video | foxglove.CompressedVideo |
/sensor/camera/head_right/video | foxglove.CompressedVideo |
/sensor/camera/left_wrist/video | foxglove.CompressedVideo |
/sensor/camera/right_wrist/video | foxglove.CompressedVideo |
/sensor/camera/head_left/intrinsic | foxglove.CameraCalibration |
/sensor/camera/head_right/intrinsic | foxglove.CameraCalibration |
/sensor/camera/left_wrist/intrinsic | foxglove.CameraCalibration |
/sensor/camera/right_wrist/intrinsic | foxglove.CameraCalibration |
/sensor/camera/head_left/extrinsic | foxglove.FrameTransforms |
/sensor/camera/head_right/extrinsic | foxglove.FrameTransforms |
/sensor/camera/left_wrist/extrinsic | foxglove.FrameTransforms |
/sensor/camera/right_wrist/extrinsic | foxglove.FrameTransforms |
/pointcloud | foxglove.PointCloud |
/audio | foxglove.RawAudio |
Navigation
Use the sidebar to open detailed docs:
| Section | Topics / content |
|---|---|
| Metadata | /session/metadata — session/episode metadata (devices, global time, task info, operator) |
| Annotation | /annotation/segments — skill segments (task phases, labels, captions) |
| Pose | Head, hand, body pose and related frames |
| Sensor | Camera (video, intrinsic, extrinsic), PointCloud, Audio |
For Devkit installation, CLI/API usage (e.g. lwego validate, lwego convert, lwego export-video), and visualization, see EgoSuite Devkit.