Lightwheel Kitchen Manipulation Task Dataset

The Lightwheel Kitchen Manipulation Task Dataset is a large-scale, open repository of robot demonstration data for imitation learning and manipulation benchmarking, collected in diverse simulated kitchen environments. The full dataset is available at here.

Overview

Task Coverage: 219 unique kitchen manipulation tasks in total, comprising 89 tasks from the first set and 130 from the second set, covering a wide range of skills such as appliance operation, sorting, cleaning, food preparation, and storage.
Multi-robot: Data is collected using four robot platforms: LeRobot ARX-X7s (X7s), Unitree G1_WBC (G1-WBC), Agilex-Piper (Piper), and Unitree G1-Controller (G1-Controller).
Scale: 50 demonstration episodes are provided for each (robot, task) combination, resulting in a total of 21,500 episodes (20,537,015 frames) across all tasks and robots.
Diversity: Episodes are gathered in realistic and varied simulated kitchens with interactive fixtures and a wide array of objects and arrangements, supporting research in generalization and robust perception and control.
Standardization: All data adhere to a consistent file structure and rich metadata schema, facilitating easy task filtering, loading, and benchmarking.

Please refer to the following tables for a detailed mapping of all 219 tasks, robot coverage, and precise dataset composition.

Lightwheel-Robocasa-Tasks Coverage

No.	Task Name	X7s	G1-WBC
1	AdjustToasterOvenTemperature	✓
2	AfterwashSorting	✓
3	ArrangeTea	✓
4	ArrangeVegetables	✓
5	ArrangeVegetablesSimple	✓
6	BeverageOrganization	✓	✓
7	BowlAndCup	✓
8	BreadAndCheese	✓	✓
9	BreadSelection	✓
10	BreadSetupSlicing	✓	✓
11	CheesyBread	✓	✓
12	ClearClutter	✓
13	ClearingCleaningReceptacles	✓
14	ClearingTheCuttingBoard	✓
15	CloseCabinet	✓
16	CloseDishwasher	✓	✓
17	CloseDrawer	✓	✓
18	CloseElectricKettleLid	✓	✓
19	CloseFridge	✓
20	CloseMicrowave	✓
21	CloseOven	✓	✓
22	CloseStandMixerHead	✓
23	CloseToasterOvenDoor	✓	✓
24	CoffeeServeMug	✓	✓
25	CoffeeSetupMug	✓	✓
26	ColorfulSalsa	✓	✓
27	CupcakeCleanup	✓
28	DefrostByCategory	✓
29	DessertAssembly	✓	✓
30	DessertUpgrade	✓	✓
31	DrawerUtensilSort	✓
32	DryDishes	✓
33	DryDrinkware	✓
34	FoodCleanup	✓
35	KettleBoiling	✓
36	MealPrepStaging	✓
37	NavigateKitchen	✓	✓
38	OpenDishwasher	✓
39	OpenDrawer	✓
40	OpenElectricKettleLid	✓	✓
41	OpenFridge	✓
42	OpenMicrowave	✓
43	OpenOven	✓
44	OpenStandMixerHead	✓	✓
45	OpenToasterOvenDoor	✓	✓
46	OrganizeBakingIngredients	✓
47	OrganizeVegetables	✓	✓
48	PanTransfer	✓
49	PastryDisplay	✓	✓
50	PnPCounterToCabinet	✓
51	PnPCounterToMicrowave	✓
52	PnPCounterToOven	✓	✓
53	PnPCounterToSink	✓
54	PnPCounterToStandMixer	✓	✓
55	PnPCounterToStove	✓	✓
56	PnPCounterToToasterOven	✓	✓
57	PnPOvenToCounter	✓	✓
58	PnPSinkToCounter	✓
59	PnPStoveToCounter	✓	✓
60	PnPToasterOvenToCounter	✓	✓
61	PnPToasterToCounter	✓
62	PreSoakPan	✓
63	PreheatOven	✓
64	PrepMarinatingMeat	✓
65	PrepareCoffee	✓
66	PushUtensilsToSink	✓
67	QuickThaw	✓
68	RestockPantry	✓
69	ShakerShuffle	✓
70	SizeSorting	✓	✓
71	SlideOvenRack	✓
72	SlideToasterOvenRack	✓
73	SnackSorting	✓
74	StackBowlsInSink	✓
75	StartCoffeeMachine	✓	✓
76	StockingBreakfastFoods	✓
77	ThawInSink	✓
78	TurnOffMicrowave	✓
79	TurnOffSinkFaucet	✓
80	TurnOffStove	✓
81	TurnOnElectricKettle	✓	✓
82	TurnOnMicrowave	✓
83	TurnOnSinkFaucet	✓
84	TurnOnStove	✓
85	TurnOnToaster	✓	✓
86	TurnOnToasterOven	✓
87	TurnSinkSpout	✓	✓
88	VeggieDipPrep	✓
89	WarmCroissant	✓

Lightwheel-LIBERO-Tasks Coverage

No.	Task Name	Piper	X7S	G1-Controller
1	L10K3TurnOnTheStoveAndPutTheMokaPotOnIt	✓	✓
2	L10K4PutTheBlackBowlInTheBottomDrawerOfTheCabinetAndCloseIt	✓	✓
3	L10K6PutTheYellowAndWhiteMugInTheMicrowaveAndCloseIt	✓
4	L10K8PutBothMokaPotsOnTheStove	✓	✓
5	L10L1PutBothTheAlphabetSoupAndTheCreamCheeseBoxInTheBasket	✓	✓	✓
6	L10L2PutBothTheAlphabetSoupAndTheTomatoSauceInTheBasket	✓	✓	✓
7	L10L2PutBothTheCreamCheeseBoxAndTheButterInTheBasket	✓	✓
8	L10L5PutWhiteMugOnLeftPlateAndPutYellowAndWhiteMugOnRightPlate	✓	✓
9	L10L6PutWhiteMugOnPlateAndPutChocolatePuddingToRightPlate	✓	✓
10	L10S1PickUpTheBookAndPlaceItInTheBackCompartmentOfTheCaddy	✓	✓	✓
11	L90K10CloseTheTopDrawerOfTheCabinet	✓	✓	✓
12	L90K10CloseTheTopDrawerOfTheCabinetAndPutTheBlackBowlOnTopOfIt	✓	✓	✓
13	L90K10PutTheBlackBowlInTheTopDrawerOfTheCabinet	✓	✓
14	L90K10PutTheButterAtTheBackInTheTopDrawerOfTheCabinetAndCloseIt	✓	✓
15	L90K10PutTheButterAtTheFrontInTheTopDrawerOfTheCabinetAndCloseIt	✓	✓
16	L90K10PutTheChocolatePuddingInTheTopDrawerOfTheCabinetAndCloseIt	✓	✓
17	L90K1OpenTheBottomDrawerOfTheCabinet	✓	✓
18	L90K1OpenTheTopDrawerOfTheCabinet	✓	✓
19	L90K1OpenTheTopDrawerOfTheCabinetAndPutTheBowlInIt	✓	✓
20	L90K1PutTheBlackBowlOnThePlate	✓	✓	✓
21	L90K1PutTheBlackBowlOnTopOfTheCabinet	✓	✓	✓
22	L90K2OpenTheTopDrawerOfTheCabinet	✓	✓
23	L90K2PutTheBlackBowlAtTheBackOnThePlate	✓	✓	✓
24	L90K2PutTheBlackBowlAtTheFrontOnThePlate	✓	✓
25	L90K2PutTheMiddleBlackBowlOnThePlate	✓	✓	✓
26	L90K2PutTheMiddleBlackBowlOnTopOfTheCabinet	✓	✓	✓
27	L90K2StackTheBlackBowlAtTheFrontOnTheBlackBowlInTheMiddle	✓	✓	✓
28	L90K2StackTheMiddleBlackBowlOnTheBackBlackBowl	✓	✓
29	L90K3PutTheFryingPanOnTheStove	✓	✓
30	L90K3PutTheMokaPotOnTheStove	✓	✓
31	L90K3TurnOnTheStove	✓	✓	✓
32	L90K3TurnOnTheStoveAndPutTheFryingPanOnIt	✓	✓
33	L90K4CloseTheBottomDrawerOfTheCabinet	✓	✓	✓
34	L90K4CloseTheBottomDrawerOfTheCabinetAndOpenTheTopDrawer	✓	✓
35	L90K4PutTheBlackBowlInTheBottomDrawerOfTheCabinet	✓	✓
36	L90K4PutTheBlackBowlOnTopOfTheCabinet	✓	✓	✓
37	L90K4PutTheWineBottleInTheBottomDrawerOfTheCabinet	✓	✓
38	L90K4PutTheWineBottleOnTheWineRack	✓	✓	✓
39	L90K5CloseTheTopDrawerOfTheCabinet	✓	✓	✓
40	L90K5PutTheBlackBowlInTheTopDrawerOfTheCabinet	✓	✓
41	L90K5PutTheBlackBowlOnThePlate	✓	✓	✓
42	L90K5PutTheBlackBowlOnTopOfTheCabinet	✓	✓	✓
43	L90K5PutTheKetchupInTheTopDrawerOfTheCabinet	✓	✓	✓
44	L90K6CloseTheMicrowave	✓	✓	✓
45	L90K6PutTheYellowAndWhiteMugToTheFrontOfTheWhiteMug	✓	✓
46	L90K7OpenTheMicrowave	✓	✓
47	L90K7PutTheWhiteBowlOnThePlate	✓	✓	✓
48	L90K7PutTheWhiteBowlToTheRightOfThePlate	✓	✓
49	L90K8PutTheRightMokaPotOnTheStove	✓	✓
50	L90K8TurnOffTheStove	✓	✓	✓
51	L90K9PutTheFryingPanOnTheCabinetShelf	✓	✓
52	L90K9PutTheFryingPanOnTopOfTheCabinet	✓	✓
53	L90K9PutTheFryingPanUnderTheCabinetShelf	✓	✓
54	L90K9PutTheWhiteBowlOnTopOfTheCabinet	✓	✓	✓
55	L90K9TurnOnTheStove	✓	✓	✓
56	L90K9TurnOnTheStoveAndPutTheFryingPanOnIt	✓	✓
57	L90L1PickUpTheAlphabetSoupAndPutItInTheBasket	✓	✓	✓
58	L90L1PickUpTheCreamCheeseBoxAndPutItInTheBasket	✓	✓
59	L90L1PickUpTheKetchupAndPutItInTheBasket	✓	✓	✓
60	L90L1PickUpTheTomatoSauceAndPutItInTheBasket	✓	✓	✓
61	L90L2PickUpTheAlphabetSoupAndPutItInTheBasket	✓	✓	✓
62	L90L2PickUpTheButterAndPutItInTheBasket	✓	✓	✓
63	L90L2PickUpTheMilkAndPutItInTheBasket	✓	✓	✓
64	L90L2PickUpTheOrangeJuiceAndPutItInTheBasket	✓	✓	✓
65	L90L2PickUpTheTomatoSauceAndPutItInTheBasket	✓	✓	✓
66	L90L3PickUpTheAlphabetSoupAndPutItInTheTray	✓	✓	✓
67	L90L3PickUpTheButterAndPutItInTheTray	✓	✓	✓
68	L90L3PickUpTheCreamCheeseAndPutItInTheTray	✓	✓
69	L90L3PickUpTheKetchupAndPutItInTheTray	✓	✓
70	L90L3PickUpTheTomatoSauceAndPutItInTheTray	✓	✓	✓
71	L90L4PickUpTheBlackBowlOnTheLeftAndPutItInTheTray	✓	✓	✓
72	L90L4PickUpTheChocolatePuddingAndPutItInTheTray	✓	✓	✓
73	L90L4PickUpTheSaladDressingAndPutItInTheTray	✓	✓	✓
74	L90L4StackTheLeftBowlOnTheRightBowlAndPlaceThemInTheTray	✓	✓
75	L90L4StackTheRightBowlOnTheLeftBowlAndPlaceThemInTheTray	✓	✓
76	L90L5PutTheRedMugOnTheLeftPlate	✓	✓	✓
77	L90L5PutTheRedMugOnTheRightPlate	✓	✓	✓
78	L90L5PutTheWhiteMugOnTheLeftPlate	✓	✓	✓
79	L90L5PutTheYellowAndWhiteMugOnTheRightPlate	✓	✓	✓
80	L90L6PutTheChocolatePuddingToTheLeftOfThePlate	✓	✓
81	L90L6PutTheChocolatePuddingToTheRightOfThePlate	✓	✓
82	L90L6PutTheRedMugOnThePlate	✓	✓	✓
83	L90L6PutTheWhiteMugOnThePlate	✓	✓	✓
84	L90S1PickUpTheBookAndPlaceItInTheFrontCompartmentOfTheCaddy	✓	✓	✓
85	L90S1PickUpTheBookAndPlaceItInTheLeftCompartmentOfTheCaddy	✓	✓	✓
86	L90S1PickUpTheBookAndPlaceItInTheRightCompartmentOfTheCaddy	✓	✓	✓
87	L90S1PickUpTheYellowAndWhiteMugAndPlaceItToTheRightOfTheCaddy	✓	✓
88	L90S2PickUpTheBookAndPlaceItInTheBackCompartmentOfTheCaddy	✓	✓	✓
89	L90S2PickUpTheBookAndPlaceItInTheFrontCompartmentOfTheCaddy	✓	✓
90	L90S2PickUpTheBookAndPlaceItInTheLeftCompartmentOfTheCaddy	✓	✓	✓
91	L90S2PickUpTheBookAndPlaceItInTheRightCompartmentOfTheCaddy	✓	✓	✓
92	L90S3PickUpTheBookAndPlaceItInTheFrontCompartmentOfTheCaddy	✓	✓	✓
93	L90S3PickUpTheBookAndPlaceItInTheLeftCompartmentOfTheCaddy	✓	✓	✓
94	L90S3PickUpTheBookAndPlaceItInTheRightCompartmentOfTheCaddy	✓	✓	✓
95	L90S3PickUpTheRedMugAndPlaceItToTheRightOfTheCaddy	✓	✓	✓
96	L90S3PickUpTheWhiteMugAndPlaceItToTheRightOfTheCaddy	✓	✓
97	L90S4PickUpTheBookInTheMiddleAndPlaceItOnTheCabinetShelf	✓	✓
98	L90S4PickUpTheBookOnTheLeftAndPlaceItOnTopOfTheShelf	✓	✓
99	L90S4PickUpTheBookOnTheRightAndPlaceItOnTheCabinetShelf	✓	✓
100	L90S4PickUpTheBookOnTheRightAndPlaceItUnderTheCabinetShelf	✓	✓
101	LGOpenTheMiddleDrawerOfTheCabinet	✓	✓
102	LGOpenTheTopDrawerAndPutTheBowlInside	✓	✓
103	LGPushThePlateToTheFrontOfTheStove	✓
104	LGPutTheBowlOnThePlate	✓	✓	✓
105	LGPutTheBowlOnTheStove	✓	✓
106	LGPutTheBowlOnTopOfTheCabinet	✓	✓	✓
107	LGPutTheCreamCheeseInTheBowl	✓	✓	✓
108	LGPutTheWineBottleOnTheRack	✓	✓	✓
109	LGPutTheWineBottleOnTopOfTheCabinet	✓	✓	✓
110	LGTurnOnTheStove	✓	✓	✓
111	LOPickUpTheAlphabetSoupAndPlaceItInTheBasket	✓
112	LOPickUpTheBbqSauceAndPlaceItInTheBasket	✓
113	LOPickUpTheButterAndPlaceItInTheBasket	✓
114	LOPickUpTheChocolatePuddingAndPlaceItInTheBasket	✓
115	LOPickUpTheKetchupAndPlaceItInTheBasket	✓
116	LOPickUpTheMilkAndPlaceItInTheBasket	✓
117	LOPickUpTheOrangeJuiceAndPlaceItInTheBasket	✓
118	LOPickUpTheSaladDressingAndPlaceItInTheBasket	✓
119	LOPickUpTheTomatoSauceAndPlaceItInTheBasket	✓
120	LOPutCreamCheeseInBasket	✓
121	LSPickUpBlackBowlBetweenPlateAndRamekinAndPlaceItOnPlate	✓	✓
122	LSPickUpBlackBowlInTopDrawerOfWoodenCabinetAndPlaceItOnPlate	✓
123	LSPickUpTheBlackBowlFromTableCenterAndPlaceItOnThePlate	✓	✓	✓
124	LSPickUpTheBlackBowlNextToTheCookieBoxAndPlaceItOnThePlate	✓	✓
125	LSPickUpTheBlackBowlNextToThePlateAndPlaceItOnThePlate	✓	✓	✓
126	LSPickUpTheBlackBowlNextToTheRamekinAndPlaceItOnThePlate	✓	✓
127	LSPickUpTheBlackBowlOnTheCookieBoxAndPlaceItOnThePlate	✓	✓
128	LSPickUpTheBlackBowlOnTheRamekinAndPlaceItOnThePlate	✓	✓
129	LSPickUpTheBlackBowlOnTheStoveAndPlaceItOnThePlate	✓	✓
130	LSPickUpTheBlackBowlOnTheWoodenCabinetAndPlaceItOnThePlate	✓	✓

Data Structure & Access

Dataset structure example (from HuggingFace LightwheelAI/lightwheel_tasks):

lightwheel_tasks/
├── lightwheel_robocasa_tasks_g1_wbc/
│   └── BeverageOrganization/
│       ├── BeverageOrganization_1763692193809507/
│       │   ├── isaac_replay_state_product.mp4   # Robot demonstration video
│       │   ├── running_args.json                # Collection parameters and environment metadata
│       │   └── trajectories.hdf5                # Trajectory data (multi-modal: state, actions, joint targets etc.)
│       └── ...
└── lightwheel_libero_tasks_piper/
    └── L90K3TurnOnTheStove/
        ├── L90K3TurnOnTheStove_1758698193690141/
        │   ├── isaac_replay_state.mp4
        │   ├── running_args.json
        │   └── trajectories.hdf5
        └── ...

Each (task/robot/episode) subfolder contains one robot demonstration video (mp4), a JSON file of parameters and metadata, and an HDF5 trajectory with full sequence observations.
File structure and naming are highly standardized for easy task filtering and programmatic loading.

Loading Example:

from datasets import load_dataset

ds = load_dataset("LightwheelAI/lightwheel_tasks")
print(ds)

Applications

Imitation learning
Reinforcement learning
Generalization, transfer, and multi-robot evaluation
High-fidelity kitchen manipulation research

Citation

If you use this dataset, please cite and reference Lightwheel Kitchen Manipulation Task Dataset.

Overview​

Lightwheel-Robocasa-Tasks Coverage​

Lightwheel-LIBERO-Tasks Coverage​

Data Structure & Access​

Applications​

Citation​

See Also​