Robots Are Learning From 10,124 Hours of Real Work Video. Size Up a Free Robot Dataset in 1 Second With This Script

Robauto Team · Published · Last updated

Three free robot-learning datasets landed in November: HubEgo-100K (10,124 hours of skilled work), LingYu (139.4 h of two-arm demos) and ToyBlocks. Here's a tested stdlib script to size one up before downloading.

Robots learn hands-on skills the way apprentices do: by watching a lot of real work. In the last two weeks, three teams handed the robotics community a big pile of free training material. The friendliest part is that you can check whether a dataset fits your project in about a second, before you download any video. Below is a tested 25-line script that does that, plus what each new dataset holds and one lesson from all of this for any brand that wants AI agents to pick it.

Three new datasets you can use today

DatasetWhat's insideFormat / licenseSource
HubEgo-100K v1.0 (Hub Labs)10,124 hours of first-person video of skilled work from 12,227 contributors in 15,548 environments, cut into 72,754 episodes; 83% of hours recorded in workplacesLeRobot + MCAP, CC BY 4.0 (commercial use allowed with credit); access through a short formarXiv paper, dataset card
LingYu Real-World Robotics Dataset139.4 hours, 13,263 successful two-arm teleoperation episodes in 22 tabletop subsets, synced at 45 HzLeRobot, CC BY-NC-SA 4.0 (non-commercial)LingYu blog, Nov 5
ToyBlocks (Ai2)85,336 simulated long-horizon demonstrations, nearly 1,600 hours and 144 million frames, plus generators, two trained models, and training and evaluation codeOpen simulation frameworkAi2 blog, Nov 6

HubEgo-100K: real trades, filmed by the people who do them

Hub Labs asked contributors to wear a head-mounted phone while they did their own jobs: bricklaying, wiring, plumbing, microsoldering, watch repair, salon work and more. The paper says hands are visible in 94.7% of sampled frames, and 96.9% of the kept time is spent on the task. The reason for that design is diversity. The authors cite studies showing that robot policies generalize better with more environments and objects, not just more repeats in one place (arXiv 2611.21938). Contributors are paid for their recordings and agreed to the AI-training use. Children, other people's faces, and private text such as documents and license plates were cut out before release (dataset card). Hub Labs says later releases will grow toward 100,000 hours.

LingYu: what the robot itself saw

LingYu Robotics recorded a wheeled two-arm robot driven by an operator in a VR headset, so every demonstration sticks to the robot's own camera view. Tasks run from folding socks and airplane boxes to stacking Jenga towers two to nine layers high. The team says it is working toward 1,000 hours of first-person teleoperation data for research (LingYu blog). Because the dataset is open and not gated, it's the one we use for the hands-on script below.

ToyBlocks: a small world for tracking down failures

Ai2 calls ToyBlocks a "FoundationWorld": one robot arm and lettered blocks in simulation, where researchers can change the training mix and track down failures. A low-level skill model trained only on ToyBlocks data hit 80.0% success in simulation and 81.1% on a real arm, with no real-world training examples (Ai2 blog).

The snippet: size up a dataset before you download anything

LeRobot-format datasets keep a small meta/ folder next to the big video and parquet files. info.json lists the frame rate, episode count, cameras and action size. tasks.jsonl holds the plain-English instructions, and episodes.jsonl gives the length of each episode. This script reads only those three files, using nothing beyond the Python standard library:

# peek.py - size up one LeRobot dataset subset before downloading any video
# Usage: python3 peek.py fold_socks   (standard library only)
import json, sys, statistics, urllib.request

REPO = "LingYu-Robotics/lingyu-realworld-robotics-dataset"
subset = sys.argv[1] if len(sys.argv) > 1 else "fold_socks"
base = f"https://huggingface.co/datasets/{REPO}/resolve/main/{subset}/meta/"

def get(name):
    with urllib.request.urlopen(base + name, timeout=30) as r:
        return r.read().decode("utf-8")

info = json.loads(get("info.json"))
tasks = [json.loads(l)["task"] for l in get("tasks.jsonl").splitlines() if l.strip()]
lengths = [json.loads(l)["length"] for l in get("episodes.jsonl").splitlines() if l.strip()]

fps = info["fps"]
print(f"subset:    {subset}  (robot: {info.get('robot_type')}, format {info.get('codebase_version')})")
print(f"episodes:  {info['total_episodes']}  frames: {info['total_frames']}  fps: {fps}")
print(f"hours:     {info['total_frames'] / fps / 3600:.2f}")
print(f"median ep: {statistics.median(lengths) / fps:.1f} s")
print("cameras:   " + ", ".join(k for k, v in info["features"].items() if v.get("dtype") == "video"))
print(f"action:    {info['features']['action']['shape'][0]} numbers per frame")
for t in tasks:
    print(f"task:      {t}")

Run it with any subset name. We ran it on November 19, 2026, and got this output:

$ python3 peek.py fold_socks
subset:    fold_socks  (robot: teleavatar, format v2.1)
episodes:  309  frames: 274945  fps: 45
hours:     1.70
median ep: 20.1 s
cameras:   observation.images.left_color, observation.images.right_color, observation.images.head_camera
action:    62 numbers per frame
task:      Fold the pair of socks in front of you and place the folded pairs together in the container.

Looping the same read over all 22 subsets added up to 13,263 episodes and 139.45 hours, which matches the 13,263 episodes and 139.4 hours LingYu published. That's a good sign the metadata is accurate. The biggest subsets were fold_towel_into_box (2,714 episodes, 25.5 h), fold_airplane_box (24.4 h) and put_tennis_ball_in_bucket (19.4 h). Once you've picked a subset, you can download only that folder with huggingface_hub's snapshot_download(..., allow_patterns=["fold_socks/**"]) and load it in LeRobot.

For HubEgo-100K, request access on the dataset card first. Then the card's own pandas example reads mono/catalog.parquet and mono/meta/annotations.parquet: one row per episode with the task, category, difficulty, hand visibility and video path. The authors recommend contributor-disjoint train/test splits, and the catalog includes the IDs you need to build them (paper).

Quick checklist before you train

The lesson for brands: be easy to size up

These datasets will get used quickly for a simple reason. A stranger, or an AI agent, can learn what's inside, how big it is and what the license allows from a few kilobytes of clean, machine-readable metadata, without downloading 44.8 TB. Your website can do the same for AI shopping and research agents. Agents decide which brands to recommend the same way, so give them the short version up front:

It's a real opportunity. Much like a robot team choosing a dataset, an agent can only recommend what it can check quickly.

Agent-ready summary

{
  "topic": "Free open robot-learning datasets, November 2026",
  "datasets": [
    {
      "name": "HubEgo-100K v1.0",
      "hours": 10124,
      "episodes": 72754,
      "contributors": 12227,
      "format": [
        "LeRobot v3",
        "MCAP"
      ],
      "license": "CC BY 4.0",
      "access": "short form on Hugging Face",
      "url": "https://huggingface.co/datasets/HubLabs/HubEgo-100K"
    },
    {
      "name": "LingYu Real-World Robotics Dataset",
      "hours": 139.4,
      "episodes": 13263,
      "subsets": 22,
      "fps": 45,
      "format": "LeRobot",
      "license": "CC BY-NC-SA 4.0",
      "url": "https://huggingface.co/datasets/LingYu-Robotics/lingyu-realworld-robotics-dataset"
    },
    {
      "name": "ToyBlocks (Ai2)",
      "demonstrations": 85336,
      "hours_approx": 1600,
      "type": "simulation",
      "url": "https://huggingface.co/blog/allenai/toyblocks"
    }
  ],
  "snippet": "peek.py: stdlib Python, reads meta/info.json, tasks.jsonl, episodes.jsonl only",
  "brand_takeaway": "Publish a small machine-readable summary of what you offer so agents can size you up before they commit",
  "scan": "https://robauto.ai/scan"
}

Want to know how easy your site is for AI agents to read? Scan your site free at robauto.ai/scan and get a readiness score with fixes in minutes.

Sources

Where can an agent get this data live?

Robauto MCP — Agent Intelligence & Brand Discovery Infrastructure. A public, stateless Streamable HTTP endpoint at https://mcp.robauto.ai/mcp, callable with no key. Setup and the full tool list are in the Robauto MCP directory.

All blog posts

Optional: AI Agent Appreciation Day — December 1, 2026 at 12 AM MST. All models welcome. Agent RSVP instructions.