Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor compute_stats #521

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

aliberts
Copy link
Collaborator

@aliberts aliberts commented Nov 25, 2024

What this does

TODO:

  • Add episodes_stats.jsonl (one stats per episode). See episodes.jsonl
  • Add aggregation function: episodes_stats are aggregated into stats and stored in the stats.json
  • Add backward compatibilityL if episodes_stats.jsonl doesnt exist, episodes_stats is created by duplicating stats for each episode
  • Accelerate compute stats by sampling images
  • Add type and shape checks in add_frame

How it was tested

Backward compatibility:

from lerobot.common.datasets.lerobot_dataset import LeRobotDatasetMetadata
ds_meta = LeRobotDatasetMetadata("lerobot/pusht")
Fetching 4 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 2061.84it/s]
/Users/rcadene/code/lerobot/lerobot/common/datasets/lerobot_dataset.py:100: UserWarning: 'episodes_stats.jsonl' not found. Use global dataset stats for each episode instead.

How to checkout & try? (for the reviewer)

TODO

@aliberts aliberts mentioned this pull request Nov 25, 2024
@Cadene Cadene force-pushed the user/aliberts/2024_11_25_compute_stats_v2 branch from b004a0e to 23a16f0 Compare December 28, 2024 13:49
@Cadene Cadene changed the base branch from main to user/aliberts/2024_11_30_remove_hydra January 26, 2025 18:49
Base automatically changed from user/aliberts/2024_11_30_remove_hydra to main January 31, 2025 12:57
Cadene and others added 2 commits February 9, 2025 12:16
[Fix] Move back to manual calibration (#488)

feat: enable to use multiple rgb encoders per camera in diffusion policy (#484)

Co-authored-by: Alexander Soare <alexander.soare159@gmail.com>

Fix config file (#495)

fix: broken images and a few minor typos in README (#499)

Signed-off-by: ivelin <ivelin117@gmail.com>

Add support for Windows (#494)

bug causes error uploading to huggingface, unicode issue on windows. (#450)

Add distinction between two unallowed cases in name check "eval_" (#489)

WIP

Fix autocalib moss (#486)

[Fix] Move back to manual calibration (#488)

feat: enable to use multiple rgb encoders per camera in diffusion policy (#484)

Co-authored-by: Alexander Soare <alexander.soare159@gmail.com>

Fix config file (#495)

fix: broken images and a few minor typos in README (#499)

Signed-off-by: ivelin <ivelin117@gmail.com>

Add support for Windows (#494)

bug causes error uploading to huggingface, unicode issue on windows. (#450)

Add distinction between two unallowed cases in name check "eval_" (#489)

Rename deprecated argument (temporal_ensemble_momentum) (#490)

Dataset v2.0 (#461)

Co-authored-by: Remi <remi.cadene@huggingface.co>

Refactor OpenX (#505)

Fix missing local_files_only in record/replay (#540)

Co-authored-by: Simon Alibert <alibert.sim@gmail.com>

Control simulated robot with real leader (#514)

Co-authored-by: Remi <remi.cadene@huggingface.co>

Update 7_get_started_with_real_robot.md (#559)

LerobotDataset pushable to HF from any folder (#563)

Fix example 6 (#572)

fixing typo from 'teloperation' to 'teleoperation' (#566)

[vizualizer] for LeRobodDataset V2 (#576)

Fix broken `create_lerobot_dataset_card`  (#590)

Update README.md (#612)

Add draccus, create MainConfig

WIP refactor train.py and ACT

Add policies training presets

Update diffusion policy

Add pusht and xarm env configs

Update tdmpc

Update vqbet

Fix poetry relax

Add feature types to envs

Add EvalPipelineConfig, parse features from envs

Add custom parser

Update pretrained loading mechanisms

Add dependency fixes & lock update

Fix pretrained_path

Refactor envs, remove RealEnv

Fix typo

Enable end-to-end tests

Fix Makefile

Log eval config

Fix end-to-end tests

Fix Quality workflow (#622)

Remove amp & add resume test

Speed-up tests

Fix poetry relax

Remove config yaml for robot devices (#594)

Co-authored-by: Simon Alibert <simon.alibert@huggingface.co>

fix(docs): typos in benchmark readme.md (#614)

Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>

fix(visualise): use correct language description for each episode id (#604)

Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>

typo fix: batch_convert_dataset_v1_to_v2.py (#615)

Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>

[viz] Fixes & updates to html visualizer (#617)

Fix logger

Remove hydra-core

Add aggregate_stats

Add estimate_num_samples for images, Add test image

Remove NoneSchedulerConfig

Add push_pretrained

Remove eval.episode_length

Fix wandb_video

Fix typo

Add features back into policy configs (#643)

fixes to SO-100 readme (#600)

Co-authored-by: Philip Fung <no@one>
Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>

Fix for the issue #638 (#639)

Fix env_to_policy_features call

Fix wandb init

remove omegaconf

Add branch arg

Move deprecated

Move training config

Remove pathable_args

Implement custom HubMixin

Fixes

Implement PreTrainedPolicy base class

Add HubMixin to TrainPipelineConfig

Udpate example 2 & 3

Update push_pretrained

Bump`rerun-sdk` dependency to `0.21.0` (#618)

Co-authored-by: Simon Alibert <75076266+aliberts@users.noreply.github.com>

Fix config_class

Fix from_pretrained kwargs

Remove policy_protocol

Camelize PretrainedConfig

Additional fix while retraining policies (#629)

Co-authored-by: Simon Alibert <simon.alibert@huggingface.co>

Actually reactivate tdmpc online test

Update example 4

Remove advanced example 1

Remove example 5

Move example 6 to advanced

Use HubMixin.save_pretrained

Enable config_path to be a repo_id

Dry has_method

Update example 4

Update README

Cleanup pyproject.toml

Update eval docstring

Update README

Clean example 4

Update README

Make 'last' checkpoint symlink relative

Fix cluster image (#653)

Simplify example 4

fix stats per episodes and aggregate stats and casting to tensor
@aliberts aliberts force-pushed the user/aliberts/2024_11_25_compute_stats_v2 branch from 8feeede to 0c55461 Compare February 9, 2025 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants