Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add stochastic taxi (rainy+fickle) #1315

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

foreverska
Copy link
Contributor

Description

Adds rainy transition probabilities and fickle passenger to align environment with paper.

Fixes #161

Type of change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Checklist:

  • I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Copy link
Member

@pseudo-rnd-thoughts pseudo-rnd-thoughts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @foreverska, thanks for the PR.

Generally the PR looks good.
Could you change the np.random to self.np_random and revert the environment version increment (new features that don't affect default behaviour shouldn't require version bumps)

and a couple of questions before approving and merging

  1. Is this backward compatible with default parameters? If it does currently, could we make it backward compatible.
  2. Is the probability of the behaviour reported in step and reset correctly?

@foreverska
Copy link
Contributor Author

@pseudo-rnd-thoughts

Addressed Comments.

Is this backward compatible with default parameters

Yes, default values for rainy and fickle are both False. This aligns functionality with pre-commit default behavior.

Is the probability of the behaviour reported in step and reset correctly?

It matches the other ToyText environments (Cliff/Frozen) with step returning the probability of the taken transition and reset always returning 1. I added lines in the unit test to guard against regression.

Copy link
Member

@pseudo-rnd-thoughts pseudo-rnd-thoughts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the tests and documentation updates, then should be good to merge.
Thanks for making the changes

Copy link
Collaborator

@Kallinteris-Andreas Kallinteris-Andreas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me,

@foreverska
Copy link
Contributor Author

@pseudo-rnd-thoughts I think this needs one more review since there was a commit after your last one. Thanks.

@pseudo-rnd-thoughts
Copy link
Member

Hey @foreverska, sorry I'm on holiday currently. Looking over the PR again, I'm a tad worried about the is_rainy change.
Is there a way of making the code only run if is_rainy is true?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Proposal] Add transitional probabilities to Taxi and Cliff Walking toy text environments
3 participants