Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: enhance shed_tools to install from native data_manager files #93

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

WIP: enhance shed_tools to install from native data_manager files #93

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion ephemeris/shed_tools.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -328,6 +328,10 @@ def _parser():
"-y", "--yaml_tool",
dest="tool_yaml",
help="Install tool represented by yaml string",)
command_parser.add_argument(
"-d", "--yaml_data_manager",
dest="data_manager_yaml",
help="Install tools represented in a data manager yaml file",)
command_parser.add_argument(
"--name",
help="The name of the tool to install (only applicable "
Expand Down Expand Up @@ -579,6 +583,20 @@ def get_install_repository_manager(options):
'install_tool_dependencies', INSTALL_TOOL_DEPENDENCIES)
elif options.tool_yaml:
repositories = [yaml.safe_load(options.tool_yaml)]
elif options.data_manager_yaml:
repositories = list()
dms = yaml.load(open(options.data_manager_yaml))
for dm in dms['data_managers']:
tokens = dm['id'].split('/')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a data managers file you can also just give the tool ID so then this does not work, and a lot of errors will follow...
There should be some error handling here. Also tests should be added to include a parsable file and a non-parsable (but still valid) file.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can/Should we enforce long-tool-ids for reproducibility reasons?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The data manager job is saved in galaxy's history. So that is not necessary IMO.Also, long tool ids are quite inconvenient. Are there any people who have multiple fasta indexes of the same genome on their galaxy just because of the version of their indexer?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh its not about reproducibility in one Galaxy instance, but across instances... https://github.com/bgruening/idc

Are there any people who have multiple fasta indexes of the same genome on their galaxy just because of the version of their indexer?

Not sure, but they are different data managers where the version is crucial. And keep in mind that data managers can be way more than genome indices.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are valid reasons to support long tool IDs but not necessarily to enforce them.
Would it be possible to put in an if statement here that recognizes a long tool ID from a short tool ID and else throws a proper exception? For the long tool ID the following code looks good. For a short tool ID some code could be added that fetches the repo owner from toolshed, preferring iuc and devteam if there are multiple owners and selecting the first one in alphabetic order otherwise whilst throwing a warning that there were multiple owners.

owner = tokens[2]
tool_shed_url = tokens[0]
repo_name = tokens[3]
repositories.append({
"owner": owner,
"name": repo_name,
"tool_panel_section_label": 'Data Managers',
"tool_shed_url": tool_shed_url,
})
elif options.action == "update":
get_repository_list = GiToToolYaml(
gi=gi,
Expand Down Expand Up @@ -926,7 +944,7 @@ def main():
options = _parse_cli_options()
log = setup_global_logger(name=__name__, log_file=options.log_file)
install_tool_manager = None
if options.tool_list_file or options.tool_yaml or \
if options.tool_list_file or options.tool_yaml or options.data_manager_yaml or \
options.name and options.owner and (options.tool_panel_section_id or options.tool_panel_section_label):
if options.action == "update":
sys.exit("update command can not be used together with tools to be installed.")
Expand Down