-
Notifications
You must be signed in to change notification settings - Fork 74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a preview version of the de-novo pipeline #641
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @VJalili for getting this in and your patience with my response. A lot of detective work you must've done to get the docker working - well done!
I have some comments about the WDLs to clean them up a bit - there are quite a few places where we can re-use code.
Thank you, @mwalker174! |
This PR adds a preview version of the de-novo pipeline, initially implemented by @alsanju, @nicolecalamari, and Steph Hao. This PR keeps the original functionality intact and makes the minimally required changes listed below.
Dockerfile,
which was created by taking a brute-force approach trying to make an image as the experimental image used during the initial development phase that was created without aDockerfile.
set -exuo pipefail
to all the tasks.head
andtail
commands with theirawk
equivalent sincehead
/tail
failpipefail.
|| true
to thegrep
calls to address cases when it fails to find a pattern in its input.tabix_query
method because of issues such as “Popen” yielding by char instead of a line and errors such as “device is busy.”${sep=" " input}
to~{sep=" " input}
, replacing$
with~
).python3.x
(e.g., python3.6
and3.9
) calls withpython
, hence consistently using the version of python that comes with the Docker image (i.e.,python3.10
), and implement the minimally necessary changes in the python scripts of the pipeline."~{select_first ...} GB"
toselect_first ... + " GB"
.list
orfile
are all refactored).return (a)
toreturn a
).if
conditions andfor
loops (e.g., refactorif (a == 10)
toif a==10
).a == None
witha is None
).if a == True
toif a
).Improvements to be implemented in the follow-up PRs:
NA
; for instance, dataframe columns are set to"NA"
that may not function as intended when asserting fornp.nan
in that column.Testing
Successfully tested on the methods' team Cromwell server:
60d3c93f-7c14-420c-9b3b-381ee4d6e789