Python- Re-implementation of command-line parsing

[Lestropie]

This keeps getting raised as an issue, e.g. #1392, #1607.

Ideally a custom command-line parsing implementation would use a comparable syntax to the C++ binaries, with the same capabilities (and possibly also remove some capabilities of argparse that are technically ambiguous according to the MRtrix3 command-line option ordering philosophy), with corresponding re-implementation of the help page / documentation generation, and would also require some kind of implementation that would satisfy what is currently achieved using Subparsers within argparse and the algorithm module.

[maxpietsch]

while we're at it:

• an additional layer of grouping (optional) arguments would be great. Useful for multi-contrast registration where setting parameters for individual registration stages and contrasts is required (proposed + separator: 250ee47)
• automatic inclusion of default values and valid choices to the help description

[thijsdhollander]

While I fully support the idea to look at these aspects, may I carefully suggest we keep it on a wish list type of level for now? It'd be neat if we can close off an RC4 before ISMRM; given what's already ready to "ship" for some time (and the amount of it), and how long it's been since RC3. Lots of users will be looking forward to RC4, and we'll be facing many of them at ISMRM.

Disclaimer: no hidden intentions above. Could've made this into an issue by itself, so there's no reason why I added it specifically to this thread, other than this thread having the wishlist label assigned to it.

[Lestropie]

• Familiarise with MRtrix3 command-line interface capabilities:

  • MRtrix3 NeuroImage manuscript
  • Online documentation
  • C++ Code (multiple relevant files within core/)
    (Note that the C++ code does not have the capability for subparsers as used in the algorithm module; see multi-level command line parsing #1840; this is a functionality that would need to be preserved in the Python re-implementation)
  • Python code
  • Experimenting with running various commands

• Generate template module / parsing class

  • Note that ideally some care would be taken to split the generic command-line parsing capabilities (that may be applicable to many softwares) to the augmentation of such that is MRtrix3-specific; the former could potentially be distributed more widely

• Primary goals for changes compared to argparse:

  • Integrate current MRtrix3 augmentation of such:

    • Hidden command-line options for generating self-documentation in various formats
    • Developer can inherit from class and add various defaults to apply across multiple commands:
      • Default copyright
      • Default author
      • Set of command-line options that are common to multiple commands
    • Ability to add scientific manuscript citations
    • More informative feedback upon parsing errors
    • Flagging mutually exclusive options
    • More nicely formatted help page with pagination

  • Options have arguments attributed to them
    Rather than indicating a number of entries expected to follow a command-line options, instead specify some number of positional arguments to follow that option, each with their own type & help information

  • Better argument typing

    • Longer list of built-in types
      Currently: int, float, string, file
      Add: bool, input directory, output directory, integer sequence, floating-point sequence
    • Developer can provide custom types
      MRtrix3 types desired: input image, output image, input streamlines, output streamlines

---

Note that in retrospect, while writing this I've realized it may be a better investment of effort to investigate a potential tailored solution to #1392 rather than attempting to compete with argparse. It would mean we wouldn't get the "Options have arguments attributed to them" change above, but we could live with that.