add conda mambabuild subcommand to aid transitioning away from boa

[AaronOpfer]

Checklist

• I added a descriptive title
• I searched open requests and couldn't find a duplicate

What is the idea?

Add a subcommand to conda called mambabuild which is equivalent to performing conda build --solver=libmamba.

Why is this needed?

My company wants to upgrade to the latest stable versions of conda and conda-build. We also want to off-board boa as it appears to a concluded project (development has switched to rattler-build).

However, my company has developed a large number of pipelines that invoke mamba mambabuild to get the build performance to an acceptable level. This was well before conda itself added the --solver=libmamba option.

If we attempt to upgrade conda and conda-build to the latest versions and remove boa, then all of these pipelines will break. We lack the CI robots that made this easy to fix for conda-forge.

What should happen?

Performing:

conda mambabuild ...

should be equivalent to performing:

conda build --solver=libmamba ...

for a generous transition period.

Additional Context

We discussed this in-person at the Conda open source room at PyCon 2024 with @wolfv and @beeankha and others.

[jaimergp]

conda-build could export a new subcommand plugin for this, setting the right context variables. I don't know if this might result into conflicts with boa's mambabuild subcommand (who wins a conda mambabuild parsing battle?). Safest way would be to implement this in a separate package that simply provides the right kind of subcommand plugin and can be safely uninstalled if problematic.

[AaronOpfer]

Yes, I agree that a separate package would work well.

[jaimergp]

This would be all the Python code you need:

In let's say conda_mambabuild.py:

from conda import plugins
from conda.common.io import env_var

def mambabuild(*args):
    from conda_build.cli.main_build import execute

    with env_var("CONDA_SOLVER", "libmamba"):
        return execute(*args)

@plugins.hookimpl
def conda_subcommands():
    yield plugins.CondaSubcommand(
        name="mambabuild",
        summary="Run conda-build with forced solver=libmamba.",
        action=mambabuild,
    )

And then in your pyproject.toml:

[project.entry-points.conda]
conda-mambabuild = "conda_mambabuild"

[AaronOpfer]

That does indeed appear to work.

package:
  name: fakeboa
  version: 0.0.1

source:
  path: .

build:
  noarch: python
  number: 0
  script: python -m pip install --no-deps --ignore-installed .

requirements:
  host:
    - python
    - pip
  run:
    - python
    - conda
  run_constrained:
    - boa > 999.999.999

[project]
name = "fakeboa"
version = "0.0.1"

[project.entry-points.conda]
conda-mambabuild = "fakeboa"

[AaronOpfer]

Ah, actually, playing around with this package installed in the base envrionment alongside conda 24.1.2 and mamba 1.5.7, mamba mambabuild . says:

$ mamba mambabuild x
Currently, only install, create, list, search, run, info, clean, remove, update, repoquery, activate and deactivate are supported through mamba.

so it seems as though I am missing some additional fakery.

[wolfv]

you wrote mamba mambabuild (did that ever work?)

[AaronOpfer]

Yes, most of my company's CI scripts invoke mamba mambabuild recipe_dir/, which works on conda 23.5.2 mamba 1.4.9 boa 0.14.0.

[AaronOpfer]

I have gotten it to work, but I had to ditch conda plugins and instead create a script named conda-mambabuild in bin/. It looks like this is necessary, as it allows p.parse_args() in mamba's do_call to immediately pass execution to this binary through some mechanism that I do not understand, whereas the conda plugin instead returned a argparse.Namespace and then failed later on inside mamba.

fakeboa/__init__.py

from conda.common.io import env_var


def main():
    from conda_build.cli.main_build import main

    with env_var("CONDA_SOLVER", "libmamba"):
        main()

pyproject.toml

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[project]
name = "fakeboa"
version = "0.0.4"

[tool.setuptools]
packages = ["fakeboa"]

[project.scripts]
conda-mambabuild = "fakeboa:main"

recipe/meta.yaml

package:
  name: fakeboa
  version: 0.0.4

source:
  path: ..

build:
  noarch: python
  number: 0
  script: python -m pip install --no-deps --ignore-installed .

requirements:
  host:
    - setuptools
    - python
    - pip
  run:
    - python
    - conda
  run_constrained:
    - boa > 999.999.999

test:
  import:
    - fakeboa

[AaronOpfer]

Ah, with Conda upgraded to 24.3.0 and mamba 1.5.8, this fakeboa no longer works with mamba mambabuild.

   Traceback (most recent call last):
      File "/miniconda/lib/python3.10/site-packages/conda/exception_handler.py", line 17, in __call__
        return func(*args, **kwargs)
      File "/miniconda/lib/python3.10/site-packages/mamba/mamba.py", line 959, in exception_converter
        raise e
      File "/miniconda/lib/python3.10/site-packages/mamba/mamba.py", line 952, in exception_converter
        exit_code = _wrapped_main(*args, **kwargs)
      File "/miniconda/lib/python3.10/site-packages/mamba/mamba.py", line 898, in _wrapped_main
        result = do_call(parsed_args, p)
      File "/miniconda/lib/python3.10/site-packages/mamba/mamba.py", line 723, in do_call
        raise ValueError("Unrecognized 'args' object: %r" % args)
    ValueError: Unrecognized 'args' object: Namespace(verbosity=<conda.auxlib._Null object at 0x7fcf7b29de10>, debug=<conda.auxlib._Null object at 0x7fcf7b29de10>, trace=<conda.auxlib._Null object at 0x7
fcf7b29de10>, json=<conda.auxlib._Null object at 0x7fcf7b29de10>, no_plugins=True, cmd='mambabuild', _executable='mambabuild', _args=('.',))

Seems like I need to use different techniques to play nice with different version combinations. If I continue to explore this space, I will probably skip the conda 23.5.2 / mamba 1.4.9 compatibility, since I already have boa 0.14.0 for that, and try to get my fake boa to work for conda 24.3.0 and mamba 1.5.8.

[AaronOpfer]

@wolfv can you tell me if this is crazy? Patching mamba.py in mamba 1.5.8 like this actually makes mamba mambabuild work again with either boa 0.17.0 or with the fakeboa I pasted above. The sys.argv interaction is hacky, as it won't work with command like mamba --json mambabuild, but I have no idea who would do this, and it currently raises an exception, so how much worse could we possibly make the behavior?

def do_call(args, parser):
    if hasattr(args, "func"):
        relative_mod, func_name = args.func.rsplit(".", 1)
        # make module relative following https://github.com/conda/conda/pull/13173
        if relative_mod.startswith("conda.cli."):
            relative_mod = f'.{relative_mod.split(".")[-1]}'
    elif hasattr(args, "_plugin_subcommand"):
        action = args._plugin_subcommand.action
        relative_mod = f'.{action.__module__.split(".")[-1]}'
        func_name = action.__name__
    else:

        # NEW!!!!!!!!
        if hasattr(args, "cmd"):
            executable = shutil.which(f"conda-{args.cmd}")
            if executable is not None:
                os.execl(executable, executable, *sys.argv[2:])
        # end new.

        raise ValueError("Unrecognized 'args' object: %r" % args)