Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems with conda-build #87

Closed
2 tasks done
travishathaway opened this issue Jan 6, 2025 · 11 comments · Fixed by #96
Closed
2 tasks done

Problems with conda-build #87

travishathaway opened this issue Jan 6, 2025 · 11 comments · Fixed by #96
Assignees
Labels
bug Something isn't working

Comments

@travishathaway
Copy link
Collaborator

travishathaway commented Jan 6, 2025

Checklist

  • I added a descriptive title
  • I searched open reports and couldn't find a duplicate

What happened?

The following issue was encountered on CI for conda-standalone:

https://github.com/conda/conda-standalone/actions/runs/12586646383/job/35080929499?pr=123

Conda Info

n/a

Conda Config

n/a

Conda list

n/a

Additional Context

There was another independent report of this bug that also encountered using conda-build on osx-arm64.

Traceback

2025-01-02T18:04:59.0502180Z # >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<
2025-01-02T18:04:59.0503070Z 
2025-01-02T18:04:59.0503250Z     Traceback (most recent call last):
2025-01-02T18:04:59.0503870Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/core/index.py", line 289, in data
2025-01-02T18:04:59.0504360Z         return self._data
2025-01-02T18:04:59.0504900Z                ^^^^^^^^^^
2025-01-02T18:04:59.0505230Z     AttributeError: 'Index' object has no attribute '_data'. Did you mean: 'data'?
2025-01-02T18:04:59.0505640Z     
2025-01-02T18:04:59.0506350Z     During handling of the above exception, another exception occurred:
2025-01-02T18:04:59.0506710Z     
2025-01-02T18:04:59.0506920Z     Traceback (most recent call last):
2025-01-02T18:04:59.0507430Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/exception_handler.py", line 18, in __call__
2025-01-02T18:04:59.0507870Z         return func(*args, **kwargs)
2025-01-02T18:04:59.0508170Z                ^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0508580Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/cli/main.py", line 61, in main_subshell
2025-01-02T18:04:59.0509570Z         exit_code = do_call(args, parser)
2025-01-02T18:04:59.0509880Z                     ^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0510300Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/cli/conda_argparse.py", line 181, in do_call
2025-01-02T18:04:59.0510850Z         result = plugin_subcommand.action(getattr(args, "_args", args))
2025-01-02T18:04:59.0511250Z                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0511710Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/plugin.py", line 17, in build
2025-01-02T18:04:59.0512290Z         return execute(args)
2025-01-02T18:04:59.0512510Z                ^^^^^^^^^^^^^
2025-01-02T18:04:59.0512940Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/cli/main_build.py", line 648, in execute
2025-01-02T18:04:59.0513440Z         api.build(
2025-01-02T18:04:59.0513830Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/api.py", line 211, in build
2025-01-02T18:04:59.0514240Z         return build_tree(
2025-01-02T18:04:59.0515060Z                ^^^^^^^^^^^
2025-01-02T18:04:59.0515530Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/build.py", line 3653, in build_tree
2025-01-02T18:04:59.0515960Z         packages_from_this = build(
2025-01-02T18:04:59.0516240Z                              ^^^^^^
2025-01-02T18:04:59.0516640Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/build.py", line 2337, in build
2025-01-02T18:04:59.0517190Z         output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
2025-01-02T18:04:59.0517560Z                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0519890Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/render.py", line 926, in expand_outputs
2025-01-02T18:04:59.0520470Z         for output_dict, m in deepcopy(_m).get_output_metadata_set(
2025-01-02T18:04:59.0523820Z                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0524320Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/metadata.py", line 2662, in get_output_metadata_set
2025-01-02T18:04:59.0524850Z         conda_packages = finalize_outputs_pass(
2025-01-02T18:04:59.0525110Z                          ^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0525620Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/metadata.py", line 1009, in finalize_outputs_pass
2025-01-02T18:04:59.0526090Z         fm = finalize_metadata(
2025-01-02T18:04:59.0526320Z              ^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0526790Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/render.py", line 649, in finalize_metadata
2025-01-02T18:04:59.0527250Z         build_unsat, host_unsat = add_upstream_pins(
2025-01-02T18:04:59.0527520Z                                   ^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0528020Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/render.py", line 464, in add_upstream_pins
2025-01-02T18:04:59.0528570Z         build_deps, build_unsat, extra_run_specs_from_build = _read_upstream_pin_files(
2025-01-02T18:04:59.0528990Z                                                               ^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0529620Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/render.py", line 443, in _read_upstream_pin_files
2025-01-02T18:04:59.0530100Z         deps, precs, unsat = get_env_dependencies(
2025-01-02T18:04:59.0530420Z                              ^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0530870Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/render.py", line 157, in get_env_dependencies
2025-01-02T18:04:59.0531330Z         precs = environ.get_package_records(
2025-01-02T18:04:59.0531950Z                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0532400Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/environ.py", line 890, in get_install_actions
2025-01-02T18:04:59.0532980Z         _actions = _install_actions(prefix, index, specs, subdir=subdir)
2025-01-02T18:04:59.0533300Z                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0533750Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_build/environ.py", line 1299, in install_actions
2025-01-02T18:04:59.0534230Z         if index:
2025-01-02T18:04:59.0534830Z            ^^^^^
2025-01-02T18:04:59.0535220Z       File "/Users/runner/miniconda3/lib/python3.12/collections/__init__.py", line 1128, in __len__
2025-01-02T18:04:59.0535600Z         return len(self.data)
2025-01-02T18:04:59.0535820Z                    ^^^^^^^^^
2025-01-02T18:04:59.0536240Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/core/index.py", line 291, in data
2025-01-02T18:04:59.0536630Z         self._realize()
2025-01-02T18:04:59.0537050Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/core/index.py", line 354, in _realize
2025-01-02T18:04:59.0537740Z         self._data.update((prec, prec) for prec in subdir_data.iter_records())
2025-01-02T18:04:59.0538100Z                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0538620Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/core/subdir_data.py", line 284, in iter_records
2025-01-02T18:04:59.0539040Z         self.load()
2025-01-02T18:04:59.0539410Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/core/subdir_data.py", line 255, in load
2025-01-02T18:04:59.0539890Z         _internal_state = self._load()
2025-01-02T18:04:59.0540190Z                           ^^^^^^^^^^^^
2025-01-02T18:04:59.0540580Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/core/subdir_data.py", line 301, in _load
2025-01-02T18:04:59.0541080Z         repodata, state = fetcher.fetch_latest_parsed()
2025-01-02T18:04:59.0541410Z                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0541920Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/gateways/repodata/__init__.py", line 730, in fetch_latest_parsed
2025-01-02T18:04:59.0542480Z         parsed, state = self.fetch_latest()
2025-01-02T18:04:59.0542760Z                         ^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0543250Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/gateways/repodata/__init__.py", line 847, in fetch_latest
2025-01-02T18:04:59.0543880Z         raw_repodata = repo.repodata_parsed(cache.state)  # type: ignore
2025-01-02T18:04:59.0544210Z                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0544720Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/gateways/repodata/jlap/interface.py", line 87, in repodata_parsed
2025-01-02T18:04:59.0545420Z         repodata_json_or_none = fetch.request_url_jlap_state(
2025-01-02T18:04:59.0545760Z                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0546470Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/gateways/repodata/jlap/fetch.py", line 309, in request_url_jlap_state
2025-01-02T18:04:59.0547060Z         response = download_and_hash(
2025-01-02T18:04:59.0547350Z                    ^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0547920Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/gateways/repodata/jlap/fetch.py", line 243, in download_and_hash
2025-01-02T18:04:59.0548830Z         response = session.get(url, stream=True, timeout=timeout, headers=headers)
2025-01-02T18:04:59.0549200Z                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0549690Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/requests/sessions.py", line 602, in get
2025-01-02T18:04:59.0550310Z         return self.request("GET", url, **kwargs)
2025-01-02T18:04:59.0550620Z                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0551030Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/requests/sessions.py", line 575, in request
2025-01-02T18:04:59.0551450Z         prep = self.prepare_request(req)
2025-01-02T18:04:59.0551740Z                ^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0552250Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/gateways/connection/session.py", line 241, in prepare_request
2025-01-02T18:04:59.0552790Z         **context.plugin_manager.get_cached_session_headers(
2025-01-02T18:04:59.0553220Z           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0553760Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/plugins/manager.py", line 411, in get_session_headers
2025-01-02T18:04:59.0554470Z         for hook in self.get_hook_results("session_headers", host=host)
2025-01-02T18:04:59.0554820Z                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0555550Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda/plugins/manager.py", line 236, in get_hook_results
2025-01-02T18:04:59.0556200Z         plugins = [item for items in hook(**kwargs) for item in items]
2025-01-02T18:04:59.0556550Z                                                                 ^^^^^
2025-01-02T18:04:59.0557180Z       File "/Users/runner/miniconda3/lib/python3.12/site-packages/conda_anaconda_telemetry/hooks.py", line 244, in conda_session_headers
2025-01-02T18:04:59.0557730Z         if context.plugins.anaconda_telemetry:
2025-01-02T18:04:59.0558030Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-01-02T18:04:59.0558450Z     AttributeError: 'PluginConfig' object has no attribute 'anaconda_telemetry'
2025-01-02T18:04:59.0558680Z 
@travishathaway travishathaway added the bug Something isn't working label Jan 6, 2025
@travishathaway travishathaway self-assigned this Jan 6, 2025
@travishathaway
Copy link
Collaborator Author

travishathaway commented Jan 6, 2025

One idea I have for fixing this is to change our conda_session_headers hook function to the following:

@hookimpl
def conda_session_headers(host: str) -> Iterator[CondaRequestHeader]:
    """Return a list of custom headers to be included in the request."""
    try:
        if context.plugins.anaconda_telemetry:
            if host in REQUEST_HEADER_HOSTS:
                yield from validate_headers(_conda_request_headers())
    except Exception as exc:
        logger.debug("Failed to collect telemetry data", exc_info=exc)

Basically, we would just move accessing the context.plugins.anaconda_telemetry attribute into the try... except... clause. It would work, but I'm also curious about the underlying cause 🤔. I also wonder why this is only happening on osx-*...

@jezdez
Copy link
Member

jezdez commented Jan 7, 2025

I think that's just hiding the source of the problem. If conda-build isn't correctly initializing conda plugins, that's an issue.

@jezdez
Copy link
Member

jezdez commented Jan 7, 2025

@travishathaway When looking at the traceback carefully, I wonder if it didn't set up the plugin setting hook implementation correctly, best would be to try to reproduce the conda build run on mac os to see if we can retrace this

@marcoesters
Copy link

I have an additional observation that might help: this problem only seems to occur in the base environment. When I run the build in a separate environment, this goes away: conda/conda-standalone#125

This would explain why canary builds still work without problems.

@travishathaway
Copy link
Collaborator Author

Here's an additional location we are seeing this error:

@travishathaway
Copy link
Collaborator Author

@marcoesters,

Were you ever able to recreate this locally? I'm struggling with that at the moment. I have tested across most of the major platforms now (win-64, linux-64, and osx-arm64).

@marcoesters
Copy link

Yes, I have on macOS 14. Steps to reproduce:

  • Install Miniconda 24.11.1.
  • Activate the base environment.
  • Install conda-build
  • cd into the conda-standalone repo directory.
  • Run conda build recipe --override-channels -c conda-forge.

@marcoesters
Copy link

Strange, this only fails the first time. When I retry the build, the error does not occur.

@marcoesters
Copy link

It seems fairly flaky. The tests fail after installing test dependencies with the following errors:

AttributeError("'PluginConfig' object has no attribute 'anaconda_telemetry'")
AttributeError("'PluginConfig' object has no attribute 'anaconda_telemetry'")
AttributeError("'PluginConfig' object has no attribute 'anaconda_telemetry'")
AttributeError("'PluginConfig' object has no attribute 'anaconda_telemetry'")
AttributeError("'PluginConfig' object has no attribute 'anaconda_telemetry'")
AttributeError("'PluginConfig' object has no attribute 'anaconda_telemetry'")
AttributeError("'PluginConfig' object has no attribute 'anaconda_telemetry'")

@jezdez
Copy link
Member

jezdez commented Jan 9, 2025

@travishathaway and I made some progress in debugging this and believe that it's related to the way conda loads plugin settings that depend on an import-time side effect accidentally and how conda-build calls conda-internal API. We have some ideas how to fix this in conda-build and maybe in a minor release for the plugin, and lastly how to make conda's plugin settings loading more resilient.

@travishathaway
Copy link
Collaborator Author

This update to conda would fix this issue:

I would still like to add a fix to this repository too though. PR incoming...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants