[warning] 2022-03-23 22:43:34 UTC - invalid artifacts

[conda-forge-curator]

Hey @conda-forge/systems!

It appears that we found some invalid artifacts:

+omniorb-libs:
+  osx-64/omniorb-libs-4.2.5-h942079c_1.tar.bz2:
+    bad_paths:
+      openssl:
+      - include/openssl/**/*
+  osx-64/omniorb-libs-4.2.5-hca56917_1.tar.bz2:
+    bad_paths:
+      openssl:
+      - include/openssl/**/*

We might want to mark these as broken if they are serious!

[beckermr]

Cc @conda-forge/core

@conda-forge/omniorb This needs to be fixed and these packages marked as broken.

[beckermr]

We cannot have other packages vendoring files from openssl. This is a big security risk.

[jakirkham]

The recipe does seem to list openssl in host for all of the packages (for example). Would think conda-build would have seen this and ignored those headers. So it is pretty weird that any headers from openssl got included.

Edit: Is it possible there is a conda-build or boa bug here?

[beckermr]

We should open up the packages and see what it found in there. Might shed some light on this.

[jakirkham]

This looks interesting. Would still expect conda-build to filter that correctly, but perhaps that is not happening. Idk how many people use files to include things in packages given install scripts are more common.

Are there any things outside of the headers that are getting picked up that shouldn't be? If the theory above is correct, would expect including bin/ to be a bigger problem.

[beckermr]

We should mark these as broken right away.

[jakirkham]

Doing that in PR ( conda-forge/admin-requests#415 )

Though still need to figure out/fix the recipe to ensure this doesn't happen in new packages

[beenje]

Hi! Sorry, this is my mistake. I noticed that yesterday and created a new MR to fix the issue: conda-forge/omniorb-feedstock#28

I'm quite confused about how conda-build handles outputs. The recipe has 2 outputs: omniorb-libs and omniorb

The build 0 of the package had - include in the omniorb list of files. Only proper files were included.
In build 1, I moved - include to the omniorb-libs output. conda-build added extra files (openssl and zlib). Why is that?
I thought conda-build does a comparison to only add new files. Why did that work on the omniorb output but not omniorb-libs?

In build 2, I listed explicitly the directories and files to be included.

[jakirkham]

Thanks for taking a look, Benjamin 🙏

Yeah I'm with you. Suspect that something in the build toolchain is not handling things correctly. It might be worth coming up with a trivial package that reproduces the issue that will help us debug this with the conda-build team. Maybe a split package that depends on openssl and uses files to include things in the final packages?

Listing the files explicitly should work. The one thing that gets tricky there is it is easy to miss adding new files. Think this is why most people end up using the install script strategy to include contents in each split package. Though either should be fine as long as it works well for your use case.

[beenje]

OK, I'll try to make a dummy package to reproduce the issue.

Do you have an example on how to use the script strategy? I'm interested in testing that.

[jakirkham]

Sure would take a look at openssl, which uses install scripts. Please let us know if you have any questions 🙂

[beenje]

I created an issue here conda/conda-build#4406
There is a link to a simple recipe to reproduce the problem.

[jakirkham]

That's great! Thank you for working on this Benjamin! 😄

[jakirkham]

Idk what the right behavior is. Have mixed feelings. That said, maybe we can save that discussion for the conda-build issue?

[beckermr]

I was wrong about the bootstrap compilers FWIW.

[jakirkham]

Ah ok. Thanks for looking into that further 🙂

Noticed zlib is doing the same thing. Admittedly that is lower in the stack so this particular issue is less likely to show up.