Revert "Update .gitignore to exclude everything except recipe/ and conda-forge.yml"

[xhochy]

Currently, all builds using maturin are broken due to this change. As there seems to be no easy workaround, I would rather revert this and have a more in-depth discussion to avoid the issues stemming from it (e.g. we could also adapt our folder placements in the linux containers).

For more context see conda-forge/pydantic-core-feedstock#82

Checklist

• Added a news entry

cc @hmaarrfk

[isuruf]

Where in maturin is the gitignore file used?

[xhochy]

The PR that introduced the behavior is PyO3/maturin#695 but this only refers to other internals. I have seen other issues asking on how to solve it but they are all open/unaddressed.

[xhochy]

There is the following option to set something on a per-project basis: PyO3/maturin@ee86fd8

But, I think that this issue will not be limited to maturin. A lot of other tools (we just had a similar issue internally with rattler-build) also read the gitignore file for copying files to a destination. We should find a more general solution where a .gitignore does not pollute into the recipe build.

[davidhewitt]

cc @messense

There might be an argument that maturin should not be reading a .gitignore below the repository root? Though I guess based on what you say about rattler-build, it sounds like a lot of tools do this, whether it's correct or not.

[xhochy]

The rattler-build issue has been something different but probably using the same code path. We had am internal .gitignore that meant that rattler-build didn't copy the .git folder and thus setuptools_scm no longer worked.

For me, this was simply an example of "this problem is not limited to maturin".

[mbargull]

Having tools look arbitrarily up in the filesystem hierarchy is pretty annoying...
The changes to .gitignore from gh-1413 are very good and helpful for maintenance.

But I agree, let's revert this for now and later make changes to properly separate the build directories.
The necessary changes are likely to touch conda-forge-ci-setup too, so let's do this in an orderly fashion after the holidays and make a new patch release in the interim.

[isuruf]

Can't we just do conda-forge/pydantic-core-feedstock#84 ?

[mbargull]

Can't we just do conda-forge/pydantic-core-feedstock#84 ?

No, if we pick and choose that fine-grained, we have to consider things like
https://github.com/conda-forge/conda-forge-ci-setup-feedstock/blob/e403f01fd447e48ba3a777a9b642354ab1b7e746/recipe/meta.yaml#L23
which would break.
Yes, that file could also be bound.
And yes, that's a brittle thing to do generally.
Point is, IDK what else non-standard stuff is out there.

[xhochy]

We also need ti adjust how build-locally.py works in macOS as this also places the build directory below the feedstock root. There are a multitude of problems where this will come up.

[hmaarrfk]

My understanding is that xhochy is suggesting:

1. We revert
2. We reorganize hierchy
3. We reconsider reinstating it later.

Thr side effect caused by the current strategy took me 2-3 hours to troubleshoot with the help of two other very adept programmers.

Let's avoid this problem poping up when many of us may be trying to be away from our computers around the new years.

[erykoff]

I think the problem (if problem is the right word) is that the “ignore” file walker by default uses all parent gitignores https://docs.rs/ignore/latest/src/ignore/walk.rs.html#707 and this is used by maturin here https://github.com/PyO3/maturin/blob/1d7ad9da746fb252bb9c23e76c5df4a36c616c2a/src/source_distribution.rs#L536.

Also maturin is not the only rust tool that uses this. So there could other places that can behave strangely with new gitignores living anywhere in the parent directory tree.

[isuruf]

conda-forge/pydantic-core-feedstock#86 brings back the old behaviour for the build_artifacts directory

[isuruf]

See #1819

[isuruf]

Superseded by #1819