Botorch with cardinality constraint via sampling

[Waschenbacher]

(edited by @AdrianSosic)

This PR adds support for cardinality constraints to BotorchRecommender. The core idea is to tackle the problem in an exhaustive search like manner, i.e. by

• enumerating the possible combinations of in-/active parameters dictated by the cardinality constraints
• optimizing the corresponding restricted subspaces, where the cardinality constraint can then be simply removed since the in-/active sets are fixed within these subspaces
• aggregating the optimization results of the individual subspaces into a single recommendation batch.

The PR implements two mechanisms for determining the configuration of inactive parameters:

• When the combinatorial list of possible inactive parameter configurations is not too large, we iterate the full list
• otherwise, a fixed amount of inactive parameter configurations is randomly selected

The current aggregation step is to simply optimize all subspaces independently of each other and then return the batch from the subspace where the highest acquisition value is achieved. This has the side-effect that the set of inactive parameters is the same across the entire recommendation batch. This can be a desirable property in many use cases but potentially higher acquisition values can be obtained by altering the in-/activity sets across the batch. A simple way to achieve this (though out of scope for this PR) is by generalizing the sequential greedy principle to multiple subspaces.

Out of scope

• Fulfilling cardinality constraints by passing them in suitable form as nonlinear constraints to the optimizer
• Sequential greedy optimization to achieve varying in-/activity sets (see explanation above)

[AdrianSosic]

@Scienfitz, this beast is finally ready for ready for review. Let's try to really merge it in this time. As already mentioned, the entire PR would be significantly smaller had we omitted the (potentially use case irrelevant?) option of setting a min_cardinality. But what shall I say ... now it's there 🙃

Happy to see your feedback

[Scienfitz]

massive PR is massive

[Scienfitz]

isnt the cols == 0.0 part of obsolete? its also checked by is_between since 0 is enforced to be inside the interval?

[AdrianSosic]

No, not quite. There is a subtle issue because is_between gives False when the value is 0 and one of the bounds itself is zero (due to inequality check). Alternatively, we could resolve this by making is_between a true generalization of pandas.Series.is_between, which gives control on whether equality or inequality should be used. What do you prefer?

[Scienfitz]

that would mak sene to me and would be prefereable, the current line here feels weird

[AdrianSosic]

Ah no, my bad. The generalization does actually not solve the situation here. The special case was added for a different reason, namely to satisfy what is explained in the docstring of the cardinality constraint:

So is_between really needs to be invoked with < instead of <= to correctly count inactive parameters. Can you confirm that I'm not talking BS, @Waschenbacher?

[Waschenbacher]

Synced with @AdrianSosic. We decide to rework on this part that should solve this issue as well as the other open thread here.

[Scienfitz]

Besides the open comments I have nothing else other than the usual please add something to the userguides for both discrete and conti cardinality constraints