Refactor Grouper objects #11575

Job	Run time
Event File	7s
detect ci trigger	7s
ubuntu-latest py3.9	8m 17s
ubuntu-latest py3.11	9m 5s
ubuntu-latest py3.12	8m 13s
macos-latest py3.9	10m 46s
macos-latest py3.11	10m 31s
macos-latest py3.12	11m 12s
windows-latest py3.9	15m 11s
windows-latest py3.11	18m 8s
windows-latest py3.12	17m 41s
ubuntu-latest py3.9 bare-minimum	1m 43s
ubuntu-latest py3.9 min-all-deps	7m 5s
ubuntu-latest py3.10 all-but-dask	6m 26s
ubuntu-latest py3.10 flaky	8m 39s
	2h 13m 11s

Provide feedback