tests: Test relevant prereleases and allow to ignore releases

[sentrivana]

If a package has a prerelease of a higher version than the highest released stable version, make sure to test it, too. We consider alpha, beta, and RC releases.

Also add an option to ignore specific releases (this is related to the above since the script now pulls in two irrelevant alpha releases of starlite).

Closes #4030

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.53%. Comparing base (67f0491) to head (c1f96e3).
Report is 1 commits behind head on master.

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4073      +/-   ##
==========================================
- Coverage   79.54%   79.53%   -0.01%     
==========================================
  Files         140      140              
  Lines       15512    15512              
  Branches     2628     2628              
==========================================
- Hits        12339    12338       -1     
  Misses       2337     2337              
- Partials      836      837       +1     

see 1 file with indirect coverage changes

[szokeasaurusrex]

this is related to the above since the script now pulls in two irrelevant alpha releases of starlite

@sentrivana Just so I understand properly: can you please clarify why these releases are irrelevant? Is it because starlite has been replaced by Litestar?

Never mind, I found the answer: the reason for ignoring these releases is that Litestar replaced Starlite

[szokeasaurusrex]

I have one suggestion (marked [suggestion]) and two questions (marked [question]. The items marked [question] are mainly to improve my understanding and should not be considered requests to change anything, unless of course the question prompts you to want to change something.

I also have one potential concern here: what if a prerelease fails in testing due to a bug in the prerelease rather than a problem in the Sentry SDK? Would we just manually ignore these prereleases before merging the generated tox.ini?

[sentrivana]

I also have one potential concern here: what if a prerelease fails in testing due to a bug in the prerelease rather than a problem in the Sentry SDK? Would we just manually ignore these prereleases before merging the generated tox.ini?

Exactly, we would need to add them to the ignore. This is one reason why I'm explicitly ignoring dev releases since I expect they might be less stable. The rest should be reasonably ok (as in, I don't expect we'll need to ignore a lot of prereleases), at least judging from the prereleases I've been adding manually over the past years/months -- they're usually ok and if not, they require a couple of changes in the SDK, but they've never been really broken.

[sentrivana]

@szokeasaurusrex fyi, I've changed ignore to include with inverted logic here. The reason is that when ignoring releases, we'll probably want to specify just a handful of releases to ignore and allow the script to use everything else, but there's no way to have an or in a version specifier (you can do ==1.0,==1.1, but the comma acts as a logical and, so the version specifier actually doesn't make sense). You can have a negative in a version specifier though (!=1.0,!=1.1), which is exactly what we're looking for. But then the name ignore doesn't make sense anymore, so renamed it to include instead.