Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEV-1520 - get cost report working #335

Merged
merged 7 commits into from
Feb 10, 2025
Merged

DEV-1520 - get cost report working #335

merged 7 commits into from
Feb 10, 2025

Conversation

aelkiss
Copy link
Member

@aelkiss aelkiss commented Feb 5, 2025

This incorporates work from #329 and #333, and we will need to rebase after that's merged (and likely there will be more changes needed after #331 is merged as well). The relevant work here is my commits at the end. I'll leave some comments inline about specific things I'd like to think about.

@aelkiss aelkiss changed the base branch from mariadb to DEV-1488_ocnless_clusters February 5, 2025 22:46
@aelkiss aelkiss changed the base branch from DEV-1488_ocnless_clusters to mariadb February 5, 2025 22:47
@@ -131,25 +138,6 @@
end
end

describe "#matching_clusters" do
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We aren't implementing this at all now, so tests are gone.

@@ -83,6 +80,11 @@
end

config.before(:each) do
# mock HT member data to use in tests. Tests may change this data, so
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know why this worked before. What I observed was that data added with add_temp leaked between tests and caused issues. This seemed like a reasonable fix.

@@ -91,28 +85,43 @@ def add_cluster_htitem(*htitems)

describe "#cost_per_volume" do
it "calculates cost per volume" do
expect(cr.cost_per_volume).to eq(cr.target_cost / cr.num_volumes)
# 3 volumes, target cost $10
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found it hard to understand what the state of the database was given the various before blocks and additional stuff being added. I figured being explicit in each test about what's there was the best option.

expect(cr.cost_per_volume).to eq(cr.target_cost / cr.num_volumes)
# 3 volumes, target cost $10
add_cluster_htitem(spm, mpm, ht_allow)
expect(cr.cost_per_volume).to be_within(0.01).of(3.33)
Copy link
Member Author

@aelkiss aelkiss Feb 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also wanted to be explicit here about the expected result, rather than recomputing it using the code from the class. That makes the tests somewhat more brittle, but that brittleness is kind of endemic in this class anyway given the fixture data..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cost report tests might be (or at least have been) some of the most brittle in this repo. Some of that we just have to live with, but I appreciate the things you are doing to at least make things more explicit.

spec/integration_spec.rb Outdated Show resolved Hide resolved
@aelkiss aelkiss requested review from moseshll and mwarin February 5, 2025 22:58
@aelkiss aelkiss mentioned this pull request Feb 6, 2025
@aelkiss aelkiss force-pushed the DEV-1520-cost-report branch from 21e576f to cfa7b85 Compare February 7, 2025 20:58
@aelkiss aelkiss marked this pull request as ready for review February 7, 2025 20:59
@aelkiss
Copy link
Member Author

aelkiss commented Feb 7, 2025

Rebased; ready for review. See comments inline.

Copy link
Contributor

@moseshll moseshll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the changes made to parts I have had my hands on. The rest I find comprehensible. No issues in the tests. APPROVE

Copy link
Contributor

@mwarin mwarin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks good to me.

spec/integration_spec.rb Outdated Show resolved Hide resolved
lib/reports/cost_report.rb Outdated Show resolved Hide resolved
expect(cr.cost_per_volume).to eq(cr.target_cost / cr.num_volumes)
# 3 volumes, target cost $10
add_cluster_htitem(spm, mpm, ht_allow)
expect(cr.cost_per_volume).to be_within(0.01).of(3.33)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cost report tests might be (or at least have been) some of the most brittle in this repo. Some of that we just have to live with, but I appreciate the things you are doing to at least make things more explicit.

@aelkiss
Copy link
Member Author

aelkiss commented Feb 10, 2025

@mwarin thanks for the feedback. I will try to address those items.

* remove cluster_tap_save; new add_cluster_htitem helper for putting it
  in the hathifiles table and adding its OCNs to a cluster
* replace mongo queries from cost report (DEV-1523)
* remove matching_clusters (DEV-1522)
* alo -> allow; dni -> deny
* many tests are failing with wrong numbers, will investigate.
* add additional collections for use in integration tests
* add note about reliance on specific organizations listed here
* reset organizations/collections between tests
* billing_entity -> collection code for htitems & associated reorganization
* clarify for many tests what exactly we expect to be present in the
  database; avoid before block that creates stuff leaking scope
* use cr.compile_frequency_table instead of repeating that code in tests
  (which relied on matching_clusters, which is no longer relevant for
  mariadb since we iterate directly over htitems)
This is a bit messy in terms of spelled-out fixture data being included
in the tests since we don't have the ability to directly load JSON data
any more.
@aelkiss aelkiss force-pushed the DEV-1520-cost-report branch from cfa7b85 to 974e491 Compare February 10, 2025 21:00
* remove trivial methods in costreport for counts
* Move data for integration spec to support context
* cluster_tap_save -> load_test_data
@aelkiss aelkiss force-pushed the DEV-1520-cost-report branch from ded40e4 to 6a8c867 Compare February 10, 2025 21:55
@aelkiss
Copy link
Member Author

aelkiss commented Feb 10, 2025

@mwarin I've addressed feedback and will go ahead and merge once specs & standardrb are happy.

@aelkiss aelkiss merged commit 1f929b1 into mariadb Feb 10, 2025
1 check passed
@aelkiss aelkiss deleted the DEV-1520-cost-report branch February 10, 2025 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants