feat(namespaces): add empty namespace detection and removal

[isindir]

What this PR does / why we need it?

This PR finds empty and non-default kubernetes namespaces, and if instructed removes these from the cluster.

PR Checklist

• This PR adds K8s exceptions (false positives)
• This PR adds new code
• This PR includes tests for new/existing code
• This PR adds docs

GitHub Issue

Closes [#92]

Notes for your reviewers

[isindir]

fixes #92

This PR is still WIP, as it lacks parallel processing of the namespaces as well as unit tests, could you please review if in principle it is what you'd be happy to accept ?

[yonahd]

fixes #92

This PR is still WIP, as it lacks parallel processing of the namespaces as well as unit tests, could you please review if in principle it is what you'd be happy to accept ?

Hey @isindir
I'll take a look at the code in the next couple of days

[doronkg]

Hi @isindir,
In order for the new feature to be completed, several additional files should be modified accordingly.

Take a look at https://github.com/yonahd/kor/blob/main/CONTRIBUTING.md#repository-structure.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 64.41718% with 58 lines in your changes missing coverage. Please review.

Project coverage is 44.36%. Comparing base (3a8f77b) to head (199343d).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
pkg/kor/namespaces.go	76.98%	21 Missing and 8 partials ⚠️
cmd/kor/namespaces.go	0.00%	22 Missing ⚠️
pkg/kor/delete.go	53.33%	7 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #249      +/-   ##
==========================================
+ Coverage   43.58%   44.36%   +0.78%     
==========================================
  Files          65       67       +2     
  Lines        4199     4355     +156     
==========================================
+ Hits         1830     1932     +102     
- Misses       2111     2157      +46     
- Partials      258      266       +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[yonahd]

fixes #92

This PR is still WIP, as it lacks parallel processing of the namespaces as well as unit tests, could you please review if in principle it is what you'd be happy to accept ?

This is definitely a good direction(this is not a simple one).
Definitely need to review this thoroughly but in principal this is good

[isindir]

@doronkg,

Hi @isindir, In order for the new feature to be completed, several additional files should be modified accordingly.

Take a look at https://github.com/yonahd/kor/blob/main/CONTRIBUTING.md#repository-structure.

Helm chart I think needs extra changes:

• at the moment it is not possible to configure kor in a deployment for example to automatically remove any orphaned resources - rbac lacks such permissions.
• I think it would be more useful to split cronjobs so that several such jobs are configured with different schedules (may become handy when removing some resources and then namespaces in a separate run.

I'm looking to work on unit tests if time allows these weekends.

[doronkg]

Helm chart I think needs extra changes:

• at the moment it is not possible to configure kor in a deployment for example to automatically remove any orphaned resources - rbac lacks such permissions.

Correct, currently the deployment is only used to export metrics, hence read-only RBAC, but we can consider going that way.

• I think it would be more useful to split cronjobs so that several such jobs are configured with different schedules (may become handy when removing some resources and then namespaces in a separate run.

Interesting, especially when scanning unused resources in high-scaled clusters that might take more time than the interval set.

We can continue with progress on both comments in separated issues/discussions (or in our Discord channel) as they are out-of-scope for this PR, but should be dippen into. I'd like to futher discuss them.

Referring to https://github.com/yonahd/kor/blob/main/CONTRIBUTING.md#repository-structure was made to make sure you modify all required files, as adding any new unused resource support should address them.

[isindir]

@doronkg ,

Referring to https://github.com/yonahd/kor/blob/main/CONTRIBUTING.md#repository-structure was made to make sure you modify all required files, as adding any new unused resource support should address them.

Thank you, I think I covered most of the files except helm chart and some unit tests, atm I'm trying to figure out if I can use fake clients to reliably test a feature I'm implementing. I deliberately left chart untouched as the change already looks very big.

btw, for chart - it will be better to have fine-tuned RBAC - possibility to enable only those read/write/per namespace permissions per feature (resource).

[isindir]

@yonahd @luisdavim @doronkg, I finally found time and adding some more testing to the namespace removal - but I was not following if json configs with resources to skip from namespace evaluation for different k8s distros is fully implemented or not. Could you please suggest and review if my testing in pkg/kor/namespacesMoreTests_test.go is acceptable ? Thanks

[yonahd]

@yonahd @luisdavim @doronkg, I finally found time and adding some more testing to the namespace removal - but I was not following if json configs with resources to skip from namespace evaluation for different k8s distros is fully implemented or not. Could you please suggest and review if my testing in pkg/kor/namespacesMoreTests_test.go is acceptable ? Thanks

The resource exceptions are merged fully.
Not sure we will use the namespaces exceptions as we worked around it

[yonahd]

Let me know when this is ready for review. This is a massive pr and will take quite a while to review

[isindir]

@yonahd I feel that I'll not be able to do 100% unit test coverage for my new code, please review this PR. Thanks.

[isindir]

@yonahd - I rebased the PR to the latest, would you have time to review and may be merge it ? We could negotiate next course of actions if needed from my side on a call, please reach me out on keybase , same uid as here.

[yonahd]

@yonahd - I rebased the PR to the latest, would you have time to review and may be merge it ? We could negotiate next course of actions if needed from my side on a call, please reach me out on keybase , same uid as here.

Thanks for the PR
I'm currently very short in time. Any chance @doronkg or @luisdavim can you review this?

[doronkg]

@yonahd - I rebased the PR to the latest, would you have time to review and may be merge it ? We could negotiate next course of actions if needed from my side on a call, please reach me out on keybase , same uid as here.

Thanks for the PR

I'm currently very short in time. Any chance @doronkg or @luisdavim can you review this?

Hi, sorry for the late response.
I'm currently on vacation so my time is very limited. From a quick glimpse - I won't be able to review this PR thoroughly in the following weeks unfortunately.