Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a tool to find uncovered files #529

Open
liamappelbe opened this issue Jun 18, 2024 · 5 comments · May be fixed by #2029
Open

Add a tool to find uncovered files #529

liamappelbe opened this issue Jun 18, 2024 · 5 comments · May be fixed by #2029
Labels
good first issue A good starting issue for contributors (issues with this label will appear in /contribute) package:coverage type-enhancement A request for a change that isn't a bug

Comments

@liamappelbe
Copy link
Contributor

If a file is not imported by the tests, it will not be compiled, so won't be mentioned in the source report. The compiler/VM won't be aware that the file even exists. This means that entire files can be uncovered without affecting the coverage report.

We could add a tool (or an option in an existing tool) that searches for all the .dart files in the package directory and reports any that are not mentioned in the coverage data.

The tool would need some filtering options, because it's common to have scripts that are not intended to be tested or used outside of the package (eg scripts in the tool directory). Maybe by default it should only look in the src subdirectory.

The tool will also need to support workspaces when they are added.

Context: flutter/flutter#145609

@liamappelbe liamappelbe added type-enhancement A request for a change that isn't a bug good first issue A good starting issue for contributors (issues with this label will appear in /contribute) labels Jun 18, 2024
@mosuem mosuem transferred this issue from dart-archive/coverage Aug 28, 2024
@khushi-hura
Copy link

hey @liamappelbe! As far as I can understand, this issue focusses on developing a tool which could search for some uncovered files during the test of the project. Could you please correct me if I am wrong or could you please ellaborate the issue?
I think I can work on it, will be sharing my approach in a few days!
Your response will be thankful :)

@liamappelbe
Copy link
Contributor Author

hey @liamappelbe! As far as I can understand, this issue focusses on developing a tool which could search for some uncovered files during the test of the project. Could you please correct me if I am wrong or could you please ellaborate the issue? I think I can work on it, will be sharing my approach in a few days! Your response will be thankful :)

Yep, that's the basic idea. Thinking a bit more about the design, it probably shouldn't be a separate tool. Instead it should be baked into the formatter. Add an optional parameter to FileHitMapsFormatter.formatLcov (and probably also FileHitMapsFormatter.prettyPrint) that controls the behavior of this feature.

The API could look something like this:

extension FileHitMapsFormatter on Map<String, HitMap> {
  String formatLcov(
    Resolver resolver, {
    String? basePath,
    List<String>? reportOn,
    Set<Glob>? ignoreGlobs,
    Function<bool(String)>? includeUncovered,  // <- new option
  }) {

If includeUncovered is null the behavior should stay the same as it is now. Otherwise it should be a function that returns whether the given path should be included in the repot if it's not covered. We'll also need a way for users to easily specify a sensible default behavior (eg only searching in the lib/src directory), so maybe add static function that does this and users can pass that for the typical behavior. That's just an idea though, so feel free to propose other APIs.

Once that's working you'll need to add a flag for it to bin/format_coverage.dart, and bin/test_with_coverage.dart. Once it's landed and published I'll add the option to Flutter. That way we'll get the new behavior across all the Dart and Flutter coverage use cases.

@khushi-hura
Copy link

Okay got that, will try working and let u know if I need any further help!

@Victowolf
Copy link

@liamappelbe @khushi-hura
Hello everyone, I'm Yash!

To tackle this issue, we can start by scanning all the .dart files in the package and comparing them with the coverage report. This would help identify files that aren’t covered at all (the ones never tested).

I had a few questions:

1)Should this be integrated into package: coverage, or would it be better as a standalone tool?
2)What kind of filtering options should be considered? Would a simple ignore list (like .gitignore) be enough, or should it be more flexible?
3)Should the default behavior apply only to lib/src/, or should we allow users to configure directories?

Victowolf added a commit to Victowolf/tools that referenced this issue Mar 6, 2025
@Victowolf Victowolf linked a pull request Mar 6, 2025 that will close this issue
1 task
@liamappelbe
Copy link
Contributor Author

1)Should this be integrated into package: coverage, or would it be better as a standalone tool?

It should be integrated into the formatter functions at first, then we'll add options for it to bin/format_coverage.dart and bin/test_with_coverage.dart. The command line option will probably be a list of globs.

2)What kind of filtering options should be considered? Would a simple ignore list (like .gitignore) be enough, or should it be more flexible?

Make it a Function<bool(String)>. That's the most flexible approach.

3)Should the default behavior apply only to lib/src/, or should we allow users to configure directories?

Default should be lib/src/. If the user wants to configure it more, they can pass a function or set command line flags.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue A good starting issue for contributors (issues with this label will appear in /contribute) package:coverage type-enhancement A request for a change that isn't a bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants