Skip to content

Commit

Permalink
Mutate == to != and vice versa (#170)
Browse files Browse the repository at this point in the history
* Big refactor to allow smaller-than-function mutants
* Mutate `==` to `!=` and vice versa
* Add `--line-col=false`, mostly to allow tests to be less brittle
* Include column numbers in mutant listings to disambiguate multiple
potential mutations within one line
* Refactor code to represent mutants as styled or plain text so that the
two always stay consistent
* Add a `span` to mutant JSON, and a structured `function` submessage
  • Loading branch information
sourcefrog authored Nov 30, 2023
2 parents 0ddaa78 + 7f9f68c commit e18869a
Show file tree
Hide file tree
Showing 65 changed files with 5,946 additions and 1,340 deletions.
4 changes: 2 additions & 2 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,8 @@ applied.
`source.rs` -- A source tree and files within it, including visiting each source
file to find mutations.

`textedit.rs` -- A (line, column) addressing within a source file, and edits to
the content based on those addresses.
`span.rs` -- A (line, column) addressing within a source file, a range between
two positions, and edits to the content based on those addresses.

`visit.rs` -- Walk a source file's AST, and guess at likely-legal-but-wrong
replacements. The interface to the `syn` parser is localized here, and also the
Expand Down
12 changes: 12 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,16 @@

## 23.11.2

A big internal refactor to allow mutations smaller than a whole function. Only one pattern is added in this release, mutation of `==` operators, but many more are possible.

- New: Mutate `==` to `!=` and vice versa.

- Changed: Include column numbers in text listings of mutants and output to disambiguate smaller-than-function mutants, for example if there are several operators that can be changed on one line. This also applies to the names used for regex matching, so may break some regexps that match the entire line (sorry). The new option `--line-col=false` turns them both off in `--list` output.

- Changed: Replaced the `function`, `line`, and `return_type` fields in the mutants json message with a `function` submessage (including the name and return type) and a `span` indicating the entire replaced region.

## 23.11.2

- Changed: If `--file` or `--exclude` are set on the command line, then they replace the corresponding config file options. Similarly, if `--re` is given then the `examine_re` config key is ignored, and if `--exclude-re` is given then `exclude_regex` is ignored. (Previously the values were combined.) This makes it easier to use the command line to test files or mutants that are normally not tested.

- Improved: By default, files matching gitignore patterns (including in parent directories, per-user configuration, and `info/exclude`) are excluded from copying to temporary build directories. This should improve performance in some large trees with many files that are not part of the build. This behavior can be turned off with `--gitignore=false`.
Expand All @@ -12,6 +22,8 @@

- Added: Accept `--manifest-path` as an alternative to `-d`, for consistency with other cargo commands.

- Changed: The json mutants format now includes a `function` sub-message, which includes the function name and return type, rather than them being direct attributes of the mutant, to better accomodate smaller-than-function mutants. Also, the `function` includes the line-column span of the entire function.

## 23.11.1

- New `--in-diff FILE` option tests only mutants that are in the diff from the
Expand Down
22 changes: 20 additions & 2 deletions book/src/mutants.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,12 @@ cargo mutants generates mutants by inspecting the existing
source code and applying a set of rules to generate new code
that is likely to compile but have different behavior.

Mutants each have a "genre". In the current release, the only mutation genre is
`FnValue`, where a function's body is replaced with a value of the same type.
Mutants each have a "genre", each of which is described below.

## Replace function body with value

The `FnValue` genre of mutants replaces a function's body with a value that is guessed to be of the right type.

This checks that the tests:

1. Observe any side effects of the original function.
Expand Down Expand Up @@ -50,3 +54,17 @@ Some of these values may not be valid for all types: for example, returning
`Default::default()` will work for many types, but not all. In this case the
mutant is said to be "unviable": by default these are counted but not printed,
although they can be shown with `--unviable`.

## Binary operators

Binary operators are replaced with other binary operators in expressions
like `a == 0`.

| Operator | Replacements | Description |
| -------- | ------------ | ----------- |
| `==` | `!=` | Equality |
| `!=` | `==` | Equality |

Equality operators are not currently replaced with comparisons like `<` or `<=`
because they are
too prone to generate false positives, for example when unsigned integers are compared to 0.
30 changes: 4 additions & 26 deletions src/console.rs
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ impl Console {
write!(
s,
"{} ... {}",
style_scenario(scenario),
style_scenario(scenario, true),
style_outcome(outcome)
)
.unwrap();
Expand Down Expand Up @@ -493,7 +493,7 @@ impl ScenarioModel {
fn new(scenario: &Scenario, start: Instant, log_file: Utf8PathBuf) -> ScenarioModel {
ScenarioModel {
scenario: scenario.clone(),
name: style_scenario(scenario),
name: style_scenario(scenario, true),
phase: None,
phase_start: start,
log_file,
Expand Down Expand Up @@ -591,28 +591,6 @@ pub fn style_outcome(outcome: &ScenarioOutcome) -> StyledObject<&'static str> {
}
}

pub(crate) fn style_mutant(mutant: &Mutant) -> String {
// This is like `impl Display for Mutant`, but with colors.
// The text content should be the same.
let mut s = String::with_capacity(200);
write!(
&mut s,
"{}:{}",
mutant.source_file.tree_relative_slashes(),
mutant.span.start.line,
)
.unwrap();
s.push_str(": replace ");
s.push_str(&style(mutant.function_name()).bright().magenta().to_string());
if !mutant.return_type().is_empty() {
s.push(' ');
s.push_str(&style(mutant.return_type()).magenta().to_string());
}
s.push_str(" with ");
s.push_str(&style(mutant.replacement_text()).yellow().to_string());
s
}

fn style_elapsed_secs(since: Instant) -> String {
style_secs(since.elapsed())
}
Expand Down Expand Up @@ -640,10 +618,10 @@ fn style_mb(bytes: u64) -> StyledObject<String> {
style(format_mb(bytes)).cyan()
}

pub fn style_scenario(scenario: &Scenario) -> Cow<'static, str> {
pub fn style_scenario(scenario: &Scenario, line_col: bool) -> Cow<'static, str> {
match scenario {
Scenario::Baseline => "Unmutated baseline".into(),
Scenario::Mutant(mutant) => style_mutant(mutant).into(),
Scenario::Mutant(mutant) => mutant.name(line_col, true).into(),
}
}

Expand Down
4 changes: 2 additions & 2 deletions src/in_diff.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,8 @@ pub fn diff_filter(mutants: Vec<Mutant>, diff_text: &str) -> Result<Vec<Mutant>>
trace!(
?path,
line,
%mutant,
"diff matched mutant"
mutant = mutant.name(true, false),
"diff matched mutant"
);
matched.push(mutant);
continue 'mutant;
Expand Down
11 changes: 5 additions & 6 deletions src/list.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ use std::sync::Arc;

use serde_json::{json, Value};

use crate::console::style_mutant;
use crate::mutate::Mutant;
use crate::path::Utf8PathSlashes;
use crate::source::SourceFile;
Expand Down Expand Up @@ -48,11 +47,11 @@ pub(crate) fn list_mutants<W: fmt::Write>(
out.write_str(&serde_json::to_string_pretty(&list)?)?;
} else {
for mutant in mutants {
if options.colors {
writeln!(out, "{}", style_mutant(mutant))?;
} else {
writeln!(out, "{}", mutant)?;
}
writeln!(
out,
"{}",
mutant.name(options.show_line_col, options.colors)
)?;
if options.emit_diffs {
writeln!(out, "{}", mutant.diff())?;
}
Expand Down
9 changes: 8 additions & 1 deletion src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ mod pretty;
mod process;
mod scenario;
mod source;
mod textedit;
mod span;
mod visit;
mod workspace;

Expand Down Expand Up @@ -62,6 +62,9 @@ use crate::workspace::{PackageFilter, Workspace};
const VERSION: &str = env!("CARGO_PKG_VERSION");
const NAME: &str = env!("CARGO_PKG_NAME");

/// A comment marker inserted next to changes, so they can be easily found.
static MUTATION_MARKER_COMMENT: &str = "/* ~ changed by cargo-mutants ~ */";

#[derive(Parser)]
#[command(name = "cargo", bin_name = "cargo")]
enum Cargo {
Expand Down Expand Up @@ -177,6 +180,10 @@ struct Args {
#[arg(long)]
no_times: bool,

/// include line & column numbers in the mutation list.
#[arg(long, action = ArgAction::Set, default_value = "true")]
line_col: bool,

/// create mutants.out within this directory.
#[arg(long, short = 'o')]
output: Option<Utf8PathBuf>,
Expand Down
Loading

0 comments on commit e18869a

Please sign in to comment.