Skip to content

Commit

Permalink
Merge pull request #69 from DanShatford/main
Browse files Browse the repository at this point in the history
docs: fix typo "spurius" > "spurious"
  • Loading branch information
ivyleavedtoadflax authored Sep 28, 2023
2 parents 36fd20e + 996e05b commit 99e6187
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ We can use the following five metrics to consider difference categories of error
|Incorrect (INC)|the output of a system and the golden annotation don’t match|
|Partial (PAR)|system and the golden annotation are somewhat “similar” but not the same|
|Missing (MIS)|a golden annotation is not captured by a system|
|Spurius (SPU)|system produces a response which doesn’t exit in the golden annotation|
|Spurious (SPU)|system produces a response which doesn’t exit in the golden annotation|

These five metrics can be measured in four different ways:

Expand Down Expand Up @@ -172,7 +172,7 @@ __Putting all together:__
|Incorrect|2|0|2|3|
|Partial|0|2|0|0|
|Missed|1|1|1|1|
|Spurius|1|1|1|1|
|Spurious|1|1|1|1|
|Precision|0.5|0.66|0.5|0.33|
|Recall|0.5|0.66|0.5|0.33|
|F1|0.5|0.66|0.5|0.33|
Expand All @@ -182,9 +182,9 @@ __Putting all together:__

In scenarios IV and VI the entity type of the `true` and `pred` does not match, in both cases we only scored against
the true entity, not the predicted one. You can argue that the predicted entity could also be scored as spurious,
but according to the definition of `spurius`:
but according to the definition of `spurious`:

* Spurius (SPU) : system produces a response which does not exist in the golden annotation;
* Spurious (SPU) : system produces a response which does not exist in the golden annotation;

In this case there exists an annotation, but with a different entity type, so we assume it's only incorrect.

Expand Down

0 comments on commit 99e6187

Please sign in to comment.