Fuzzy matching #209
Closed
Fuzzy matching #209
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* update fuzzy match algorithm to cycle through multiple "same distance" matches until one passes the "first letter" rules. This was includes, because found an instance where there was an equal closest match that was a completely different genus and because multiple matches the fuzzy matches all returned NA. This will also mean that if there are multiple equally good matches it will align with the first.
further edits to fuzzy match - distances only calculated for names where the first letter of the first and second words in the input text matches names in the reference list with identical first letters for those words - this greatly sped up running the test dataset.
# Conflicts: # R/fuzzy_match.R
Reinstate new fuzzy match option This causes problems if input doesn't start with a letter -- which should be fixed during name standardising
Follow up changes to get tests passing with new fuzzy-matching algorithm that requires that the first letter of the input name is a-z. Still to consider: - In the fuzzy match, so matches where there are multiple equally-close distances, the first match is now being used. An additional column should be added with other possible matches. In our tests there were a number of genus-level matches that were returning NA's instead of the closest match, because of this (shorter strings are more likely to have multiple best matches).
Part of previous commit
I've gone back to requiring that `length(i) <= n_allowed`. The rest of the code I added (loop that starts with `for (j in 1:length(i)) {`) is required if a user ever manually set n_allowed > 1 - the old code would have broken. So for the main APCalign functions (`align_taxa` and `create_taxonomic_update_lookup` n_allowed is already set to 1, but now the fuzzy matching algorithm can be used on its own with a higher limit set.
Tests tweaked again to accommodate this.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Amending fuzzy matching algorithm to only consider subset of accepted list with same first letter
Greatly speeds up fuzzy matching