Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mechanism to test accuracy and precision of engines #181

Open
3 tasks
nirizr opened this issue Feb 5, 2017 · 1 comment
Open
3 tasks

Mechanism to test accuracy and precision of engines #181

nirizr opened this issue Feb 5, 2017 · 1 comment

Comments

@nirizr
Copy link
Owner

nirizr commented Feb 5, 2017

In order to properly develop decent engines, it would be convenient to have a easy way to benchmark and test results of available engines against a properly-sized labeled dataset.

  • Collect a properly-sized labeled dataset.
  • Create a mechanism to easily test engines against said set (without going through the IDA UI manually, see Add a command line rematch client #198 ).
  • Include as part of tests to streamline Engine creation (kinda requires IDA running in CI).
@nirizr
Copy link
Owner Author

nirizr commented Feb 21, 2017

Those are two out of three binaries from the old manual test set we used. Third is >50M compressed and above the 10M limit github has.

zues_variants.zip
file_dump.zip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant