Skip to content

v8

Compare
Choose a tag to compare
@spbnick spbnick released this 13 Oct 11:44

Another major release. Changes include:

  • Support processing JSON streams for all command-line tools. Now it's possible to feed multiple JSON report objects, one after another, into a single KCIDB command, and have them processed appropriately. RFC 7464 is supported as well. This removes the overhead of starting the tool (and connecting to the cloud) for every submitted report.
  • Make kcidb-merge accept the reports to merge on standard input, as a JSON stream, instead of expecting them as file arguments.
  • Make kcidb-notify accept the "new" reports on standard input, as a JSON stream, instead of expecting them as file arguments.
  • Support splitting the data retrieved from the database into multiple reports, limited by the number of objects, when using the library or the command-line tools. This allows retrieving large amounts of data without running out of memory. Support output either as simple concatenated-JSON streams, one-report-per-line, or using the RFC 7464 format.
  • Extract the kcidb.io package into a separate distribution called kcidb-io, to minimize the number of dependencies required for validating report data. See its v1 and v2 release notes for changes since kcidb v7. One important change brought by this is enabling enforcement of format rules in JSON schema for fields containing URLs, timestamps, and email addresses. If any of those were incorrect in your data before, now they will fail to validate.
  • Make sure the report is successfully sent to the message queue before returning from the submit()/publish() function in kcidb library, to avoid data loss. Before this the report could be handled later by a separate thread for the purpose of batching multiple submissions. Provide a function (future_publish()) to still allow batching and delayed submission.
  • Make kcidb-submit and kcidb-mq-publisher-publish print "submission IDs" (message queue message IDs) of each sent report. Note that due to batching the IDs could be printed with a delay, even after multiple following report were accepted, but they would still be printed in order.
  • Reduce amount of internal consistency verification in KCIDB code, by default. This improves performance when processing multiple/large datasets.
  • Ignore Syzbot test results in PoC subscriptions until we implement issues/incidents and can handle its frequent test failures.
  • Stop sorting JSON object keys in command-line tool output. The order will change, but will still stay stable mostly.
  • Add SUBMISSION_HOWTO.md explaining how to start submitting to KCIDB.
  • Add a minimal Dockerfile for a container with KCIDB installed.