feat: Add Validation for PK Mode Against PK Fields and Unit Tests #335
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Joel-hanson
force-pushed
the
validation-pk-mode
branch
from
a79a78f to
56b5e7e
Compare
C0urante
reviewed
Jun 29, 2024
|
|
||
| final Config config = new JdbcSinkConnector().validate(props); | ||
|
|
||
| assertFalse(config.configValues().stream().allMatch(cv -> cv.errorMessages().size() > 0)); |
There was a problem hiding this comment.
This asserts that there's at least one property that didn't have an error message. I'm guessing we actually want to assert that no properties have error messages?
Suggested change
| assertFalse(config.configValues().stream().allMatch(cv -> cv.errorMessages().size() > 0)); | |
| assertFalse(config.configValues().stream().anyMatch(cv -> cv.errorMessages().size() > 0)); |
C0urante
reviewed
Jun 29, 2024
There was a problem hiding this comment.
Thanks @Joel-hanson, this looks fantastic! Only have one small comment about a test case, everything else LGTM. If the suggestion makes sense, you can just amend the single current commit and force push, then we can merge.
Signed-off-by: Joel Hanson <joelhanson025@gmail.com> Signed-off-by: Joel Hanson <joel.hanson2@ibm.com>
Joel-hanson
force-pushed
the
validation-pk-mode
branch
from
56b5e7e to
fa39b97
Compare
|
@C0urante you are right and I have added in that change. |
C0urante
approved these changes
Jul 2, 2024
Merged
jclarysse
added a commit
to jclarysse/jdbc-connector-for-apache-kafka
that referenced
this pull request
Jan 31, 2025
Recently added validations (see PR Aiven-Open#335) are preventing the support of schema evolution use-cases, e.g. Documentation: ``` If 'pk.mode' is 'record_key' and 'pk.fields' is empty, then all fields from the key struct will be used. ``` Configuration: ``` "pk.mode": "record_key", "auto.create": "true", "auto.evolve": "true", ``` Validation rule: ``` Primary key fields must be set when pkMode is 'record_key'. ``` The purpose of this PR is to re-enable this feature for `record_key` mode.
jclarysse
added a commit
to jclarysse/jdbc-connector-for-apache-kafka
that referenced
this pull request
Feb 17, 2025
Recently added validations (see PR Aiven-Open#335) are preventing the support of schema evolution use-cases, e.g. Documentation: ``` If 'pk.mode' is 'record_key' and 'pk.fields' is empty, then all fields from the key struct will be used. ``` Configuration: ``` "pk.mode": "record_key", "auto.create": "true", "auto.evolve": "true", ``` Validation rule: ``` Primary key fields must be set when pkMode is 'record_key'. ``` The purpose of this PR is to re-enable this feature for both `record_key` and `record_value` modes.
adamstrickland
added a commit
to moment-technology/jdbc-connector-for-apache-kafka
that referenced
this pull request
Feb 25, 2025
* chore(deps): bump testcontainersVersion from 1.20.1 to 1.20.2 Bumps `testcontainersVersion` from 1.20.1 to 1.20.2. Updates `org.testcontainers:junit-jupiter` from 1.20.1 to 1.20.2 - [Release notes](https://github.com/testcontainers/testcontainers-java/releases) - [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md) - [Commits](testcontainers/testcontainers-java@1.20.1...1.20.2) Updates `org.testcontainers:kafka` from 1.20.1 to 1.20.2 - [Release notes](https://github.com/testcontainers/testcontainers-java/releases) - [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md) - [Commits](testcontainers/testcontainers-java@1.20.1...1.20.2) Updates `org.testcontainers:testcontainers` from 1.20.1 to 1.20.2 - [Release notes](https://github.com/testcontainers/testcontainers-java/releases) - [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md) - [Commits](testcontainers/testcontainers-java@1.20.1...1.20.2) Updates `org.testcontainers:postgresql` from 1.20.1 to 1.20.2 - [Release notes](https://github.com/testcontainers/testcontainers-java/releases) - [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md) - [Commits](testcontainers/testcontainers-java@1.20.1...1.20.2) Updates `org.testcontainers:oracle-free` from 1.20.1 to 1.20.2 - [Release notes](https://github.com/testcontainers/testcontainers-java/releases) - [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md) - [Commits](testcontainers/testcontainers-java@1.20.1...1.20.2) --- updated-dependencies: - dependency-name: org.testcontainers:junit-jupiter dependency-type: direct:production update-type: version-update:semver-patch - dependency-name: org.testcontainers:kafka dependency-type: direct:production update-type: version-update:semver-patch - dependency-name: org.testcontainers:testcontainers dependency-type: direct:production update-type: version-update:semver-patch - dependency-name: org.testcontainers:postgresql dependency-type: direct:production update-type: version-update:semver-patch - dependency-name: org.testcontainers:oracle-free dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * fix: re-allow empty `pk.fields` for some `pk.mode` Recently added validations (see PR Aiven-Open#335) are preventing the support of schema evolution use-cases, e.g. Documentation: ``` If 'pk.mode' is 'record_key' and 'pk.fields' is empty, then all fields from the key struct will be used. ``` Configuration: ``` "pk.mode": "record_key", "auto.create": "true", "auto.evolve": "true", ``` Validation rule: ``` Primary key fields must be set when pkMode is 'record_key'. ``` The purpose of this PR is to re-enable this feature for both `record_key` and `record_value` modes. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Anatoly Popov <anatolii.popov@aiven.io> Co-authored-by: Julien Clarysse <julien.clarysse@aiven.io> Co-authored-by: Jonas Keeling <jonas.keeling@aiven.io>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces validation logic for the primary key mode (
pk.mode) against the primary key fields (pk.fields) in theJdbcSinkConfigclass, ensuring that the configurations adhere to the documented requirements. Additionally, unit tests have been added to verify the correctness of the validation logic.Changes:
validatePKModeAgainstPKFieldsmethod to validate the primary key mode against the primary key fields.pk.modeandpk.fieldsto ensure configurations are valid according to the documentation.