Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perform single publication for ShadowJar/JavaJar to overcome JFrog upload issue #58

Merged
merged 2 commits into from
Mar 27, 2024

Conversation

HotSushi
Copy link
Collaborator

@HotSushi HotSushi commented Mar 27, 2024

Summary

This is a follow up PR to #57.

The build for that PR succeeded locally but failed for JFrog because JFrog doesn't allow overriding existing artifacts. See the error stacktrace here:

> Task :tables-test-fixtures_2.12:publishShadowPublicationToLinkedInJFrogRepository FAILED
Multiple publications with coordinates 'com.linkedin.openhouse:tables-test-fixtures_2.12:0.5.27' are published to repository 'LinkedInJFrog'. The publications will overwrite each other!

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':tables-test-fixtures_2.12:publishShadowPublicationToLinkedInJFrogRepository'.
> Failed to publish publication 'shadow' to repository 'LinkedInJFrog'
   > Could not PUT 'https://linkedin.jfrog.io/artifactory/openhouse/com/linkedin/openhouse/tables-test-fixtures_2.12/0.5.27/tables-test-fixtures_2.12-0.5.27.pom'. Received status code 403 from server: 

In this PR we avoid overriding existing artifacts, instead make single publication either for mavenJava or shadowPublish

Note: the ShadowJar plugin doesn't support merging both publications, this is a known issue, see the comment

Changes

  • Bug Fixes

Testing Done

Also verified that dual publication doesn't occur, ie we dont see the warning:

Multiple publications with coordinates 'com.linkedin.openhouse:tables-test-fixtures_2.12:0.5.27' are published to repository 'LinkedInJFrog'. The publications will overwrite each other!

Investigated pom files and the mavenLocal publication manually to ensure all jars and poms are correctly published:

/Users/sraikar/.m2/repository/com/linkedin/openhouse/
├── common
│   ├── 1.0.0
│   │   ├── common-1.0.0-javadoc.jar
│   │   ├── common-1.0.0-sources.jar
│   │   ├── common-1.0.0-test-fixtures.jar
│   │   ├── common-1.0.0.jar
│   │   ├── common-1.0.0.module
│   │   └── common-1.0.0.pom
│   └── maven-metadata-local.xml
├── configs
│   ├── 1.0.0
│   │   ├── configs-1.0.0-javadoc.jar
│   │   ├── configs-1.0.0-sources.jar
│   │   ├── configs-1.0.0.jar
│   │   ├── configs-1.0.0.module
│   │   └── configs-1.0.0.pom
│   └── maven-metadata-local.xml
├── dummytokens
│   ├── 1.0.0
│   │   ├── dummytokens-1.0.0-javadoc.jar
│   │   ├── dummytokens-1.0.0-sources.jar
│   │   ├── dummytokens-1.0.0.jar
│   │   ├── dummytokens-1.0.0.module
│   │   └── dummytokens-1.0.0.pom
│   └── maven-metadata-local.xml
├── housetables
│   ├── 1.0.0
│   │   ├── housetables-1.0.0-javadoc.jar
│   │   ├── housetables-1.0.0-sources.jar
│   │   ├── housetables-1.0.0.jar
│   │   ├── housetables-1.0.0.module
│   │   └── housetables-1.0.0.pom
│   └── maven-metadata-local.xml
├── hts
│   ├── 1.0.0
│   │   ├── hts-1.0.0-javadoc.jar
│   │   ├── hts-1.0.0-sources.jar
│   │   ├── hts-1.0.0.jar
│   │   ├── hts-1.0.0.module
│   │   └── hts-1.0.0.pom
│   └── maven-metadata-local.xml
├── htscatalog
│   ├── 1.0.0
│   │   ├── htscatalog-1.0.0-javadoc.jar
│   │   ├── htscatalog-1.0.0-sources.jar
│   │   ├── htscatalog-1.0.0.jar
│   │   ├── htscatalog-1.0.0.module
│   │   └── htscatalog-1.0.0.pom
│   └── maven-metadata-local.xml
├── internalcatalog
│   ├── 1.0.0
│   │   ├── internalcatalog-1.0.0-javadoc.jar
│   │   ├── internalcatalog-1.0.0-sources.jar
│   │   ├── internalcatalog-1.0.0-test-fixtures.jar
│   │   ├── internalcatalog-1.0.0.jar
│   │   ├── internalcatalog-1.0.0.module
│   │   └── internalcatalog-1.0.0.pom
│   └── maven-metadata-local.xml
├── jobs
│   ├── 1.0.0
│   │   ├── jobs-1.0.0-javadoc.jar
│   │   ├── jobs-1.0.0-sources.jar
│   │   ├── jobs-1.0.0.jar
│   │   ├── jobs-1.0.0.module
│   │   └── jobs-1.0.0.pom
│   └── maven-metadata-local.xml
├── jobsclient
│   ├── 1.0.0
│   │   ├── jobsclient-1.0.0-javadoc.jar
│   │   ├── jobsclient-1.0.0-sources.jar
│   │   ├── jobsclient-1.0.0.jar
│   │   ├── jobsclient-1.0.0.module
│   │   └── jobsclient-1.0.0.pom
│   └── maven-metadata-local.xml
├── metrics
│   ├── 1.0.0
│   │   ├── metrics-1.0.0-javadoc.jar
│   │   ├── metrics-1.0.0-sources.jar
│   │   ├── metrics-1.0.0.jar
│   │   ├── metrics-1.0.0.module
│   │   └── metrics-1.0.0.pom
│   └── maven-metadata-local.xml
├── openhouse-java-itest
│   ├── 1.0.0
│   │   ├── openhouse-java-itest-1.0.0-javadoc.jar
│   │   ├── openhouse-java-itest-1.0.0-sources.jar
│   │   ├── openhouse-java-itest-1.0.0.jar
│   │   ├── openhouse-java-itest-1.0.0.module
│   │   └── openhouse-java-itest-1.0.0.pom
│   └── maven-metadata-local.xml
├── openhouse-java-runtime
│   ├── 1.0.0
│   │   ├── openhouse-java-runtime-1.0.0-javadoc.jar
│   │   ├── openhouse-java-runtime-1.0.0-sources.jar
│   │   ├── openhouse-java-runtime-1.0.0.jar
│   │   └── openhouse-java-runtime-1.0.0.pom
│   └── maven-metadata-local.xml
├── openhouse-spark-apps_2.12
│   ├── 1.0.0
│   │   ├── openhouse-spark-apps_2.12-1.0.0-all.jar
│   │   ├── openhouse-spark-apps_2.12-1.0.0-javadoc.jar
│   │   ├── openhouse-spark-apps_2.12-1.0.0-sources.jar
│   │   ├── openhouse-spark-apps_2.12-1.0.0.jar
│   │   └── openhouse-spark-apps_2.12-1.0.0.pom
│   └── maven-metadata-local.xml
├── openhouse-spark-itest
│   ├── 1.0.0
│   │   ├── openhouse-spark-itest-1.0.0-javadoc.jar
│   │   ├── openhouse-spark-itest-1.0.0-sources.jar
│   │   ├── openhouse-spark-itest-1.0.0.jar
│   │   ├── openhouse-spark-itest-1.0.0.module
│   │   └── openhouse-spark-itest-1.0.0.pom
│   └── maven-metadata-local.xml
├── openhouse-spark-runtime_2.12
│   ├── 1.0.0
│   │   ├── openhouse-spark-runtime_2.12-1.0.0-javadoc.jar
│   │   ├── openhouse-spark-runtime_2.12-1.0.0-sources.jar
│   │   ├── openhouse-spark-runtime_2.12-1.0.0.jar
│   │   └── openhouse-spark-runtime_2.12-1.0.0.pom
│   └── maven-metadata-local.xml
├── secureclient
│   ├── 1.0.0
│   │   ├── secureclient-1.0.0-javadoc.jar
│   │   ├── secureclient-1.0.0-sources.jar
│   │   ├── secureclient-1.0.0.jar
│   │   ├── secureclient-1.0.0.module
│   │   └── secureclient-1.0.0.pom
│   └── maven-metadata-local.xml
├── storage
│   ├── 1.0.0
│   │   ├── storage-1.0.0-javadoc.jar
│   │   ├── storage-1.0.0-sources.jar
│   │   ├── storage-1.0.0.jar
│   │   ├── storage-1.0.0.module
│   │   └── storage-1.0.0.pom
│   └── maven-metadata-local.xml
├── tableclient
│   ├── 1.0.0
│   │   ├── tableclient-1.0.0-javadoc.jar
│   │   ├── tableclient-1.0.0-sources.jar
│   │   ├── tableclient-1.0.0.jar
│   │   ├── tableclient-1.0.0.module
│   │   └── tableclient-1.0.0.pom
│   └── maven-metadata-local.xml
├── tables
│   ├── 1.0.0
│   │   ├── tables-1.0.0-javadoc.jar
│   │   ├── tables-1.0.0-sources.jar
│   │   ├── tables-1.0.0.jar
│   │   ├── tables-1.0.0.module
│   │   └── tables-1.0.0.pom
│   └── maven-metadata-local.xml
└── tables-test-fixtures_2.12
    ├── 1.0.0
    │   ├── tables-test-fixtures_2.12-1.0.0-all.jar
    │   ├── tables-test-fixtures_2.12-1.0.0-javadoc.jar
    │   ├── tables-test-fixtures_2.12-1.0.0-sources.jar
    │   ├── tables-test-fixtures_2.12-1.0.0.jar
    │   └── tables-test-fixtures_2.12-1.0.0.pom
    └── maven-metadata-local.xml

@HotSushi HotSushi changed the title if else publish Perform single publication for ShadowJar/JavaJar to overcome JFrog upload issue Mar 27, 2024
Copy link
Member

@abhisheknath2011 abhisheknath2011 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Sushant for the fix. Publishing might change with gradle 7.

@HotSushi HotSushi merged commit 2ed5182 into linkedin:main Mar 27, 2024
1 check passed
@HotSushi HotSushi deleted the fat_jar_1 branch May 13, 2024 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants