Get publishing to Sonatype working #3

rbreslow · 2019-12-11T18:06:22Z

Overview

The following changes were made in an effort to get publishing to Sonatype working:

Add secret environment variables to CircleCI using contexts
Version artifacts with sbt-git
Add publishSettings to core project
Move dependencies to Dependencies.scala to align with other Scala project organization and to fix failing tests due to Versions.ScapegoatVersion reference error
Remove onLoad setting and aggregate core under a root project instead so that scalafmtSbtCheck has visibility into top-level *.sbt and project/*.scala files

Connects https://github.com/azavea/raster-foundry-platform/issues/899

Testing Instructions

See CircleCI build: https://app.circleci.com/jobs/github/azavea/stac4s/13

Retrieve the Azavea public key:

$ gpg --recv-keys 0x713F9F29598CFFF3
gpg: directory '/home/rocky/.gnupg' created
gpg: keybox '/home/rocky/.gnupg/pubring.kbx' created
gpgpg: /home/rocky/.gnupg/trustdb.gpg: trustdb created
gpg: key 713F9F29598CFFF3: public key "Azavea Inc. <systems@azavea.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1

Verify the signature of the published artifact:

$ curl -O https://oss.sonatype.org/content/repositories/snapshots/com/azavea/stac4s/core_2.12/ba29caf-SNAPSHOT/core_2.12-ba29caf-SNAPSHOT.jar
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  322k  100  322k    0     0  1772k      0 --:--:-- --:--:-- --:--:-- 1772k
$ curl -O https://oss.sonatype.org/content/repositories/snapshots/com/azavea/stac4s/core_2.12/ba29caf-SNAPSHOT/core_2.12-ba29caf-SNAPSHOT.jar.asc
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   833  100   833    0     0  10412      0 --:--:-- --:--:-- --:--:-- 10412

$ gpg --verify core_2.12-ba29caf-SNAPSHOT.jar.asc core_2.12-ba29caf-SNAPSHOT.jar
gpg: Signature made Wed 11 Dec 2019 12:54:10 PM EST
gpg:                using RSA key 2A780CD99D6FD6711BDD70623C45FEB5E8460BAD
gpg: Good signature from "Azavea Inc. <systems@azavea.com>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: FD92 9B9D 1047 3BD1 7B56  A718 713F 9F29 598C FFF3
     Subkey fingerprint: 2A78 0CD9 9D6F D671 1BDD  7062 3C45 FEB5 E846 0BAD

.circleci/config.yml

rbreslow · 2019-12-11T18:08:11Z

.circleci/config.yml

          requires:
            - "openjdk8-scala2.12"
          filters:
            tags:
              only:
                - /^(.*)$/
            branches:
-              ignore:
-                - /^(.*)$/
+              only:


Reminder to restore this before merging.

rbreslow · 2019-12-11T18:10:15Z

build.sbt


-cancelable in Global := true
-onLoad in Global ~= (_ andThen ("project core" :: _))


This was preventing scalafmtSbtCheck from having visibility into the top-level *.sbt and project/*.scala files.

See the alternative approach below with a root project.

build.sbt

rbreslow · 2019-12-11T18:17:06Z

build.sbt

  )
 )

+lazy val noPublishSettings = Seq(


This is so we don't publish the root project.

rbreslow · 2019-12-11T18:18:36Z

build.sbt

-      Resolver.ivyStylePatterns
-    )
-  )
+lazy val credentialSettings = Seq(


Some improvements to this block, courtsey of Eugene, geotrellis/vectorpipe@227de4f.

rbreslow · 2019-12-11T18:21:41Z

scripts/cipublish

@@ -2,7 +2,7 @@

 set -e

-if [[ -n "${STAC4S_SERVER_DEBUG}" ]]; then


My guess is this was borrowed from GeoTrellis Server and there was a control + find and replace operation that left this behind.

notthatbreezy

Looks good, I'd like to see the changes around Dependencies.scala reverted before merging though.

One small nitpick on the updated ./scripts/test too

notthatbreezy · 2019-12-11T19:54:13Z

project/Dependencies.scala

-  val ScapegoatVersion = "1.3.8"
-  val Specs2Version    = "4.6.0"
-  val TapirVersion     = "0.10.1"
+  val CatsVersion           = "1.6.0"


I rather keep the versions and dependencies in build.sbt and delete this file - or at the very least only have library versions in here. It's a lot easier to keep dependencies in check by keeping it in one file -- there's an argument made for that here for franklin in that it helps us when using sbt-explicit-dependencies.

To replicate what is done in the PR referenced there would require inlining the dependencies, but that's kind of outside the scope originally for the issue this PR so I'm happy to do that myself later.

sbt-explicit-dependencies is really really nice. I agree that the approach in Franklin looks better too.

scripts/test

build.sbt

rbreslow · 2019-12-12T15:41:33Z

scripts/test

+    if [[ "${1:-}" == "--help" ]]; then
+        usage
+    else
+        echo "Linting Scala source code and executing tests"


I decided not to carry over the color codes because they weren't working on my system:

maiden:~ rbreslow$ echo -e "\e[32m[stac4s] Executing Scala Tests and Linting\e[0m" \e[32m[stac4s] Executing Scala Tests and Linting\e[0m

Also, for consistency, because we weren't using them in cipublish.

If we were to include color codes, I would suggest an alternative approach that is portable (with printf) and provides more readable code:

diff --git a/scripts/test b/scripts/test index 8a4de5f..816a1e7 100755 --- a/scripts/test +++ b/scripts/test @@ -13,11 +13,16 @@ Lint Scala source code and execute tests. " } +function success() { + printf "$(tput setaf 2)%s$(tput sgr0)\n" $1 +} + if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then if [[ "${1:-}" == "--help" ]]; then usage else - echo "Linting Scala source code and executing tests" + success "Linting Scala source code and executing tests" + ./sbt ";\ scalafix --check; \ scalafmtCheck; \

I would also consider whether the value the color codes add to the script outweighs what they subtract in terms of complexity and decreased readability. If it is something we want to standardize on, which is totally cool, it may be worth extending the STRTA ADR so we can get every repository on the same page going forward.

Cool, I like your change -- in general I find the colors to be helpful in parsing the output of our STRTA commands (what's the command vs what's log output from processes the command invokes to be helpful). As to whether the increase in complexity for STRTA outweighs its usefulness - I would say that once written our STRTA commands aren't edited that much, but we run them often so it seems like a fair tradeoff.

Nevertheless, I don't think this change should hold up the PR since it's more a matter of preference and the original edit didn't seem related to the goal of publishing artifacts.

in general I find the colors to be helpful in parsing the output of our STRTA commands (what's the command vs what's log output from processes the command invokes to be helpful).

Agreed. It's definitely helpful.

As to whether the increase in complexity for STRTA outweighs its usefulness - I would say that once written our STRTA commands aren't edited that much, but we run them often so it seems like a fair tradeoff.

The thing in my mind is, if we replicate a success or log function across all the STRTA, there's more to parse and possibly distract when reading each file. Embedding the color codes in the printf statement introduces a different set of readability concerns. I agree with your perspective though, if we standardize on this, it might eventually become background noise.

At a higher level, it may be worth revisiting the STRTA ADR, because many projects have been deviating from the standards and that document should be updated for how we're developing apps today (Vagrant seems to be disused, etc).

rbreslow added 2 commits December 11, 2019 12:38

Get publishing to Sonatype working

37e2a76

Test publishing on CircleCI

ba29caf

rbreslow self-assigned this Dec 11, 2019

rbreslow commented Dec 11, 2019

View reviewed changes

rbreslow marked this pull request as ready for review December 11, 2019 18:22

rbreslow requested a review from notthatbreezy December 11, 2019 18:22

notthatbreezy suggested changes Dec 11, 2019

View reviewed changes

rbreslow added 7 commits December 11, 2019 16:45

Move dependencies back to Dependencies.scala

f015b05

Version CircleCI cache by Versions.scala

7039410

Add echo to scripts/test

492a959

Rename CircleCI context

1a1b291

Comment out unused dependencies

09ed9db

fixup! Comment out unused dependencies

984f227

fixup! fixup! Comment out unused dependencies

9f425b2

rbreslow commented Dec 12, 2019

View reviewed changes

rbreslow requested a review from notthatbreezy December 12, 2019 15:43

notthatbreezy approved these changes Dec 12, 2019

View reviewed changes

rbreslow merged commit 135006a into master Dec 12, 2019

rbreslow deleted the jrb/get-publishing-working branch December 12, 2019 16:11

rbreslow added a commit that referenced this pull request Dec 12, 2019

fixup! Get publishing to Sonatype working (#3)

091d2d0

pomadchin mentioned this pull request Jun 26, 2020

STAC 1.0.0-beta.1 support #116

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Get publishing to Sonatype working #3

Get publishing to Sonatype working #3

rbreslow commented Dec 11, 2019 •

edited

Loading

rbreslow Dec 11, 2019

rbreslow Dec 11, 2019

rbreslow Dec 11, 2019

rbreslow Dec 11, 2019

rbreslow Dec 11, 2019

notthatbreezy left a comment

notthatbreezy Dec 11, 2019

rbreslow Dec 12, 2019

rbreslow Dec 12, 2019 •

edited

Loading

notthatbreezy Dec 12, 2019

rbreslow Dec 12, 2019


		cancelable in Global := true
		onLoad in Global ~= (_ andThen ("project core" :: _))

		@@ -2,7 +2,7 @@

		set -e

		if [[ -n "${STAC4S_SERVER_DEBUG}" ]]; then

Get publishing to Sonatype working #3

Get publishing to Sonatype working #3

Conversation

rbreslow commented Dec 11, 2019 • edited Loading

Overview

Testing Instructions

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

notthatbreezy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rbreslow Dec 12, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rbreslow commented Dec 11, 2019 •

edited

Loading

rbreslow Dec 12, 2019 •

edited

Loading