Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get publishing to Sonatype working #3

Merged
merged 9 commits into from
Dec 12, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ aliases:
- checkout
- restore_cache:
keys:
- sbt-cache-{{ checksum "build.sbt" }}
- sbt-cache-{{ checksum "project/Versions.scala" }}
- run:
name: Executing cibuild
command: ./scripts/test
- save_cache:
key: sbt-cache-{{ checksum "build.sbt" }}
key: sbt-cache-{{ checksum "project/Versions.scala" }}
paths:
- "~/.ivy2/cache"
- "~/.sbt"
Expand All @@ -18,7 +18,7 @@ aliases:
- checkout
- restore_cache:
keys:
- sbt-cache-{{ checksum "build.sbt" }}
- sbt-cache-{{ checksum "project/Versions.scala" }}
- run:
name: "Import signing key"
command: |
Expand All @@ -40,20 +40,24 @@ workflows:
build:
jobs:
- "openjdk8-scala2.12":
filters: # required since `openjdk8-scala2.12.8_deploy` has tag filters AND requires `openjdk8-scala2.12.8`
# required since openjdk8-scala2.12_deploy has tag filters AND requires
# openjdk8-scala2.12
# https://circleci.com/docs/2.0/workflows/#executing-workflows-for-a-git-tag
filters:
tags:
only:
- /^(.*)$/
- "openjdk8-scala2.12_deploy":
context: sonatype-azavea-signing-key
requires:
- "openjdk8-scala2.12"
filters:
tags:
only:
- /^(.*)$/
branches:
ignore:
- /^(.*)$/
only:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reminder to restore this before merging.

- jrb/get-publishing-working

jobs:
"openjdk8-scala2.12":
Expand Down
186 changes: 87 additions & 99 deletions build.sbt
Original file line number Diff line number Diff line change
@@ -1,23 +1,61 @@
import xerial.sbt.Sonatype._

cancelable in Global := true
onLoad in Global ~= (_ andThen ("project core" :: _))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was preventing scalafmtSbtCheck from having visibility into the top-level *.sbt and project/*.scala files.

See the alternative approach below with a root project.


lazy val credentialSettings = Seq(
credentials += Credentials(
"GnuPG Key ID",
"gpg",
System.getenv().get("GPG_KEY_ID"),
"ignored"
lazy val commonSettings = Seq(
// We are overriding the default behavior of sbt-git which, by default, only
// appends the `-SNAPSHOT` suffix if there are uncommitted changes in the
// workspace.
version := {
if (git.gitDescribedVersion.value.isEmpty)
git.gitHeadCommit.value.get.substring(0, 7) + "-SNAPSHOT"
else if (git.gitCurrentTags.value.isEmpty || git.gitUncommittedChanges.value)
git.gitDescribedVersion.value.get + "-SNAPSHOT"
else
git.gitDescribedVersion.value.get
},
scalaVersion := "2.12.10",
cancelable in Global := true,
scalafmtOnCompile := true,
scapegoatVersion in ThisBuild := Versions.ScapegoatVersion,
scapegoatDisabledInspections := Seq("ObjectNames", "EmptyCaseClass"),
unusedCompileDependenciesFilter -= moduleFilter("com.sksamuel.scapegoat", "scalac-scapegoat-plugin"),
addCompilerPlugin("org.spire-math" %% "kind-projector" % "0.9.6"),
addCompilerPlugin("com.olegpy" %% "better-monadic-for" % "0.2.4"),
addCompilerPlugin(
"org.scalamacros" % "paradise" % "2.1.0" cross CrossVersion.full
),
credentials += Credentials(
"Sonatype Nexus Repository Manager",
"oss.sonatype.org",
System.getenv().get("SONATYPE_USERNAME"),
System.getenv().get("SONATYPE_PASSWORD")
addCompilerPlugin(scalafixSemanticdb),
autoCompilerPlugins := true,
externalResolvers := Seq(
DefaultMavenRepository,
Resolver.sonatypeRepo("snapshots"),
Resolver.typesafeIvyRepo("releases"),
Resolver.bintrayRepo("azavea", "maven"),
Resolver.bintrayRepo("azavea", "geotrellis"),
"locationtech-releases" at "https://repo.locationtech.org/content/groups/releases",
"locationtech-snapshots" at "https://repo.locationtech.org/content/groups/snapshots",
Resolver.bintrayRepo("guizmaii", "maven"),
Resolver.bintrayRepo("colisweb", "maven"),
"jitpack".at("https://jitpack.io"),
Resolver.file("local", file(Path.userHome.absolutePath + "/.ivy2/local"))(
Resolver.ivyStylePatterns
)
)
)

lazy val noPublishSettings = Seq(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is so we don't publish the root project.

publish := {},
publishLocal := {},
publishArtifact := false
)

lazy val publishSettings = Seq(
organization := "com.azavea.stac4s",
organizationName := "Azavea",
organizationHomepage := Some(new URL("https://azavea.com/")),
description := "stac4s is a scala library with primitives to build applications using the SpatioTemporal Asset Catalogs specification",
publishArtifact in Test := false
) ++ sonatypeSettings ++ credentialSettings

lazy val sonatypeSettings = Seq(
publishMavenStyle := true,
sonatypeProfileName := "com.azavea",
Expand Down Expand Up @@ -52,96 +90,46 @@ lazy val sonatypeSettings = Seq(
publishTo := sonatypePublishTo.value
)

lazy val publishSettings = Seq(
organization := "com.azavea.stac4s",
organizationName := "Azavea",
organizationHomepage := Some(new URL("https://azavea.com/")),
description := "stac4s is a scala library with primitives to build applications using the SpatioTemporal Asset Catalogs specification",
publishArtifact in Test := false
) ++ sonatypeSettings ++ credentialSettings

// Versions
val CatsVersion = "1.6.0"
val CirceFs2Version = "0.11.0"
val CirceVersion = "0.11.1"
val GeoTrellisVersion = "3.0.0-M3"
val RefinedVersion = "0.9.3"
val ScapegoatVersion = "1.3.8"
val ShapelessVersion = "2.3.3"
val spdxCheckerVersion = "1.0.0"
val scalacheckCatsVersion = "0.1.1"
val scalatestVersion = "3.0.4"
val sprayVersion = "1.3.4"
val scalacheckVersion = "1.14.0"

// Dependencies
val cats = "org.typelevel" %% "cats-core" % CatsVersion
val circeCore = "io.circe" %% "circe-core" % CirceVersion
val circeFs2 = "io.circe" %% "circe-fs2" % CirceFs2Version
val circeGeneric = "io.circe" %% "circe-generic" % CirceVersion
val circeParser = "io.circe" %% "circe-parser" % CirceVersion
val circeRefined = "io.circe" %% "circe-refined" % CirceVersion
val circeShapes = "io.circe" %% "circe-shapes" % CirceVersion
val geotrellisVector = "org.locationtech.geotrellis" %% "geotrellis-vector" % GeoTrellisVersion
val refined = "eu.timepit" %% "refined" % RefinedVersion
val scalacheck = "org.scalacheck" %% "scalacheck" % scalacheckVersion % Test
val scalacheckCats = "io.chrisdavenport" %% "cats-scalacheck" % scalacheckCatsVersion % Test
val scalatest = "org.scalatest" %% "scalatest" % scalatestVersion % Test
val spdxChecker = "com.github.tbouron" % "spdx-license-checker" % spdxCheckerVersion
val shapeless = "com.chuusai" %% "shapeless" % ShapelessVersion
val spray = "io.spray" %% "spray-json" % sprayVersion

lazy val settings = Seq(
organization := "com.azavea",
name := "stac4s",
version := "0.0.1-SNAPSHOT",
scalaVersion := "2.12.10",
scalafmtOnCompile := true,
scapegoatVersion in ThisBuild := Versions.ScapegoatVersion,
scapegoatDisabledInspections := Seq("ObjectNames", "EmptyCaseClass"),
unusedCompileDependenciesFilter -= moduleFilter("com.sksamuel.scapegoat", "scalac-scapegoat-plugin"),
addCompilerPlugin("org.spire-math" %% "kind-projector" % "0.9.6"),
addCompilerPlugin("com.olegpy" %% "better-monadic-for" % "0.2.4"),
addCompilerPlugin(
"org.scalamacros" % "paradise" % "2.1.0" cross CrossVersion.full
),
addCompilerPlugin(scalafixSemanticdb),
autoCompilerPlugins := true,
externalResolvers := Seq(
DefaultMavenRepository,
Resolver.sonatypeRepo("snapshots"),
Resolver.typesafeIvyRepo("releases"),
Resolver.bintrayRepo("azavea", "maven"),
Resolver.bintrayRepo("azavea", "geotrellis"),
"locationtech-releases" at "https://repo.locationtech.org/content/groups/releases",
"locationtech-snapshots" at "https://repo.locationtech.org/content/groups/snapshots",
Resolver.bintrayRepo("guizmaii", "maven"),
Resolver.bintrayRepo("colisweb", "maven"),
"jitpack".at("https://jitpack.io"),
Resolver.file("local", file(Path.userHome.absolutePath + "/.ivy2/local"))(
Resolver.ivyStylePatterns
)
)
lazy val credentialSettings = Seq(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some improvements to this block, courtsey of Eugene, geotrellis/vectorpipe@227de4f.

credentials ++= List(
for {
id <- sys.env.get("GPG_KEY_ID")
} yield Credentials("GnuPG Key ID", "gpg", id, "ignored"),
for {
user <- sys.env.get("SONATYPE_USERNAME")
pass <- sys.env.get("SONATYPE_PASSWORD")
} yield Credentials("Sonatype Nexus Repository Manager", "oss.sonatype.org", user, pass)
).flatten
)

lazy val dependencies = Seq(
cats,
circeCore,
circeGeneric,
circeParser,
refined,
shapeless,
scalacheck,
scalacheckCats,
scalatest,
spdxChecker,
spray,
geotrellisVector
val coreDependencies = Seq(
"org.typelevel" %% "cats-core" % Versions.CatsVersion,
"io.circe" %% "circe-core" % Versions.CirceVersion,
"io.circe" %% "circe-generic" % Versions.CirceVersion,
"io.circe" %% "circe-parser" % Versions.CirceVersion,
"org.locationtech.geotrellis" %% "geotrellis-vector" % Versions.GeoTrellisVersion,
"eu.timepit" %% "refined" % Versions.RefinedVersion,
"org.scalacheck" %% "scalacheck" % Versions.scalacheckVersion % Test,
"io.chrisdavenport" %% "cats-scalacheck" % Versions.scalacheckCatsVersion % Test,
"org.scalatest" %% "scalatest" % Versions.scalatestVersion % Test,
"com.github.tbouron" % "spdx-license-checker" % Versions.spdxCheckerVersion,
"com.chuusai" %% "shapeless" % Versions.ShapelessVersion,
"io.spray" %% "spray-json" % Versions.sprayVersion
)

lazy val root = project
.in(file("."))
.settings(moduleName := "root")
.settings(commonSettings)
.settings(publishSettings)
.settings(noPublishSettings)
.aggregate(core)

lazy val core = (project in file("modules/core"))
.settings(settings: _*)
.settings(commonSettings)
.settings(publishSettings)
.settings({
libraryDependencies ++= dependencies
libraryDependencies ++= coreDependencies
})

lazy val coreRef = LocalProject("modules/core")
59 changes: 0 additions & 59 deletions project/Dependencies.scala

This file was deleted.

13 changes: 13 additions & 0 deletions project/Versions.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
object Versions {
val CatsVersion = "1.6.0"
val CirceVersion = "0.11.1"
val GeoTrellisVersion = "3.0.0-M3"
val RefinedVersion = "0.9.3"
val ScapegoatVersion = "1.3.8"
val ShapelessVersion = "2.3.3"
val spdxCheckerVersion = "1.0.0"
val scalacheckCatsVersion = "0.1.1"
val scalatestVersion = "3.0.4"
val sprayVersion = "1.3.4"
val scalacheckVersion = "1.14.0"
}
2 changes: 1 addition & 1 deletion scripts/cipublish
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

set -e

if [[ -n "${STAC4S_SERVER_DEBUG}" ]]; then
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My guess is this was borrowed from GeoTrellis Server and there was a control + find and replace operation that left this behind.

if [[ -n "${STAC4S_DEBUG}" ]]; then
set -x
fi

Expand Down
34 changes: 19 additions & 15 deletions scripts/test
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#! /bin/bash
#!/bin/bash

set -e

Expand All @@ -7,21 +7,25 @@ if [[ -n "${STAC4S_DEBUG}" ]]; then
fi

function usage() {
echo -ne \
"Usage: $(basename "${0}")
Run linters and tests.
echo -n \
"Usage: $(basename "$0")
Lint Scala source code and execute tests.
"
}


if [ "${BASH_SOURCE[0]}" = "${0}" ]; then
case "${1}" in
--help)
usage
;;
*)
echo -e "\e[32m[stac4s] Executing Scala Tests and Linting\e[0m"
./sbt ";scalafix --check;scalafmtCheck;scalafmtSbtCheck;scapegoat;undeclaredCompileDependenciesTest;unusedCompileDependenciesTest;test"
;;
esac
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
if [[ "${1:-}" == "--help" ]]; then
usage
else
echo "Linting Scala source code and executing tests"
Copy link
Contributor Author

@rbreslow rbreslow Dec 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I decided not to carry over the color codes because they weren't working on my system:

maiden:~ rbreslow$ echo -e "\e[32m[stac4s] Executing Scala Tests and Linting\e[0m"
\e[32m[stac4s] Executing Scala Tests and Linting\e[0m

Also, for consistency, because we weren't using them in cipublish.

If we were to include color codes, I would suggest an alternative approach that is portable (with printf) and provides more readable code:

diff --git a/scripts/test b/scripts/test
index 8a4de5f..816a1e7 100755
--- a/scripts/test
+++ b/scripts/test
@@ -13,11 +13,16 @@ Lint Scala source code and execute tests.
 "
 }

+function success() {
+    printf "$(tput setaf 2)%s$(tput sgr0)\n" $1
+}
+
 if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
     if [[ "${1:-}" == "--help" ]]; then
         usage
     else
-        echo "Linting Scala source code and executing tests"
+        success "Linting Scala source code and executing tests"
+
         ./sbt ";\
             scalafix --check; \
             scalafmtCheck; \

I would also consider whether the value the color codes add to the script outweighs what they subtract in terms of complexity and decreased readability. If it is something we want to standardize on, which is totally cool, it may be worth extending the STRTA ADR so we can get every repository on the same page going forward.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I like your change -- in general I find the colors to be helpful in parsing the output of our STRTA commands (what's the command vs what's log output from processes the command invokes to be helpful). As to whether the increase in complexity for STRTA outweighs its usefulness - I would say that once written our STRTA commands aren't edited that much, but we run them often so it seems like a fair tradeoff.

Nevertheless, I don't think this change should hold up the PR since it's more a matter of preference and the original edit didn't seem related to the goal of publishing artifacts.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in general I find the colors to be helpful in parsing the output of our STRTA commands (what's the command vs what's log output from processes the command invokes to be helpful).

Agreed. It's definitely helpful.

As to whether the increase in complexity for STRTA outweighs its usefulness - I would say that once written our STRTA commands aren't edited that much, but we run them often so it seems like a fair tradeoff.

The thing in my mind is, if we replicate a success or log function across all the STRTA, there's more to parse and possibly distract when reading each file. Embedding the color codes in the printf statement introduces a different set of readability concerns. I agree with your perspective though, if we standardize on this, it might eventually become background noise.

At a higher level, it may be worth revisiting the STRTA ADR, because many projects have been deviating from the standards and that document should be updated for how we're developing apps today (Vagrant seems to be disused, etc).

./sbt ";\
scalafix --check; \
scalafmtCheck; \
scalafmtSbtCheck; \
scapegoat; \
undeclaredCompileDependenciesTest; \
unusedCompileDependenciesTest; \
test \
"
fi
fi