feat(spark): add support for more types and literals (binary, list, map, intervals, timestamps) #828
pr.yml
on: pull_request
Build and Test Java
2m 2s
cyclonedx-sbom
1m 28s
Check editorconfig
4s
Lint commits for semantic-release
9s
Security validation
4s
Build Examples
2m 4s
Dry-run release
16s
Matrix: Build Isthmus Native Image
Matrix: osv-scanner
Annotations
2 errors and 1 warning
Build and Test Java:
task ':spark:spotlessScalaCheck'#L1
Execution failed for task ':spark:spotlessScalaCheck':
The following files had format violations:
src/main/scala/io/substrait/spark/expression/ToSparkExpression.scala
@@ -46,7 +46,7 @@
······Literal.TrueLiteral
····}·else·{
······Literal.FalseLiteral
-··}
+····}
··}
··override·def·visit(expr:·SExpression.I32Literal):·Expression·=·{
····Literal(expr.value(),·ToSubstraitType.convert(expr.getType))
@@ -91,22 +91,27 @@
··}
··override·def·visit(expr:·SExpression.PrecisionTimestampLiteral):·Literal·=·{
-····Literal(Util.toMicroseconds(expr.value(),·expr.precision()),·ToSubstraitType.convert(expr.getType))
+····Literal(
+······Util.toMicroseconds(expr.value(),·expr.precision()),
+······ToSubstraitType.convert(expr.getType))
··}
··override·def·visit(expr:·SExpression.PrecisionTimestampTZLiteral):·Literal·=·{
-····Literal(Util.toMicroseconds(expr.value(),·expr.precision()),·ToSubstraitType.convert(expr.getType))
+····Literal(
+······Util.toMicroseconds(expr.value(),·expr.precision()),
+······ToSubstraitType.convert(expr.getType))
··}
··override·def·visit(expr:·SExpression.IntervalDayLiteral):·Literal·=·{
-····val·micros·=·(expr.days()·*·SECONDS_PER_DAY·+·expr.seconds())·*·Util.MICROSECOND_PRECISION·+·Util
-······.toMicroseconds(expr.subseconds(),·expr.precision())
+····val·micros·=
+······(expr.days()·*·SECONDS_PER_DAY·+·expr.seconds())·*·Util.MICROSECOND_PRECISION·+·Util
+········.toMicroseconds(expr.subseconds(),·expr.precision())
····Literal(micros,·ToSubstraitType.convert(expr.getType))
··}
··override·def·visit(expr:·SExpression.IntervalYearLiteral):·Literal·=·{
-····val·months·=expr.years()·*·12·+·expr.months()
+····val·months·=·expr.years()·*·12·+·expr.months()
····Literal(months,·ToSubstraitType.convert(expr.getType))
··}
@@ -120,7 +125,12 @@
··}
··override·def·visit(expr:·SExpression.MapLiteral):·Literal·=·{
-····val·map·=·expr.values().asScala.map·{·case·(key,·value)·=>·(key.accept(this).asInstanceOf[Literal].value,·value.accept(this).asInstanceOf[Literal].value)·}
+····val·map·=·expr.values().asScala.map·{
+······case·(key,·value)·=>
... (7 more lines that didn't fit)
Violations also present in:
src/main/scala/io/substrait/spark/expression/ToSubstraitLiteral.scala
src/test/scala/io/substrait/spark/TypesAndLiteralsSuite.scala
Run './gradlew :spark:spotlessApply' to fix these violations.
at com.diffplug.gradle.spotless.SpotlessCheck.performAction(SpotlessCheck.java:111)
at com.diffplug.gradle.spotless.SpotlessCheck.performAction(SpotlessCheck.java:53)
at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:125)
at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:58)
at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:51)
at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:29)
at org.gradle.api.internal.tasks.execution.TaskExecution$3.run(TaskExecution.java:244)
at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
at org.gradle.internal.operations.DefaultBuildOperationR
|
Build and Test Java
Process completed with exit code 1.
|
Check editorconfig
The following actions use a deprecated Node.js version and will be forced to run on node20: editorconfig-checker/action-editorconfig-checker@v2. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
|
Artifacts
Produced during runtime
Name | Size | |
---|---|---|
cyclonedx-sbom
Expired
|
66.3 KB |
|