Skip to content

feat(spark): add support for more types and literals (binary, list, map, intervals, timestamps) #828

feat(spark): add support for more types and literals (binary, list, map, intervals, timestamps)

feat(spark): add support for more types and literals (binary, list, map, intervals, timestamps) #828

Triggered via pull request October 24, 2024 12:44
Status Failure
Total duration 2m 15s
Artifacts 1

pr.yml

on: pull_request
Build and Test Java
2m 2s
Build and Test Java
Check editorconfig
4s
Check editorconfig
Lint commits for semantic-release
9s
Lint commits for semantic-release
Security validation
4s
Security validation
Build Examples
2m 4s
Build Examples
Dry-run release
16s
Dry-run release
Matrix: Build Isthmus Native Image
Matrix: osv-scanner
Fit to window
Zoom out
Zoom in

Annotations

2 errors and 1 warning
Build and Test Java: task ':spark:spotlessScalaCheck'#L1
Execution failed for task ':spark:spotlessScalaCheck': The following files had format violations: src/main/scala/io/substrait/spark/expression/ToSparkExpression.scala @@ -46,7 +46,7 @@ ······Literal.TrueLiteral ····}·else·{ ······Literal.FalseLiteral -··} +····} ··} ··override·def·visit(expr:·SExpression.I32Literal):·Expression·=·{ ····Literal(expr.value(),·ToSubstraitType.convert(expr.getType)) @@ -91,22 +91,27 @@ ··} ··override·def·visit(expr:·SExpression.PrecisionTimestampLiteral):·Literal·=·{ -····Literal(Util.toMicroseconds(expr.value(),·expr.precision()),·ToSubstraitType.convert(expr.getType)) +····Literal( +······Util.toMicroseconds(expr.value(),·expr.precision()), +······ToSubstraitType.convert(expr.getType)) ··} ··override·def·visit(expr:·SExpression.PrecisionTimestampTZLiteral):·Literal·=·{ -····Literal(Util.toMicroseconds(expr.value(),·expr.precision()),·ToSubstraitType.convert(expr.getType)) +····Literal( +······Util.toMicroseconds(expr.value(),·expr.precision()), +······ToSubstraitType.convert(expr.getType)) ··} ··override·def·visit(expr:·SExpression.IntervalDayLiteral):·Literal·=·{ -····val·micros·=·(expr.days()·*·SECONDS_PER_DAY·+·expr.seconds())·*·Util.MICROSECOND_PRECISION·+·Util -······.toMicroseconds(expr.subseconds(),·expr.precision()) +····val·micros·= +······(expr.days()·*·SECONDS_PER_DAY·+·expr.seconds())·*·Util.MICROSECOND_PRECISION·+·Util +········.toMicroseconds(expr.subseconds(),·expr.precision()) ····Literal(micros,·ToSubstraitType.convert(expr.getType)) ··} ··override·def·visit(expr:·SExpression.IntervalYearLiteral):·Literal·=·{ -····val·months·=expr.years()·*·12·+·expr.months() +····val·months·=·expr.years()·*·12·+·expr.months() ····Literal(months,·ToSubstraitType.convert(expr.getType)) ··} @@ -120,7 +125,12 @@ ··} ··override·def·visit(expr:·SExpression.MapLiteral):·Literal·=·{ -····val·map·=·expr.values().asScala.map·{·case·(key,·value)·=>·(key.accept(this).asInstanceOf[Literal].value,·value.accept(this).asInstanceOf[Literal].value)·} +····val·map·=·expr.values().asScala.map·{ +······case·(key,·value)·=> ... (7 more lines that didn't fit) Violations also present in: src/main/scala/io/substrait/spark/expression/ToSubstraitLiteral.scala src/test/scala/io/substrait/spark/TypesAndLiteralsSuite.scala Run './gradlew :spark:spotlessApply' to fix these violations. at com.diffplug.gradle.spotless.SpotlessCheck.performAction(SpotlessCheck.java:111) at com.diffplug.gradle.spotless.SpotlessCheck.performAction(SpotlessCheck.java:53) at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:125) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:58) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:51) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:29) at org.gradle.api.internal.tasks.execution.TaskExecution$3.run(TaskExecution.java:244) at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29) at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66) at org.gradle.internal.operations.DefaultBuildOperationR
Build and Test Java
Process completed with exit code 1.
Check editorconfig
The following actions use a deprecated Node.js version and will be forced to run on node20: editorconfig-checker/action-editorconfig-checker@v2. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/

Artifacts

Produced during runtime
Name Size
cyclonedx-sbom Expired
66.3 KB