Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sqlparser: support expression type hint #482

Merged
merged 3 commits into from
Feb 9, 2024
Merged

Conversation

Yaiba
Copy link
Contributor

@Yaiba Yaiba commented Feb 2, 2024

Support type hint in sql expression, so that could be used to cast/infer type of an expression.

Description

  • change sql-grammar ANTLR lexer & parser rules to support type hint
  • change parse.sql.tree.Expression to support type hint
  • changes sqlparser to populate type hint
  • related tests

Related Issue

Following prs should be merged first, then update go.mod accordingly.

Motivation and Context

How Has This Been Tested?

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update (including inline docs)
  • Other (please describe):

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Checklist Explanation:

How to Review this PR:

sql syntax changes are in kwilteam/sql-grammar#3

changes made to sqlparser are mostly just set TypeHint to certain tree.ExpressionXX

Additional Information:

'type hint' works as below:

  • it can directly be applied to literal, bind parameter, column, and function expression
  • case expression does not have type hint
  • other expressions can have type hint only when wrapped

This logic is also enforced in the sqlparser.

TODO:

  • use predefined TypeHint type?
  • define supported type for TypeHint value

@Yaiba Yaiba force-pushed the feature/sql-expr-type-hint branch from 336aaf6 to df96076 Compare February 2, 2024 22:51
@@ -38,6 +44,11 @@ func (s *SqlWriter) String() string {
s.stmt.WriteString(")")
}

if s.typeHint != "" {
s.stmt.WriteString("::")
s.stmt.WriteString(s.typeHint)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't work, because our types aren't named the same as Postgres's. Also, I don't think the writer package is the right place to do this. It seems like it should individually be written in each expression's ToSQL().

Copy link
Contributor Author

@Yaiba Yaiba Feb 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the writer package is the right place to do this

Yeah you're right, since only expression has typeHint.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because our types aren't named the same as Postgres's

For now I have no checks on typeHint, so you can pass Postgres's type

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, shouldn't we expect Kwil schema types and translate them to PG types when writing the statement out from the AST (or when generating the AST from the user code)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, but we only have int and string, which is pretty constraining...

Copy link
Contributor Author

@Yaiba Yaiba Feb 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah we need a supported type list, then translate it when ToSQL.
I think it's time to extend our supported column types?

Copy link
Member

@jchappelow jchappelow Feb 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure. Safest may be to use just the two Kwil types, and then we map those to either TEXT or INT8 in the type cast.

Seems to me that the main purpose of this (at present anyway) is to allow the author to assert type to disambiguate between integer or string, also to reinterpret type as needed for downstream instructions within an action. Maybe we add the different integer types later, with boolean and possibly our own 32-byte integer type when we are ready and have thought through the implications.

@Yaiba Yaiba force-pushed the feature/sql-expr-type-hint branch from df96076 to 39af5b0 Compare February 5, 2024 17:27
@Yaiba Yaiba force-pushed the feature/sql-expr-type-hint branch from 39af5b0 to 5c3b092 Compare February 9, 2024 17:24
@Yaiba Yaiba marked this pull request as ready for review February 9, 2024 17:26
@Yaiba Yaiba changed the title [WIP] sqlparser: support expression type hint sqlparser: support expression type hint Feb 9, 2024
@Yaiba Yaiba requested review from brennanjl and jchappelow February 9, 2024 17:40
@brennanjl brennanjl merged commit c7f33fb into main Feb 9, 2024
2 checks passed
@brennanjl brennanjl deleted the feature/sql-expr-type-hint branch February 9, 2024 17:50
@@ -93,7 +93,7 @@ require (
github.com/jmhodges/levigo v1.0.0 // indirect
github.com/klauspost/compress v1.17.0 // indirect
github.com/kwilteam/action-grammar-go v0.0.1-0.20230926160920-472768e1186c // indirect
github.com/kwilteam/sql-grammar-go v0.0.3-0.20230925230724-00685e1bac32 // indirect
github.com/kwilteam/sql-grammar-go v0.0.3-0.20240202213606-3f118696cec5 // indirect
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since 3f118696cec5 isn't on main of that repo, we should get kwilteam/sql-grammar#3 and then kwilteam/sql-grammar-go#3 before this

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, theses refs will break when those PRs merge and the feature branches are deleted

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, are you on this?

brennanjl pushed a commit that referenced this pull request Feb 26, 2024
* sqlparser: support expression type hint

* handle typeHint only in Expression

* rename to TypeCast, with predifiend types
brennanjl pushed a commit that referenced this pull request Feb 26, 2024
* sqlparser: support expression type hint

* handle typeHint only in Expression

* rename to TypeCast, with predifiend types
jchappelow pushed a commit that referenced this pull request Feb 26, 2024
* sqlparser: support expression type hint

* handle typeHint only in Expression

* rename to TypeCast, with predifiend types
brennanjl pushed a commit that referenced this pull request Feb 26, 2024
* sqlparser: support expression type hint

* handle typeHint only in Expression

* rename to TypeCast, with predifiend types
brennanjl pushed a commit that referenced this pull request Feb 26, 2024
* sqlparser: support expression type hint

* handle typeHint only in Expression

* rename to TypeCast, with predifiend types
brennanjl pushed a commit that referenced this pull request Feb 26, 2024
* sqlparser: support expression type hint

* handle typeHint only in Expression

* rename to TypeCast, with predifiend types
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants