Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: federate expression subqueries #33

Closed
wants to merge 8 commits into from
Closed

feat: federate expression subqueries #33

wants to merge 8 commits into from

Conversation

sardination
Copy link
Contributor

@sardination sardination commented May 3, 2024

Closes #31

Recursively optimize subqueries in expressions as well.

@sardination sardination changed the title Federate expression subqueries feat: federate expression subqueries May 3, 2024
@backkem backkem self-requested a review May 4, 2024 10:51
@backkem
Copy link
Collaborator

backkem commented May 4, 2024

Hi @sardination, thank you for having a look at this.

optimize_recursively recursively walks over the plan and tries to find the largest possible sub-plans that can be federated by one remote. The PR as it stands now, adds the subqueries to the recursion but, as your TODO comment mentions, it doesn't incorporate it in the logic to determine the largest possible sub-plans yet.

To make this work, I was thinking in the direction of building a custom stack to hold the inputs at a specific plan depth, similar to ExprIdentifierVisitor. That way the entire function can be re-written as a TreeNodeVisitor (which exists for expressions while the .inputs() method doesn't). I need to find more time to complete this though.
Update: Seems like a dead-end for now. Thinking up something else..

@backkem
Copy link
Collaborator

backkem commented May 10, 2024

I noticed the ScalarSubquerys aren't supported by the physical planner. They are replaced in the scalar_subquery_to_join optimiser. I guess this means we need to re-write the analyzer to an optimizer and schedule it after the scalar_subquery_to_join one.

@backkem
Copy link
Collaborator

backkem commented May 22, 2024

Continued in #37.

@backkem backkem closed this May 22, 2024
@philippemnoel philippemnoel deleted the rule-expressions branch May 27, 2024 20:53
@philippemnoel
Copy link

Thank you for picking this up @backkem!

phillipleblanc pushed a commit that referenced this pull request Jan 10, 2025
…e name (#33)

* Rewrite shouldn't be performed on a column name same as the table name

* fix comment

* Add unit test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support table scans in expressions (subquery)
3 participants