Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

source-mixpanel-native: increase funnels date window size #2440

Merged
merged 1 commit into from
Feb 24, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions source-mixpanel-native/source_mixpanel_native/streams/funnels.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,23 @@ def state_checkpoint_interval(self) -> int:
# we'll have to emit state every 15 records
return 1000

def __init__(
self,
date_window_size: int = 30,
**kwargs,
):
# This stream checks from the start_date to the end_date for each funnel. Typically,
# there's KB of data between the start_date & end_date for each funnel, and using
# small date windows causes the connector to make multiple requests for each funnel.
# Since we should be able to fetch all data for a funnel in a single request,
# we use an extremely large data window to force the connector to make a single request
# for all of a funnel's records between the start & end dates.

super().__init__(
date_window_size = 100_000,
**kwargs,
)

def path(self, **kwargs) -> str:
return "funnels"

Expand Down
Loading