Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelize account and publisher sync #43

Merged
merged 6 commits into from
Aug 15, 2024
Merged

Parallelize account and publisher sync #43

Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
9efc689
Parallelize account and publisher sync
mcamou Aug 14, 2024
ee1beff
lint
mcamou Aug 14, 2024
16c64c6
Try to get concurrency working
mcamou Aug 14, 2024
d8aa997
lintg
mcamou Aug 14, 2024
b369694
lint
mcamou Aug 14, 2024
760ab22
limit concurrency to 50 parallel txs
ayazabbas Aug 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 22 additions & 2 deletions program_admin/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import asyncio
import json
import os
from pathlib import Path
Expand Down Expand Up @@ -260,6 +261,8 @@ async def sync(

# Sync product/price accounts

transactions: List[asyncio.Task[None]] = []

product_updates: bool = False

for jump_symbol, _price_account_map in ref_permissions.items():
Expand All @@ -278,12 +281,23 @@ async def sync(

instructions.extend(product_instructions)
if send_transactions:
await self.send_transaction(product_instructions, product_keypairs)
transactions.append(
asyncio.create_task(
self.send_transaction(
product_instructions, product_keypairs
)
)
)

await asyncio.gather(*transactions)

if product_updates:
await self.refresh_program_accounts()

# Sync publishers

transactions = []
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can define price_feed_transactions and publisher_transactions as two separate lists instead of reusing the same object for the sake of safety.


for jump_symbol, _price_account_map in ref_permissions.items():
ref_product = ref_products[jump_symbol] # type: ignore

Expand All @@ -297,7 +311,13 @@ async def sync(
if price_instructions:
instructions.extend(price_instructions)
if send_transactions:
await self.send_transaction(price_instructions, price_keypairs)
transactions.append(
asyncio.create_task(
self.send_transaction(price_instructions, price_keypairs)
)
)

await asyncio.gather(*transactions)

return instructions

Expand Down
Loading