Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DaskManager can't handle list[Future] #1

Open
cobra91 opened this issue Sep 25, 2024 · 2 comments
Open

DaskManager can't handle list[Future] #1

cobra91 opened this issue Sep 25, 2024 · 2 comments

Comments

@cobra91
Copy link

cobra91 commented Sep 25, 2024

if we put this DaskManager().compute(res_compare) where res_compare is a append list of task that can't work !

@ahirner
Copy link
Member

ahirner commented Sep 25, 2024

If the list to append to isn't thread save or a distributed structure, surely. Did you see that in the repo somewhere?

@cobra91
Copy link
Author

cobra91 commented Sep 25, 2024

`class DaskManager(metaclass=Singleton):
def init(self):
self.client = Client(f'{settings.DASK_SCHEDULER_HOST}:{settings.DASK_SCHEDULER_PORT}')

def compute(self, graph):
    future = self.client.compute(graph)
    future.add_done_callback(self.task_complete)
    dask_task = DaskTask.objects.create(task_key=future.key)
    return dask_task`

Here compute work but crash on call to add_done_callback because future can be a list[Future]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants