Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support pandas.DataFrame in cache.memoize() #314

Open
LudwigAJ opened this issue Apr 17, 2024 · 1 comment
Open

support pandas.DataFrame in cache.memoize() #314

LudwigAJ opened this issue Apr 17, 2024 · 1 comment

Comments

@LudwigAJ
Copy link

LudwigAJ commented Apr 17, 2024

Since being one of the few choices available in Python's Dash library, and since there is heavy usage of pandas.DataFrames in general, could the .memoize() functions introduce support for these?

As of now I believe the functions simply tries to hash the DataFrame object and not its contents. Which doesn't guarantee the same hash for the same frame (data-wise).

pandas has the following function: pandas.util.hash_pandas_object which could be used to hash the contents.

The user could then specify which input parameters/arguments of decorated functions are DataFrames via an additional frames parameter.

It could work similarly to the ignore parameter. i.e. something like the following.

@cache.memoize(frames={0, 'myDF'})
def someFunc(myDF, someDate, someString):
    # do some operation(s)
    return someResult
@gabrielgrant
Copy link

given how commonly Pandas DFs are used, seems it would make sense to special-case this inside diskcache and not make the user manually specify the dataframe inputs

Alternatively/additionally is there a way that custom hashers/serializers could be used? (haven't seen any obvious support for this today in the docs, but maybe I missed something, or maybe it's something that could be added?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants