Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates for computation on 3h frequency #46

Open
wants to merge 37 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
2f71ca2
Update _preprocessing.py
emsonali Mar 11, 2024
085738f
Merge branch 'climate-service-center:main' into master
emsonali Mar 11, 2024
91a9105
Update _preprocessing.py
emsonali Mar 11, 2024
7df4871
Update convert_to_frequency.json
emsonali Mar 11, 2024
43bc610
Update convert_to_frequency.json
emsonali Mar 11, 2024
ad155f3
Update convert_to_frequency.json
emsonali Mar 12, 2024
89afea8
Update _preprocessing.py
indraneelgk Mar 12, 2024
63ecb6c
Update _preprocessing.py
indraneelgk Mar 12, 2024
a0646d6
some typos fixed.
indraneelgk Mar 12, 2024
39befc2
Update _preprocessing.py
emsonali Mar 12, 2024
47a35e9
Update index_calculator.py
indraneelgk Mar 12, 2024
91afaaf
Update _preprocessing.py
emsonali Mar 12, 2024
9d0abb2
disable anything related to time frequency in preprocessing
indraneelgk Mar 12, 2024
bd55c09
Update _preprocessing.py
emsonali Mar 12, 2024
0a54e78
preprocessing with time-related things.
indraneelgk Mar 12, 2024
db4a9ee
Update _preprocessing.py
indraneelgk Mar 12, 2024
d9cfa58
remove time bounds stuff from processing.
indraneelgk Mar 12, 2024
e595325
postprocessing split False
indraneelgk Mar 12, 2024
89a9410
postprocessing split False
indraneelgk Mar 12, 2024
74f17c1
Merge branch 'master' of https://github.com/emsonali/index_calculator
indraneelgk Mar 12, 2024
d60dd87
Revert "Merge branch 'master' of https://github.com/emsonali/index_ca…
indraneelgk Mar 12, 2024
c2bd461
Revert "postprocessing split False"
indraneelgk Mar 12, 2024
abdca40
Revert "postprocessing split False"
indraneelgk Mar 12, 2024
1e665ef
split force false
indraneelgk Mar 12, 2024
627025c
Keep split technique intact.
indraneelgk Mar 12, 2024
2fe9fda
removed processing commenting.
indraneelgk Mar 12, 2024
96f5e8c
processing.py we remove the connection to frequency.
indraneelgk Mar 12, 2024
0f4ccd6
Update _consts.py
emsonali Mar 12, 2024
ac9928e
Update convert_to_frequency.json
emsonali Mar 12, 2024
92df557
Update _consts.py
emsonali Mar 12, 2024
80f0c84
Update _preprocessing.py
emsonali Mar 12, 2024
09a99df
Update _processing.py
emsonali Mar 12, 2024
a07ef32
Update _processing.py
indraneelgk Mar 12, 2024
4378a28
Update _processing.py
emsonali Mar 12, 2024
83e2ec2
Update _processing.py
emsonali Mar 12, 2024
8b457a1
Update _preprocessing.py
emsonali Mar 13, 2024
7696167
Update _consts.py
emsonali Mar 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions index_calculator/_consts.py
Original file line number Diff line number Diff line change
@@ -23,6 +23,7 @@

_split = {
"fx": False,
"3hr": "5A",
"day": "5A",
"week": "10A",
"mon": "10A",
1 change: 1 addition & 0 deletions index_calculator/_preprocessing.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

convert_time_axis is set but never used again.

Original file line number Diff line number Diff line change
@@ -62,6 +62,7 @@ def __init__(
time_range=None,
crop_time_axis=True,
check_time_axis=True,
convert_time_axis=True,
**kwargs,
):
if ds is None:
13 changes: 6 additions & 7 deletions index_calculator/_processing.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please uncomment these lines. We want to keep adding time bounds. Is it not working with your data?

Original file line number Diff line number Diff line change
@@ -213,13 +213,12 @@ def _processing(self):
time_encoding = ds.time.encoding
time_encoding["dtype"] = np.float64
idx_ds.time.encoding = time_encoding
idx_ds = (
pyh.time_control(idx_ds)
.add_time_bounds(
frequency=self.freq,
)
.ds
)
#idx_ds = (
# pyh.time_control(idx_ds)
# .add_time_bounds(
# frequency=self.freq,
# ).ds
# )
self.unlimited_dims = "time"
else:
for dim in idx_ds[self.CIname].dims:
4 changes: 4 additions & 0 deletions index_calculator/index_calculator.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think some print statements could be useful. Maybe we should use logging instead of print and write those statements directly in the submodules _preprocessing.py, _processing.py and _preprocessing.py

Original file line number Diff line number Diff line change
@@ -61,11 +61,15 @@ def __init__(self, write=False, **kwargs):
def _compute(self, write=False, **kwargs):
"""Compute climate index."""
preproc_obj = preprocessing(**kwargs)
print('preprocessing done.')
proc_obj = processing(preproc_obj=preproc_obj)
print('processing done.')
postproc_obj = postprocessing(proc_obj=proc_obj)
print('postprocessing done.')
if write is True:
outputwriter(
postproc_obj=postproc_obj,
**kwargs,
)
print('writing done.')
return postproc_obj
22 changes: 21 additions & 1 deletion index_calculator/tables/convert_to_frequency.json
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, do you use hourly or 3-hourly input data?

Original file line number Diff line number Diff line change
@@ -18,5 +18,25 @@
"vas": "mean"
},
"freq": "D"
}
},
"3hr": {
"var": {
"hurs": "mean",
"pr": "mean",
"prsn": "mean",
"rlds": "mean",
"rlus": "mean",
"rsds": "mean",
"rsus": "mean",
"sfcWind": "mean",
"sfcWindmax": "max",
"snd": "mean",
"tas": "mean",
"tasmax": "max",
"tasmin": "min",
"uas": "mean",
"vas": "mean"
},
"freq": "3H"
}
}
Loading