Project: QPE exercise in the North of Italy

[jfigui]

Discussion of all topics related to the QPE exercise.

[jfigui]

@openradar/erad2024-team @wolfidan ,

I opened this topic to discuss the QPE exercise. I understood from our meeting on the 5th of August that the topic will be driven by Pyrad but other software packages can contribute with notebooks as post-course exercises.

Could we maybe start by identifying the contributors to this project? Please add your name and contribution as a reply to this comment:

• @wolfidan , @jfigui : Development of a data processing chain with Pyrad config files. (main material during the course)

[jfigui]

@openradar/erad2024-team @wolfidan Let me know if this brief description of the project suits you:

In this project we will learn how to setup a data processing chain for radar rainfall rate retrieval. The project will show the main steps involved between reading raw polarimetric moments to comparing rainfall rate retrievals with rain gauges, including clutter filtering, PhiDP estimation, KDP retrieval, etc. During the course, the processing chain implementation will be performed with Pyrad. Examples of how to implement a processing chain using other available software packages will be provided as post-course material

[kmuehlbauer]

@jfigui @wolfidan Can you please give some status update? Are you about to have a PR ready for the notebook?

[wolfidan]

@kmuehlbauer ,

Sorry for the delay, I had quite a bit of cleaning to do in pyrad and Jordi is in vacation this week. I am on it, I should be able to have a PR this week hopefully.

[kmuehlbauer]

Thanks @wolfidan, great to hear it's progressing. 👍

[wolfidan]

I want to do the PR, but have an issue. How should I manage it with the data? The thing is that the QPE exercice processes a few hours of data (otherwise it does not make much sense). So if I put the whole dataset on git, it's 2.2 Gb...How should I proceed? We don't have a way in pyrad at the moment to download data from an S3 (only to upload the output products). A possibility would be to have the dataset as an archive on a S3 and download it with an additional script during creation of the binder?

[kmuehlbauer]

@wolfidan Which data are we talking about? This data: 20240522_MeteoSwiss_ARPA_Lombardia? This is already available in the pythia s3 bucket. This can be accessed by fsspec using local scheme. So should work for pyrad.

If you talking about another dataset, please link the data here so that @mgrover1 can add it to the pythia s3 bucket.

[wolfidan]

Hi @kmuehlbauer , no it's the same dataset. I'm not familiar with fsspec, so I will have to look into it. But if this dataset is available at runtime on the local system, it's sufficient for us.
@mgrover1 : did you update the dataset with the latest version I sent with dealiased products? I'm asking because otherwise I will have to dealias the data in the pyrad routine

[kmuehlbauer]

@wolfidan Please see https://openradarscience.org/erad2024/intro-data-access for a short writeup.

[aladinor]

He @wolfidan and @jfigui, as I mentioned in #94, we have created an Analysis-ready dataset. We can use it for QPE and QVPs. This is the workflow I suggest with the Colombian radars.

Let me know if we can work together on QPE.

[jfigui]

@aladinor ,

We initially planned for a fully pyrad QPE exercise. What do you think if we let you some minutes to introduce your workflow at the beginning of the exercise as an add on?

[aladinor]

Hi @jfigui,

Thank you for the suggestion. The QPE I’ve planned for the Analysis-Ready sections primarily demonstrates the use of a theoretical Z-R relationship. My goal is to showcase how efficiently we can leverage the ARCO dataset for certain operations, rather than diving deep into advanced QPE methodologies.

I understand that this approach might be simpler compared to the more comprehensive PyRad exercise you’re planning. If you'd like, we could explore how to integrate these approaches or adjust the scope to better align with your plans. I’m happy to discuss further, and we can also review the QPE exercise together to find the best way forward.

[kmuehlbauer]

Given the time left until course this seems complete,too.