RealPEP follows the hypothesis, that Quantitative Precipitation Estimation (QPE), Quantitative Precipitation Nowcasting (QPN) and Quantitative Precipitation Forecasting (QPF) by Numerical Weather Prediction (NWP) can be significantly improved by exploiting the full observational state evolution information of the precipitation-generating atmosphere, which will be provided by this central service project. Besides polarimetric radar observations of the DWD network and NWP analyses and predictions, C1 will integrate satellite-based near- and thermal infrared observations, GNSS-derived total column water vapor fields and lightning data. In cooperation with P2, advection and convergence of total water vapor fields will enrich observation-based nowcasting with indications of convection initiation and intensifying precipitation.

The data collection and exploitation platform will employ the DWD C++ processing platform POLARA, which includes already several standard radar processing and analysis tools suitable for creating initial data for P1 to P4, and which will be expanded during the course of RealPEP. C1 will initiate and guide the updating of POLARA with developments by RealPEP and from other sources, and its integration into the RealPEP process chain from QPE to flood prediction.

Contribution of University of Bonn

(a) Near-Realtime Quantiative Precipitation Estimation

In order to prepare RealPEP developments for operational applications with the national DWD radar network, improvements in the processing of polarimetric radar data, precipitation and micro-physical retrievals have been implemented in the DWD C++ processing platform POLARA. More precisely, C1 extended the POLARA quality assurance (QA) and implemented the sophisticated hybrid polarimetric rainfall retrievals (R(Ah, KDP), R(Av, KDP), and R(Zh, KDP)) based on specific attenuation in the horizontal and vertical channel (Ah/Av), specific differential phase KDP and horizontal reflectivity Zh following Chen et al. (2021) and Chen et al. (2023). This also requires and includes an automated technique for melting layer detection based on temperature data, removal of second trip echoes exploiting Zh, the cross correlation coefficient RhoHV and the radar distance, and a compositing algorithm that considers the minimum sea level distance is implemented. Some radars showed occasionally sign flips in the measurements of total differential phase $\Phi_{DP}$, which is now automatically detected and resolved. Furthermore, the processing chain has been optimized in terms of computational complexity and code efficiency, especially for KDP calculations and smoothing routines. Additionally, polarimetric retrieval algorithms for liquid water content (Reimann et al., 2021) and ice water content (Ryzhkov and Zrnić., 2019; Bukovčić et al., 2020) are implemented. Thus, C1 enabled POLARA to execute the entire radar data processing chain for near real-time national rainfall composites (Quality Assurance → QPE → Composite; see Fig. 1).


Figure 1: The implemented POLARA workflow for polarimetric rainfall composites (e.g. R(Ah, KDP), R(Av, KDP), and R(Zh, KDP)) based on the 16 operational C-band radars of DWD. The right panel shows an example composite for July 25, 2017.

For a statistical profound evaluation of the implemented RealPEP algorithms, the POLARA processing and precipitation algorithm chain has been applied to the months of May 2019, June 2020, and July 2021.

The performance of POLARA's precipitation composites have been evaluated with rain gauges and opposed to the performance of the RADOLAN RY product (Fig. 2). Overall, POLARA's precipitation retrieval outperforms the RY product. Validation scores show root mean square error (RMSE), normalized mean bias (NMB), and correlation coefficient (CC) for both products. The most striking feature is that the overestimation in RY is rectified in POLARA, as reflected in the NMB score. Note that RADOLAN RY did undergo continuous improvements in recent years as well and is also exploiting polarimetry for clutter removal and attenuation correction.


Figure 2: Evaluation of POLARAs R(Ah, KDP) (left) and RADOLAN RY (right) precipitation composites with rain gauges, respectively. Each point represents the daily accumulation of rainfall over a rain gauge. Colorbar indicate the sample size.

(b) Improving Quantitative Precipitation Nowcasting With Machine Learning

Project C1 also aims to oppose the performance of machine learning nowcasting algorithms to the advanced localized Spectral Prognosis (SPROG) method, i.e. SPROG-LOC (Reinoso-Rondinel et al., 2022) and assess the possibility to. First, C1 exploited a predictive recurrent neural network (PredRNN; Wang et al., 2022) with the three months POLARA R(Ah, KDP) composites. To manage the data efficiently, the 1200×1100 data with 1 km spatial resolution is cropped and resized to 280×220 with 4 km horizontal resolution. The PredRNN model is trained using 33 days of moderate and heavy rainfall data and in the ensuing step evaluated using seven days of data and compared with SPROG-LOC for two hours leadtime. PredRNN performs better than SPROG -LOC in terms of the structural similarity index measurement (SSIM), peak signal-to-noise ratio (PSNR), critical success index (CSI), probability of detection (POD), and false alarm ratio (FAR). Regarding the fraction skill score (FSS), PredRNN performs better in moderate rain (Rth = 1.0 mm/h) for all length scales (8 km to 80 km) and in heavy rain (Rth = 5.0 mm/h) only for short length scales, e.g. 8 km (Fig. 3). Moreover, PredRNN produces blurry nowcasts at longer lead times.

Figure 3: The Fraction Skill Score (FSS) for PredRNN (blue) and SPROG-LOC (red) with correction factor, I=10 % for 2 hours of lead time, precipitation thresholds Rth of 1 mm/h (left) and 5 mm/h (right) and length scales of 8km, 40km and 80 km, with thinner to thicker legend respectively.

To address the underestimation of heavy rain prediction and blurry nowcasts, RealPEP is now adapting a deep generative model of radar (DGMR; Ravuri et al., 2021). The model claims to eliminate the blurry effect of nowcasts and to improve rain prediction using statistical, economic, and cognitive measures.

The fusion of satellite-based information on the pre-convective environment with radar-based nowcasting will be applied to the best-performing model between PredRNN and DGMR to further extend the lead time. Therefore, C1 partner Free University of Berlin provided already the satellite-based cloud type (CT) and MSG-SEVIRI total column water vapor (TCWV) data for 53 rainy summer days.

Contribution of University of Bonn

Figure C1-1: Flowchart for Benchmark generation with 4 months data set, RealPEP-meeting 5-6 Dec'2022.

At University of Bonn a first installation of the POLARA C++ processing framework has been set up on a dedicated server. After joining a POLARA workshop, the project scientist dives into the overall structure of the complex framework and into the details of a first set of routines for data quality, data processing (e.g. for the processing of differential phase shift DP), and compositing. Figure C1-1 shows examples of two compositing algorithms.

In May 2022, a new project scientist took over the position at Uni Bonn and installed an updated version of the POLARA C++ processing framework. The project scientist dived into the overall structure of the POLARA framework and the so-called POLARA-RECALC system. The latter allows parallel processing of multiple jobs by splitting the time period and processes and archives the output data. The POLARA-RECALC system is configured to process multiple days/months of data and can utilize generated output data files from one job into another job during runtime.

In a joint effort, the RealPEP research group aims at a benchmark data set for in-depth evaluation of RealPEP developments along the process chain from QPE to flash flood prediction based on a 4 months data set. Thus, algorithms inherent in POLARA have been extended and also complemented by algorithms developed in projects P1 and P2 (see Fig. C1-1). However, POLARA's Quality Assurance (QA) job is not completely suitable for the Quantitative Precipitation Estimation (QPE) algorithm to be implemented. Therefore, the QA job has been extended for the vertical channels (e.g. attenuation correction of vertical reflectivity ZV) and the processing of several intermediate products/ variables like reflectivity ZH/ZV, differential reflectivity ZDR and corrected differential phase ΦDP from the QA job (see Fig. C1-2) is now fed into the RealPEP job to implement the new QPE algorithm using specific differential phase KDP and specific attenuation A,R(KDP)/R(A), developed in P1. E.g., a different method to estimate the attenuation parameter α is applied. Also, the corrected ΦDP from QA job is further processed in RealPEP job (e.g. including the removal of isolated points, subtracting the system ΦDP, smoothening, and interpolation), KDP is then calculated from interpolated ΦDP (utilizing the POLARA algorithm), and the noise correction for the correlation coefficient ρHV is implemented. The hotspot detection following Gu et al. (2011) is implemented and also different formulations for (differential) attenuation correction below, within, and above the melting layer are applied. Next steps include the implementation of a ΦDP bump correction (backscatter differential phase δ), replacement of POLARAs spline interpolation for KDP calculation, and additional local KDP smoothening, as suggested in the P1-QPE algorithm. Additionally, the POLARA-RECALC system is now able to run the nowcasting (QPN) algorithm (e.g. STEPS or SPROG-LOC) scripts provided by P2 on a specific parameterized setup that aligns with the system (see again Fig. C1-1).


Figure C1-2: Data flow of POLARA QA and RealPEP jobs.





References

Chen, J.-Y., S. Trömel, A. Ryzhkov, C. Simmer, 2021: Assessing the benefits of specific attenuation for quantitative precipitation estimation with a C-band radar network, Journal of Hydrometeorology, 22(10), 2617–2631, https://doi.org/10.1175/JHM-D-20-0299.1.

Gu, J., A. V. Ryzhkov, P. Zhang, P. Neilley, M. Knight, B. Wolf, and D. Lee, 2011: Polarimetric attenuation correction in heavy rain at C Band. J. Appl. Meteor. Climatol., 50, 39-58, https://doi.org/10.1175/2010JAMC2258.1.

Reinoso-Rondinel R., M. Rempel, M. Schultze, S. Trömel, 2022: Nationwide Radar-Based Precipitation Nowcasting – A Localization Filtering Approach and its Application for Germany. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15, 1670–1691, doi: 10.1109/JSTARS.2022.3144342, https://ieeexplore.ieee.org/document/9689956.

Contribution of University of Bonn
At Uni Bonn a first installation of the POLARA C++ processing framework has been set up on a dedicated server. After joining a POLARA workshop, the project scientist dives into the overall structure of the complex framework and into the details of a first set of routines for data quality, data processing (e.g. for the processing of differential phase shift DP), and compositing. Figure C1-1 shows examples of two compositing algorithms.



Figure C1-1: Example of composites of horizontal reflectivities ZH measured with the DWD C-band radar network at 16:55 UTC on 25 July 2017. Maximum algorithm (left panel) retrieves composite values w.r.t. the maximum value within the column, while MinMSL algorithm (right panel) uses the values monitored at the minimum height above mean sea level.

Contribution Free University of Berlin
The aim is to support and improve quantitative precipitation estimation and nowcasting by exploiting satellite data. The predictive potential of organized structures and/or variability metrics in spatially (later also temporal) high resolution clear-sky water vapor fields regarding convective initiation is investigated. A set of satellite retrieved cloud properties at high temporal resolution contribute to information on cloud evolution and possibly on-going precipitation generation. Additionally, satellite data will be prepared for digestion by POLARA.

At the moment the project scientists works on both on the development of both the OLCI-Sentinel-3 integrated water vapour retrieval and SEVIRI-MSG integrated water vapour retrieval. The OLCI retrieval provides high spatial resolution and precision water vapour fields about twice a day, while the MSG retrieval provides high temporal water vapour fields at lower spatial resolution and precision. For OLCI about 1,5 years of data has been processed (see Figure C1-2 as an example) and first evaluation studies using GPS observations and model data are performed. MSG cloud products are collected and temporally and spatially matched with processed OLCI data as well as GPS data. These match-up datasets will be extended with the newly developed MSG retrievals and used in convective initiation studies.

Figure C1-2: Example of OLCI-retrieved integrated water vapor for 9:15 UTC on 16 June 2016.