Web-based radar data portal for the access of archived radar data
Project based at
Deutscher Wetterdienst (DWD), Phase 1
Deutscher Wetterdienst (DWD): Hella Riede (Scientist), Kathleen Helmert (PI)
Abstract
Within the PROM project, we will develop a radar data portal for easy, automated retrieval of archived DWD weather radar data. Primary users are selected DWD research partners at universities. With a dedicated login, they will be able to retrieve archived radar data from the non-relational DWD “SKY” database “CIRRUS” according to user-defined parameters such as time span and radar product. The user will be free of worry about database request limitations at DWD, as a server between portal and database will manage user requests. After requested data is retrieved from the DWD database and uploaded to an HTTP or SFTP server, an automated alert email will be sent to the user that data is ready for download. The complicated chain of manual actions at DWD triggered by radar data requests (see Fig. 1 left) will be replaced by a customer-friendly and provider-friendly radar data portal (see Fig. 1 right).
Figure 1: Schematic of current workflow of DWD radar data requests (left) and future workflow (right). Orange color relates to actions and entities of users; blue color relates to actions and entities of DWD.
A radar data portal prototype exists within the DWD intranet. Since the PROM project start at DWD in December 2019, a concept is developed how to best address users' needs for easy data retrieval while respecting DWD infrastructure rules and limits, and how to transfer the prototype from DWD intranet use to internet use, in close collaboration with DWD IT services.
Final Report 2023
The provision of radar data from the DWD to the researchers in PROM is a necessary condition to achieve the goals of the SPP. The aim of the project was to create software that would allow to easily provide radar data to the project partners at universities, even for longer periods of time. During the project duration (December 2019 to April 2021, 50% time), a concept was developed to best address users' needs for easy data retrieval while respecting DWD critical infrastructure rules and limits. The complicated process chain of manual actions at DWD triggered by radar data requests (see Fig. 1 left) should be replaced by a customer- and provider-friendly web-based radar data portal (see Fig. 1 right).
The idea to offer data through a web framework that uses a tool known from daily scientific work based on python was evaluated as the best solution. Therefore, the radar data portal was built on the basis of a Django web framework and offers the possibility to select the desired radar data or products for the desired period via an input mask. The underlying algorithm extracts the data from the database and prepares it in a user-friendly manner.
Figure 1: Schematic of current workflow of DWD radar data requests (left) and future workflow (right). Orange color relates to actions and entities of users; blue color relates to actions and entities of DWD.
Finally, the concept was implemented to such an extent that the radar data portal was set up within the DWD (see Fig. 2). This means that DWD employees can take over the provision of data without knowledge of the database limitations, as a server between portal and database will manage the requests.
Figure 2: Schematic of current workflow of DWD radar data requests (after project). Orange color relates to actions and entities of users; blue color relates to actions and entities of DWD.
The planned option, however, that project partners have direct access to the archived radar data via this portal could not be implemented because the project scientist left PROM prematurely. Nevertheless, with the results of the project, the radar data provision process has been considerably improved and simplified. Project partners of SPP-PROM now receive requested data with only short waiting time in an uncomplicated manner even for longer periods of time.