Skip to main content
2023 ESA Annual Meeting Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.
Back

Contributed Talk

Session: : Modeling 1

COS 200-3 - Cyberinfrastructure for End-to-end Near Real-time Forecasting Workflows in Lakes and Reservoirs: Architecture and Lessons Learned from a Multi-year Deployment

Wednesday, August 9, 2023
2:00 PM – 2:15 PM PDT
Location: B113
Abstract: Lakes and reservoirs provide essential ecosystem services, including water for drinking, fisheries, and irrigation, and thus managing the water quality of these ecosystems is critical for public health and society. As water quality in lakes and reservoirs around the globe becomes more variable due to human activities, anticipating future changes is urgently needed for managers. In response to this challenge, we developed a near real-time water quality forecasting system to inform real-time management of drinking water supplies. We use a flexible and robust cyberinfrastructure (CI) that combines in-situ observational sensor data, ensemble model execution, and data assimilation for end-to-end water quality forecasting workflows. The CI is able to accommodate both the wide geographical distribution of computational and storage resources while being usable by ecologists and forecasters.



Here, we present the innovations to the CI underpinning FLARE (Forecasting Lake And Reservoir Ecosystems), and share the lessons learned from the deployment, operation, and enhancements to the forecasting system across a multi-year deployment encompassing over ten lakes and reservoirs in the US. In particular, we overview the challenges and opportunities in the following aspects: 1) the deployment of commodity computers at the network’s edge (i.e. near in-situ sensors and data loggers) running open-source software, including in remote locations with intermittent power and lacking wireless cellular access; 2) the use of event-driven cloud computing functions (supported by systems including the open-source OpenWhisk and the commercial GitHub Actions platform) as a basis for dynamic and low-cost execution of workflow modules; and 3) the use of cloud storage systems for both reliable transfer and versioned storage for time-series sensor data (including the open-source Git protocol and the commercial GitHub service) and scalable handling of inputs, outputs and intermediate files during workflow execution (including both open-source and commercial implementations of the Amazon S3 storage service, such as Minio). Finally, we will also outline how our lessons learned can be applied to other forecasting systems, and ongoing efforts to lower the barrier to entry to the deployment of workflows based on the R language in event-driven cloud computing systems.