In order to join virtual sessions, you must be registered and logged-in(Were you registered for the in-person meeting in Denver? if yes, just log in.) 
Note: All session times are in Mountain Daylight Time.

On-demand Model Validation in R package epidemiar: Malaria Forecasts and Accuracy in Ethiopia

Authors: Dawn M. Nekorchuk*, University of Oklahoma, Justin K. Davis, University of Oklahoma, Worku Awoke, Bahir Dar University, Ethiopia, Abere Mihretie, Health, Development, and Anti-Malaria Association, Ethiopia, Aklilu Getinet, Health, Development, and Anti-Malaria Association, Ethiopia, Teklehaimanot Gebrehiwot, Amhara Public Health Institute, Ethiopia , Michael C. Wimberly, University of Oklahoma
Topics: Medical and Health Geography, Applied Geography, Spatial Analysis & Modeling
Keywords: disease forecasting, model validation, early warning, Geohealth, infectious disease modeling, malaria
Session Type: Paper
Presentation File: No File Uploaded


The Epidemic Prognosis Incorporating Disease and Environmental Monitoring for Integrated Assessment (EPIDEMIA) project produced operational malaria forecasts for the Amhara region of Ethiopia. Collaborating with our Ethiopian partners on requirements, we developed the R package epidemiar to provide a generalized set of functions for disease forecasting, plus customized code including a Google Earth Engine script for environmental data and formatting scripts for distributable reports with maps and graphs.
We built model validation tools into the epidemiar R package for on-demand evaluation for any historical period. Accuracy statistics included Mean Absolute Error (MAE), Root Mean Square Error (RMSE), proportion of observations that fell inside the prediction intervals, and an R2 (variance explained, VEcv). Evaluation can be made for one through n-week ahead predictions, including optional assumptions of how many weeks reporting of cases is delayed.
Skill scores are calculated comparing the forecast model against two naïve models: persistence of last known value, and average cases from that week of the year. All statistics and assessments are calculated for both overall and per geographic district (woreda).
Building validation into the early warning system provides more opportunities to learn about the model via the validation results. We can identify locations where the models perform best with district-level results. With on-demand implementation and time-range flexibility, we can also investigate how accuracy changes over time, which is of particular interest in places like Ethiopia with changing patterns and declining trends due to anti-malarial programs.

Abstract Information

This abstract is already part of a session. View the session here.

To access contact information login