Finalized:Wednesday, November 11, 2015
Author(s):Leonard, L., K. Madduri and C. J. Duffy
HydroTerre is a research prototype platform developed at Penn State for the hydrology community. It provides access to aggregated scientific data sets that are useful for hydrological modeling and research. HydroTerre's frontend is a web service, and a user query can request creation of a data bundle whose size can vary from a few megabytes to 100's of gigabytes. In this article, we present software tuning and optimization strategies for various hardware configurations of the HydroTerre platform. Our goal is to minimize access time to a wide range of data bundle creation queries from users. We use automated schemes to estimate the computational work required for various queries, and identify the best-performing hardware/software configuration. We hope this study is instructive for researchers developing similar data management cyberinfrastructure in other science and engineering fields.
L. Leonard, K. Madduri and C. J. Duffy, "Tuning Heterogeneous Computing Platforms for Large-Scale Hydrology Data Management," in IEEE Transactions on Parallel and Distributed Systems, vol. 27, no. 9, pp. 2753-2765, Sept. 1 2016. doi: 10.1109/TPDS.2015.2499741This material is based upon work supported by the National Science Foundation under Grant No. 1440291. Opinions, findings, conclusions or recommendations expressed are those of the authors and do not reflect the views of the NSF.