“Pangeo: An Open Source Big Data Climate Science Platform” is a project designed to solve one of climate science’s most pressing challenges: accessing and utilizing the explosive growth in the size of climate datasets, which have become a bulky but indispensable tool for scientific inquiry in climate change research.
There are several building crises facing the Atmosphere / Ocean / Land / Climate (AOC) science community:
- Big Data: datasets are growing too rapidly and legacy software tools for scientific analysis can’t handle them. This is a major obstacle to scientific progress.
- Technology Gap: a growing gap between the technological sophistication of industry solutions (high) and scientific software (low).
- Reproducibility: a fragmentation of software tools and environments renders most AOC research effectively unreproducible and prone to failure.
We believe these challenges can all be addressed through a unified effort.
Our mission is to cultivate an ecosystem in which the next generation of open-source analysis tools for ocean, atmosphere and climate science can be developed, distributed, and sustained. These tools must be scalable in order to meet the current and future challenges of big data, and these solutions should leverage the existing expertise outside of the AOC community.
We envision a collection of related but independent open-source packages that meet specific scientific needs within the AOC fields. These packages will follow modern best practices for software development, including:
- hosting on GitHub,
- continuous integration,
- comprehensive documentation, and
- a welcoming and inclusive development culture.
As much as possible, we will build on top of existing solutions and leverage expertise from the broader technology world, including industry.