A parallel, adaptive library for logically Cartesian, mapped, multiblock domains
What is ForestClaw?
ForestClaw is a parallel, multi-block adaptive finite volume library for solving PDEs on mapped, logically Cartesian meshes. For solving hyperbolic problems using explicit, single step algorithms, ForestClaw's block-structured adaptive algorithm, including multi-rate time stepping uses the Berger, Oliger and Colella AMR algorithms (JCP, 1984, 1989). The hyperbolic solvers are currently based on ClawPack (R. J. LeVeque). Future plans include support for general method-of-lines solvers in a multi-rate setting.
Where ForestClaw departs from the standard Berger-Oliger-Colella block-structured approach is that the multi-resolution grid hierarchy is not stored as overlapping, nested grids but rather as a composite structure of non-overlapping fixed sized grids, each of which is stored as a leaf in a forest of quad- or octrees. The particular code base we use for managing the tree is p4est (www.p4est.org), (Carsten Burstedde, Univ. of Bonn).
Why develop another AMR code?
Currently, there are several AMR codes for doing block-structured AMR. What are the advantages of the ForestClaw approach?
- By using an external library for grid management, we completely separate issues related to grid connectivity, load balancing, and regridding, from the underlying numerics. With p4est, we can exploit state-of-the-art technology for managing tree-based data structures, including the use of space filling curves for maintaining data locality in parallel.
- Because of the composite quad- or octree layout, ghost cell exchanges occur in a regular pattern at patch boundaries, rather than between ghost cells on the fine grid and interior grid cells on the coarse grid. This eliminates the need to store the locations of where fine grids are positioned in the coarse grid. Also, only a limited number of neighbor configurations are allowed, again reducing the code complexity and simplifying the development of new numerical algorithms. This simplification is particularly advantageous in 3d.
- p4est has been built for distributed parallelism and designed to allow transparent and efficient parallelization of numerical codes such as ForestClaw. For example, the p4est algorihms for regridding and re-balancing (ensuring that all levels are properly nested) has been shown to scale well on up to 262,144 CPU cores.
- ForestClaw inherits the multi-block domain structure from p4est and so adaptive mesh refinement for the sphere grid proposed by Calhoun, Helzel and LeVeque (SIAM Review, 2008), or the cubed sphere can all be easily handled by ForestClaw.
Progress on the ForestClaw project
- (Fall/Winter, 2016) Developing performance testing tools for running on a local linux cluster (64-nodes) and JUQUEEN (Blue Gene/Q) machines; obtain scaling performance for a variety of test problems.
- (Spring/Summer, 2015) Parallel multi-block code mostly working; parallel performance optimizations reduce regridding time significantly.
- (Winter, 2015) Parallel IO for package and user options.
- (Fall, 2014) Periodic domains
- (Summer, 2014) Mapped, multiblock capabilities for several useful domains, including cubed disk and sphere,
- (Spring, 2014) Build system using GNU Autotools
- (Winter, 2014) Parco paper appears.
- (Summer, 2013) Parallel (MPI) capabilities added; Parco paper submitted and accepted.
- (Fall, 2012) General mappings available
- (Fall, 2012) Multirate time stepping
- (Summer, 2012) Development on ForestClaw begins
Publications and Presentations
Recent talks and posters
- SIAM CSE, March, 2015 (Salt Lake City, Utah) (talk: .pdf, poster: .pdf)
- HPC3, November, 2014 (Kaust). (.pdf)
- C. Burstedde, D. Calhoun, K. Mandli, A. Terrel, "Hybrid forest-of-octrees AMR for hyperbolic conservation laws", (2014). (iospress, arXiv)
- Attend the mini-symposium at the upcoming Parallel Processing Meeting (Paris, France, April 13-16, 2016).
- Attend our minisymposium at SIAM
CSE 2015 (Salt Lake City, March 14-18, 2015)
(Sunday 3/15 : MS102 - 9:10AM, MS127 - 1:30PM)
- (September, 2014) Developer Donna Calhoun was recently featured in a Boise State update Math vs. the Volcano : Mitigating an Eruption's Aftermath
- (July, 2014) We have recently received three years of National Science Foundation funding (Award #1419108, $194K) for development of ForestClaw. Please see the project_summary for a description of the award. Collaborators on this project include Carsten Burstedde (Univ. Bonn, Germany), D. Ketcheson (Kaust, Saudi Arabia) and D. George (USGS, Vancouver, WA).
Working on the ForestClaw project
There are funding opportunities for students interested in working on developing numerical solvers and applications using ForestClaw. We are recruiting for :
- A Master's level student to work volcanic ash cloud dispersion using ForestClaw.
Potential candidates should have excellent programming skills,
as well as an interest in geophysical applications. Funding
is available for 1 year. Please contact Donna Calhoun at
Talin Mirzakhanian has started a Master's Degree in the Fall of 2015, working with D. Calhoun on developing multi-rate Runge-Kutta-Chebyschev methods for adaptive Cartesian grids.
Where can I get ForestClaw?
(January, 2016) We released an alpha version in Fall 2014, and will be releasing a beta version soon. Thank you for your patience!
Donna Calhoun would like to acknowledge the support of the Isaac Newton Institute and the program Adaptive Multiscale Numerics for the Ocean and Atmosphere, where much of the initial work in developing ForestClaw was done, and the National Science Foundation (NSF).
Last modified: Thu Feb 11 13:15:08 MST 2016