The Royal Society discussion meeting Numerical Algorithms for High-Performance Computational Science, which I organized with Jack Dongarra and Laura Grigori, was held in London, April 8-9, 2019. The meeting had 16 invited speakers and 23 contributed posters, with a poster blitz preceding the lively poster session.
The meeting brought together around 150 people who develop and implement numerical algorithms and software libraries—numerical analysts, computer scientists, and high-performance computing researchers—with those who use them in some of today’s most challenging applications. The aim was to discuss the challenges brought about by evolving computer architectures and the characteristics and requirements of applications, both to produce greater awareness of algorithms that are currently available or under development and to steer research on future algorithms.
Several key themes emerged across multiple talks, all in the context of today’s high performance computing landscape in which processor clock speeds have stagnated (with the end of Moore’s law) and exascale machine are just two or three years away.
- An important way of accelerating computations is through the use of low precision floating-point arithmetic—in particular by exploiting a hierarchy of precisions.
- We must exploit low rank matrix structure where it exists, for example in hierarchical (H-matrix) form, combining it with randomized approximations.
- Minimizing data movement (communication) is crucial, because of its increasing costs relative to the costs of floating-point arithmetic.
- Co-design (the collaborative and concurrent development of hardware, software, and numerical algorithms, with knowledge of applications) is increasingly important for numerical computing.
Applications treated included deep learning, climate modeling, computational biology, and the Square Kilometre Array (SKA) radio telescope project. We learned that the latter project needs faster FFTs in order to deal with the petabytes of data it will generate each year.
Papers from the meeting will be published in a future issue of Philosophical Transactions of the Royal Society A.
Here are the talks (in order of presentation), with links to the slides.
- Hierarchical algorithms on hierarchical architectures, David E. Keyes
- Antisocial parallelism: avoiding, hiding and managing communication, Katherine Yelick
- Big telescope, big data: towards exa-scale with the SKA, Anna Scaife
- Algorithms for in situ data analytics in next generation molecular dynamics workflows, Michela Taufer
- Numerical computing challenges for large-scale deep learning, Rick Stevens
- High-performance sampling of determinantal point processes, Jack Poulson
- Computing beyond the end of Moore’s Law, John Shalf
- Exascale applications: skin in the game, Douglas B. Kothe
- Machine learning and big scientific data, Tony Hey
- Iterative linear algebra in the exascale era, Erin Carson
- Stochastic rounding and reduced-precision fixed-point arithmetic for solving neural ODEs, Steve Furber
- Rethinking deep learning: architectures and algorithms, George Constantinides
- Reduced numerical precision and imprecise computing for ultra-accurate next-generation weather and climate models, Tim Palmer
- Memory-aware algorithms for automatic differentiation and backpropagation, Guillaume Aupy
- Post-K: the first ‘exascale’ supercomputer for convergence of HPC and big data/AI, Satoshi Matsuoka
- Accelerated sparse linear algebra: emerging challenges and capabilities for numerical algorithms and software, Michael A. Heroux
For a brief summary of the talks see George Constantinides’s insightful thoughts on the meeting.
— G A Constantinides (@gconstantinides) April 8, 2019
Here are some photos from the meeting.