May 2019 – Nick Higham

The Royal Society discussion meeting Numerical Algorithms for High-Performance Computational Science, which I organized with Jack Dongarra and Laura Grigori, was held in London, April 8-9, 2019. The meeting had 16 invited speakers and 23 contributed posters, with a poster blitz preceding the lively poster session.

The meeting brought together around 150 people who develop and implement numerical algorithms and software libraries—numerical analysts, computer scientists, and high-performance computing researchers—with those who use them in some of today’s most challenging applications. The aim was to discuss the challenges brought about by evolving computer architectures and the characteristics and requirements of applications, both to produce greater awareness of algorithms that are currently available or under development and to steer research on future algorithms.

Several key themes emerged across multiple talks, all in the context of today’s high performance computing landscape in which processor clock speeds have stagnated (with the end of Moore’s law) and exascale machine are just two or three years away.

An important way of accelerating computations is through the use of low precision floating-point arithmetic—in particular by exploiting a hierarchy of precisions.
We must exploit low rank matrix structure where it exists, for example in hierarchical (H-matrix) form, combining it with randomized approximations.
Minimizing data movement (communication) is crucial, because of its increasing costs relative to the costs of floating-point arithmetic.
Co-design (the collaborative and concurrent development of hardware, software, and numerical algorithms, with knowledge of applications) is increasingly important for numerical computing.

Applications treated included deep learning, climate modeling, computational biology, and the Square Kilometre Array (SKA) radio telescope project. We learned that the latter project needs faster FFTs in order to deal with the petabytes of data it will generate each year.

Papers from the meeting will be published in a future issue of Philosophical Transactions of the Royal Society A.

Here are the talks (in order of presentation), with links to the slides.

Hierarchical algorithms on hierarchical architectures, David E. Keyes
Antisocial parallelism: avoiding, hiding and managing communication, Katherine Yelick
Big telescope, big data: towards exa-scale with the SKA, Anna Scaife
Algorithms for in situ data analytics in next generation molecular dynamics workflows, Michela Taufer
Numerical computing challenges for large-scale deep learning, Rick Stevens
High-performance sampling of determinantal point processes, Jack Poulson
Computing beyond the end of Moore’s Law, John Shalf
Exascale applications: skin in the game, Douglas B. Kothe
Machine learning and big scientific data, Tony Hey
Iterative linear algebra in the exascale era, Erin Carson
Stochastic rounding and reduced-precision fixed-point arithmetic for solving neural ODEs, Steve Furber
Rethinking deep learning: architectures and algorithms, George Constantinides
Reduced numerical precision and imprecise computing for ultra-accurate next-generation weather and climate models, Tim Palmer
Memory-aware algorithms for automatic differentiation and backpropagation, Guillaume Aupy
Post-K: the first ‘exascale’ supercomputer for convergence of HPC and big data/AI, Satoshi Matsuoka
Accelerated sparse linear algebra: emerging challenges and capabilities for numerical algorithms and software, Michael A. Heroux

For a brief summary of the talks see George Constantinides’s insightful thoughts on the meeting.

Great turnout for @royalsociety discussion meeting. Kicking off with @nhigham introducing. pic.twitter.com/lNeQld2RN7

— G A Constantinides (@gconstantinides) April 8, 2019