Handbook of Writing for the Mathematical Sciences, Third Edition

hwms3_cover_front.jpg

The third edition of Handbook of Writing for the Mathematical Sciences was published by SIAM in January 2020. It is SIAM’s fourth best selling book of all time if sales for the first edition (1993) and second edition (1998) are combined, and it is in my top ten most cited outputs. As well as being used by individuals from students to faculty, it is a course text on transferable skills modules in many mathematics departments. A number of publishers cite the book as a reference for recommended style—see, for example, the AMS Author Handbook, the SIAM Style Manual and, outside mathematics and computing, the Chicago Manual of Style.

Parts of the second edition were becoming out of date, as they didn’t reflect recent developments in publishing (open access publishing, DOIs, ORCID, etc.) or workflow (including modern LaTeX packages, version control, and markup languages). I’ve also learned a lot more about writing over the last twenty years.

I made a variety of improvements for the third edition. I reorganized the material in a way that is more logical and makes the book easier to use for reference. I also improved the design and formatting and checked and updated all the references.

I removed content that was outdated or is now unnecessary. For example, nowadays there is no need to talk about submitting a hard copy manuscript, or one not written in LaTeX, to a publisher. I removed the 20-page appendix Winners of Prizes for Expository Writing, since the contents can now be found on the web, and likewise for the appendix listing addresses of mathematical organizations.

I also added a substantial amount of new material. Here are some of the more notable changes.

hwms1to3spines.jpg
  • A new chapter Workflow discusses how to organize and automate the many tasks involved in writing. With the increased role of computers in writing and the volume of digital material we produce it is important that we make efficient use of text editors, markup languages, tools for manipulating plain text, spellcheckers, version control, and much more.
  • The chapter on \LaTeX has been greatly expanded, reflecting both the many new and useful packages and my improved knowledge of typesetting.
  • I used the enumitem \LaTeX package to format all numbered and bulleted lists. This results in more concise lists that make better use of the page, as explained in this blog post.
  • I wrote a new chapter on indexing at the same time as I was reading the literature on indexing and making an improved index for the book. Indexing is an interesting task, but most of us do it only occasionally so it is hard to become proficient. This is my best index yet, and the indexing chapter explains pretty much everything I’ve learned abut the topic.
  • Since the second edition I have changed my mind about how to typeset tables. I am now a convert to minimizing the use of rules and to using the booktabs \LaTeX package, as explained in this blog post.
  • The chapter Writing a Talk now illustrates the use of the Beamer \LaTeX package.
  • The book uses color for syntax highlighted \LaTeX listings and examples of slides.
  • Sidebars in gray boxes give brief diversions on topics related to the text, including several on “Publication Peculiarities”.
  • An expanded chapter English Usage includes new sections on Zombie Nouns; Double Negatives; Serial, or Oxford, Comma; and Split Infinitives.
  • There are new chapters on Writing a Blog Post; Refereeing and Reviewing; Writing a Book; and, as discussed above, Preparing an Index and Workflow.
  • The bibliography now uses the backref \LaTeX package to point back to the pages on which entries are cited, hence I removed the author index.
  • As well as updating the bibliography I have added DOIs and URL links, which can be found in the online version of the bibliography in bbl and PDF form, which is available from the book’s website.

At 353 pages, and allowing for the appendices removed and the more efficient formatting, the third edition is over 30 percent longer than the second edition.

As always, working with the SIAM staff on the book was a pleasure. A special thanks goes to Sam Clark of T&T Productions, who copy edited the book. Sam, with whom I have worked on two previous book projects, not only edited for SIAM style but found a large number of improvements to the text and showed me some things I did not know about \LaTeX.

SIAM News has published an interview with me about the book and mathematical writing and publishing.

Here is a word cloud for the book, generated in MATLAB using the wordcloud function, based on words of 6 or more characters. wordcloud2.jpg

Posted in books, LaTeX, writing | Leave a comment

Accurately Computing the Softmax Function

The softmax function takes as input an n-vector x and returns a vector g(x) with elements

g_j(x) = \displaystyle\frac{\mathrm{e}^{x_j}}{\sum_{i=1}^n \mathrm{e}^{x_i}}, \quad j=1\colon n,

The elements of g are all between 0 and 1 and they sum to 1, so g can be regarded as a vector of probabilities. Softmax is a key function in machine learning algorithms.

Softmax is the gradient vector of the log-sum-exp function

f(x) = \displaystyle\log \sum_{i=1}^n \mathrm{e}^{x_i}.

This function is an approximation to the largest element, x_{\max} = \max_i x_i of the vector x, as it lies between x_{\max} and x_{\max} + \log n.

A problem with numerical evaluation of log-sum-exp and softmax is that overflow is likely even for quite modest values of x_i because of the exponentials, even though g(x) cannot overflow and f(x) is very unlikely to do so.

A standard solution it to incorporate a shift, a, and use the formulas

f(x) = a + \displaystyle\log \sum_{i=1}^n \mathrm{e}^{x_i-a}, \hspace*{4.5cm}(1)

and

g_j(x) = \displaystyle\frac{\mathrm{e}^{x_j-a}}{\sum_{i=1}^n \mathrm{e}^{x_i-a}}, \quad j=1\colon n, \hspace*{3.3cm}(2)

where a is usually set to x_{\max}.

Another formula for softmax is obtained by moving the denominator into the numerator:

g_j(x) = \exp\left(x_j - a - \log\displaystyle\sum_{i=1}^n\mathrm{e}^{x_i -a}\right). \hspace*{2cm}(3)

This formulas is used in various codes, including in the SciPy 1.4.1 function softmax.

How accurate are these formulas when evaluated in floating-point arithmetic? To my knowledge, this question has not been addressed in the literature, but it is particularly important given the growing use of low precision arithmetic in machine learning. Two questions arise. First, is there any difference between the accuracy of the formulas (2) and (3) for g_j(x)? Second, in (1) and (3), a is added to a nonnegative log term, so when a = x_{\max} is negative can there be damaging subtractive cancellation?

In a recent EPrint with Pierre Blanchard and Des Higham I have investigated these questions using rounding error analysis and analysis of the conditioning of the log-sum-exp and softmax problems. In a nutshell, our findings are that while cancellation can happen, it is not a problem: the shifted formulas (1) and (2) can be safely used.

However, the alternative softmax formula (3) is not recommended, as its rounding error bounds are larger than for (2) and we have found it to produce larger errors in practice.

Here is an example from training an artificial neural network using the MATLAB Deep Learning Toolbox. The network is trained to classify handwritten digits from the widely used MNIST data set. The following figure shows the sum of the computed elements of the softmax vector g(x) for 2000 vectors extracted from the training data, where g(x) was computed in IEEE half precision arithmetic. The sum should be 1. The red circles are for formula (2) and the blue crosses are for the division-free formula (3). Clearly, (2) gives a better approximation to a vector of probabilities (in the sense of respecting the constraint that probabilities sum to unity); the actual errors in each vector component are also smaller for (2).

sumpic3.png

Posted in research | Tagged , | Leave a comment

Top Five Posts of 2019

According to the WordPress statistics, this blog received over 39,000 visitors and 65,000 views in 2019. These are the five most-viewed posts published during the year.

list5.jpg

Image courtesy of Stuart Miles at FreeDigitalPhotos.net

  1. Who Invented the Matrix Condition Number?
  2. Numerical Algorithms for High-Performance Computational Science: Highlights of the Meeting
  3. Better LaTeX Tables with Booktabs
  4. Advances in Numerical Linear Algebra Conference and James Hardy Wilkinson Centenary
  5. The Argonne Tapes
Posted in miscellaneous | Leave a comment

Lists in LaTeX with the enumitem Package

\LaTeX provides the enumerate and itemize environments for numbered and itemized (usually bulleted) lists, respectively. Various package are available that provide more customizable list environments. While preparing the third edition of Handbook of Writing for the Mathematical Sciences I came across the enumitem package and ended up using it to typeset lists throughout the book.

As well as allowing all the customizations I could possibly need, enumitem has two very useful built-in options. By default, \LaTeX lists contain quite a lot of vertical space. The nosep option, used as in

\begin{itemize}[nosep]
...
\end{itemize}

(and similarly for enumerate) removes vertical spaces in the list. The wide option, used as in

\begin{itemize}[wide]
...
\end{itemize}

produces lists whose entries have zero indentation on the second and subsequent lines. Both options save space and look better to my eye, especially for a book. They can be combined by specifying [wide,nosep].

An example of a customization possible with enumitem is

\begin{enumerate}[label=X\arabic*.,ref=X\arabic*]
\item\label{item1}
...
\end{enumerate}

This enumerated list has labels X1, X2, etc., and a reference such as “see \ref{item1}" reproduces the label: “see X1”.

Description environments can also be customized (I use these very little).

enumitem_demo.jpg

For examples of the above customizations see the LaTeX file and PDF output (shown to the right) in my enumitem_demo repository on GitHub.

These examples barely scratch the surface of the customizations that enumitem makes possible. Consult the manual for full details.

For guidance on how to punctuate lists see Handbook of Writing for the Mathematical Sciences (section 3.26) or my blog post Punctuating Lists.

Posted in LaTeX | 1 Comment

A Mathematician Looks at the Collins English Dictionary

190324-1322-52_9515a.jpg

I have several dictionaries on my shelf, among which is a well-thumbed Collins English Dictionary (third edition, 1991). Earlier this year I acquired the thirteenth edition (2018). At 26.5cm high, 20cm wide, and 6.5cm deep, and weighing approximately 2.5kg, it’s an imposing tome. It’s printed on thin paper with minimal show-through and in a specially designed font (Collins Fedra) that is very legible.

The thirteenth edition, which I will abbreviate to CED13, is a wonderful acquisition for any dictionary lover. It has a wide coverage, including

  • new words such as micromort (“a unit of risk equal to a one-in-a million chance of dying”),
  • obscure words such as compotation (“the act of drinking together in a company”), and
  • a wide selection of proper nouns, including my home town Eccles and, somewhat unexpectedly, Laurel and Hardy and Torvill and Dean (Olympic ice dance champions, 1984).

It has no appendices on English usage, mathematical symbols, chemical elements, etc., as are found in many dictionaries—which is fine with me as I rarely use them.

I decided to take a close look at some of the mathematical words in the CED.

determinant n maths: a square array of elements that represents the sum of certain products of these elements, used to solve simultaneous equations, in vector studies, etc.”

This definition has two problems. First, a determinant is the sum, not something that represents the sum. Of, course, one will find in some textbooks statements such as “swapping two rows of a determinant changes its sign”, but it’s odd that this informal usage of determinant as array is the only one mentioned. A second problem is that the determinant is not a sum of products: it is a signed sum of products and it is the permanent (not in this dictionary) that is obtained by taking all positive signs.

matrix n maths a rectangular array of elements set out in rows and columns, used to facilitate the solution of problems, such as transformation of coordinates.”

A matrix is more than just an array: its key characteristic is that it has algebraic operations defined on it.

rounding: n computing a process in which a number is approximated as the closest number that can be expressed using the number of bits or digits available.”

Rounding is not specifically a computing term—it’s more fundamentally a mathematical operation and predates computing. Bits are special cases of digits. And rounding does not have to be to the closest number: in some situations once needs to round to the next larger or smaller number.

index n maths c a subscript or superscript to the right of a variable to express a set of variables, as in using x_i for x_1, x_2, x_3, etc”

An index does not (except maybe in informal usage) express a set, but rather identifies a member of a set.

supercomputer n a powerful computer that can process large quantities of data of a similar type very quickly.”

Supercomputers do mathematical calculations (and are ranked on their speed in doing so), which is not apparent from this definition. I’m also not sure why “of a similar type” is necessary. The PC on which I am typing is a supercomputer according to this definition!

integral n maths the limit of an increasingly large number of increasingly smaller quantities, related to the function that is being integrated (the integrand). The independent variables may be confined within certain limits (definite integral) or in the absence of limits (indefinite integral).”

This seems to be an attempt to state informally the Riemann sum definition of definite integration. Sadly, it’s technically incomplete and sure to baffle anyone who doesn’t already know about Riemann sums. It would have been much better to simply say that integration is the inverse of differentiation. The second sentence is grammatically incorrect.

fractal maths n a figure or surface generated by successive subdivisions of a simpler polygon or polyhedron, according to some iterative process.”

Surely any definition should mention fractional dimension and self-similarity? This definition excludes the fractal that is the boundary of the Mandelbrot set.

I’m not too surprised by these weaknesses, because in 1994 I wrote an article Which Dictionary for the Mathematical Scientist? (PDF file here) in which I evaluated several dictionaries (including CED3) from the point of view of their mathematical words and found problems such as those above in several of them.

Despite these criticisms, I very much like this dictionary and I use it as much as the other dictionaries on my desk. It is especially good on the computing side. I was pleased to see that my favourite editor, emacs, is included (though I’m not sure why it is not capitalized). Vi users will be sad to hear that Vi is not included. A good number of programming languages are present, including awk (uncapitalized), Java, and Javascript, but not, C++ (how would that be alphabetized?), Python, or R.

A particularly notable definition is

flops or FLOPS n acronym for floating-point operations per second: used as a measure of computer processing power (in combination with a prefix): megaflops; gigaflops.

This is much better than the Oxford English Dictionary’s definition of the singular flop as “a floating-point operation per second”. There are also entries for petaflop,10^{15} floating-point operations a second”, and teraflop, “a thousand billion floating-point operations a second”. I just wish the latter definition contained “10^{12}“, because there is scope for misunderstanding because of the alternative meaning of a billion as a million million in the UK.

Posted in books | Tagged | 1 Comment

Better LaTeX Tables with Booktabs

One of the joys of \LaTeX is that it makes it so easy to typeset tables. I’ve created many \LaTeX tables and I’ve always taken care to fine-tune them, most often by inserting commands of the form \rule{0cm}{8pt} to stop superscripts hitting a horizontal rule in the line above.

I’ve been aware for a while of a \LaTeX package called booktabs that claims to produce more beautiful tables that more closely resemble those used in traditional typesetting. I was dissuaded from trying it by three things: the results looked rather spread out, the package discourages the use of vertical rules (which I thought I needed), and I wasn’t sure if publishers such as SIAM would allow it to be used in their journals.

I was encouraged to give booktabs a try while I was preparing the third edition of Handbook of Writing for the Mathematical Sciences. I redid the tables from the previous edition using the package and, after some experimentation, became convinced that booktabs is indeed a great aid to producing better tables.

The best way to illustrate the attractions of booktabs is with a simple example. This table is from a 2017 SIAM J. Sci. Comput. paper of which I am a co-author: table_ex1.jpg

The original table did not have the vertical rules at the sides or the horizontal rule at the bottom, but I have added them since many people include them.

Here is a revised version of the table created with booktabs:

table_ex2.jpg

The second table dispenses with all the vertical rules and adds two partial horizontal rules to demarcate the span of the pairs of columns covered by the two tol values. Notice the thicker rules at the top and bottom and the greater spacing between lines. The booktabs version of the table is more aesthetically pleasing. It is also slightly easier to typeset. The rules at the top, middle, and bottom are typed with \toprule, \midrule, and \botomrule. The subrules are typed with \cmidrule{m-n} to span columns m to n. The usage \cmidrule(lr){m-n} crops (on the left and right) the rules so that adjacent subrules do not merge.

Here is the source code for the two tables:

% First version of table.
\begin{tabular}{|l|c|c|c|c|c|l|}\hline
& \multicolumn{3}{c|}{$\tol=\tols$} & \multicolumn{3}{c|}{$\tol=\told$}\\
           & $mv$  & Rel.~err & Time    & $mv$  & Rel.~err & Time   \\\hline
\trigmv    & 11034 & 1.3e-7 & 3.9 & 15846 & 2.7e-11 & 5.6 \\
\trigexpmv & 21952 & 1.3e-7 & 6.2 & 31516 & 2.7e-11 & 8.8 \\
\trigblock & 15883 & 5.2e-8 & 7.1 & 32023 & 1.1e-11 & 1.4e1\\
\expleja   & 11180 & 8.0e-9 & 4.3 & 17348 & 1.5e-11 & 6.6 \\\hline
\end{tabular}

% Second version of table, with booktabs.
\begin{tabular}{lcccccl}\toprule
& \multicolumn{3}{c}{$\tol=\tols$} & \multicolumn{3}{c}{$\tol=\told$}
\\\cmidrule(lr){2-4}\cmidrule(lr){5-7}
           & $mv$  & Rel.~err & Time    & $mv$  & Rel.~err & Time\\\midrule
\trigmv    & 11034 & 1.3e-7 & 3.9 & 15846 & 2.7e-11 & 5.6 \\
\trigexpmv & 21952 & 1.3e-7 & 6.2 & 31516 & 2.7e-11 & 8.8 \\
\trigblock & 15883 & 5.2e-8 & 7.1 & 32023 & 1.1e-11 & 1.4e1\\
\expleja   & 11180 & 8.0e-9 & 4.3 & 17348 & 1.5e-11 & 6.6 \\\bottomrule
\end{tabular}

Despite my initial skepticism, it does seem possible to manage without vertical rules in tables. Indeed, the Chicago Manual of Style (17th edition, 2017) advises that “Vertical rules should be used sparingly—for example, when a table is doubled up … or as an aid to comprehension in an especially long or complex table”. The extra space that booktabs puts around horizontal rules makes the table nicer to look at and helps to avoid the problem of superscripts hitting the rule above. And, importantly, SIAM are happy to for authors to use booktabs in their papers.

I am now using booktabs in all my \LaTeX code.

For more before-and-after examples of tables, which illustrate table design as well as the use of booktabs, see Handbook of Writing for the Mathematical Sciences (third edition, 2020).

One booktab tip that I needed in producing the Handbook was to reduce the space at the edges of some of the widest tables by shortening the horizontal rules. This can be done by putting @{} at the beginning and end of the column formatting argument. For the example above, this is done with

\begin{tabular}{@{}lcccccl@{}}\toprule

This blog contains various other posts about \LaTeX.

Posted in LaTeX | 2 Comments

What’s New in MATLAB R2019a and R2019b?

I didn’t have time earlier this year to write about the first MATLAB release of 2019, so in this post I will discuss R2019a and R2019b together. As usual in this series, I concentrate on the features most relevant to my work.

Modified Akima Cubic Hermite Interpolation (R2019b)

makima_example.jpg

A new function makima computes a piecewise cubic interpolant with continuous first-order derivatives. It produces fewer undulations than the existing spline function, with less flattening than pchip. This form of interpolant has been an option in interp1, interp2, interp3, interpn, and griddedInterpolant since R2017b and is now also a standalone function. For more details see Cleve Moler’s blog post about makima, from which the code used to generate the image above is taken.

Linear Assignment Problem and Equilibration (R2019a)

The matchpairs function solves the linear assignment problem, which requires each row of a matrix to be assigned to a column in such a way that the total cost of the assignments (given by the “sum of assigned elements” of the matrix) is minimized (or maximized). This problem can also be described as finding a minimum-weight matching in a weighted bipartite graph. The problem arises in sparse matrix computations.

The equilibrate function take as input a matrix A (dense or sparse) and returns a permutation matrix P and nonsingular diagonal matrices R and C such that R*P*A*C has diagonal entries of magnitude 1 and off-diagonal entries of magnitude at most 1. The outputs P, R, and C are sparse. This matrix equilibration can improve conditioning and can be a useful preprocessing step both in computing incomplete LU preconditioners and in iterative solvers. See, for example, the recent paper Max-Balanced Hungarian Scalings by Hook, Pestana, Tisseur, and Hogg.

The scaling produced by equilibrate is known as Hungarian scaling, and its computation involves solving a linear assignment problem.

Function Argument Validation (R2019b)

A powerful new feature allows a function to check that its input arguments have the desired class, size, and other aspects. Previously, this could be done by calling the function validateattributes with each argument in turn or by using the inputParser object (see Loren Shure’s comparison of the different approaches).

The new feature provides a more structured way of achieving the same effect. It also allows default values for arguments to be specified. Function argument validation contains many aspects and has a long help page. We just illustrate it with a simple example that should be self-explanatory.

function fav_example(a,b,f,t)
  arguments
    a (1,1) double               % Double scalar.
    b {mustBeNumeric,mustBeReal} % Numeric type and real.
    f double {mustBeMember(f,[-1,0,1])} = 0 % -1, 0 (default), or 1.
    t string = "-"               % String, with default value.
  end
  % Body of function.
end

The functions mustBeNumeric and mustBeMember are just two of a long list of validation functions. Here is an example call:

>> a = 0; b = 2; f = 11; fav_example(a,b,f)
Error using fav_example
Invalid input argument at position 3. Value must be a member of ...
this set:
    -1
    0
    1 

One caveat is that a specification within the arguments block

v (1,:) double    

allows not just 1-by-n vectors (for any n) but also n-by-1 vectors, because of the convention that in many situations MATLAB does not distinguish between row and column vectors.

For more extensive examples, see New with R2019b – Function Argument Validation and Spider Plots and More Argument Validation by Jiro Doke at MathWorks Blogs.

Dot Indexing (R2019b)

One can now dot index the result of a function call. For example, consider

>> optimset('Display','iter').Display
ans =
    'iter'

The output of the dot call can even be indexed:

>> optimset('OutputFcn',{@outfun1,@outfun2}).OutputFcn{2}
ans =
  function_handle with value:
    @outfun2

Of course, these are contrived examples, but in real codes such indexing can be useful. This syntax was not allowed in R2019a and earlier.

Hexadecimal and Binary Values (R2019b)

One can now specify hexadecimal and binary values using the prefixes 0x or 0X for hexadecimal (with upper or lower case for A to F) and 0b or 0B for binary. By default the results have integer type.

>> 0xA
ans =
  uint8
   10
>> 0b101
ans =
  uint8
   5

Changes to Function Precedence Order (R2019b)

The Release Notes say “Starting in R2019b, MATLAB changes the rules for name resolution, impacting the precedence order of variables, nested functions, local functions, and external functions. The new rules simplify and standardize name resolution”. A number of examples are given where the meaning of code is different in R2019b, and they could cause an error to be generated for codes that ran without error in earlier versions. As far as I can see, though, these changes are not likely to affect well-written MATLAB code.

Program File Size (R2019b)

Program files larger than “approximately 128 MB” will no longer open or run. This is an interesting change because I cannot imagine writing an M-file of that size. Good programming practice dictates splitting a code into separate functions and not including all of them in one program file. Presumably, some MATLAB users have produced such long files—perhaps automatically generated.

For a modern introduction and reference to MATLAB, see MATLAB Guide (third edition, 2017)

mg3-front-cover.jpg
Posted in software | Tagged | 1 Comment

Happy 80th Birthday Cleve!

190529-1704-07_5195_0028.jpg

Today is the 80th birthday of Cleve Moler. Most readers will know Cleve as the inventor of MATLAB. MATLAB was originally a Fortran program that Cleve wrote to give students easier access to the EISPACK and LINPACK libraries. Cleve and John Little later founded The MathWorks and turned MATLAB into a commercial product.

190529-1708-22_5202_0034a.jpg

Cleve was a speaker at the conference Advances in Numerical Linear Algebra: Celebrating the Centenary of the Birth of James H. Wilkinson held in Manchester last May. We took the opportunity there to celebrate Cleve’s birthday in advance. A video of Cleve’s talk is included below.

Most recently I saw Cleve at the International Congress on Industrial and Applied Mathematics (ICIAM) conference in Valencia, where he spoke in the minisymposium Bohemian Matrices and Applications that I organized with Rob Corless. See Cleve’s blog post about his talk.

Happy birthday, Cleve!

(Photo credits: Gian Maria Negri Porzio.)

190529-1703-55_5194_0027.jpg

190529-1707-01_5198_0031.jpg

Four past SIAM presidents in Manchester, May 2019: Cleve Moler, Margaret Wright, Nick Trefethen and Nick Higham.

Posted in people | 1 Comment

The Argonne Tapes

A few weeks ago, I was in contact with Chris Paige, an emeritus Professor of Computer Science at McGill University, Montreal. I mentioned that Sven Hammarling and I are collecting information and memorabilia about the numerical analyst James Hardy Wilkinson FRS (1919-1986) for our Wilkinson webpage, and asked Chris if he knew of anything we didn’t already have. He replied “I have 5 1973 Video cassettes, each about 1 hour, by Jim labelled `Eigensystem Workshop June 1973′. … His wonderful lecturing style, and his idiosynchrasies, might be of interest, as well as the marvellous content.”

190328-1832-23_0001.jpg

The tapes contain four talks by Wilkinson and one by Cleve Moler from an Eigensystem Workshop held at Argonne National Laboratory, Illinois, USA, in June 1973.

The tapes are Scotch UC60 High Energy color Videocassettes, in U-matic format, one of which is shown to the right. Chris was able to have the tapes digitized and the results are pretty good given the age of the tapes. We have put the videos on the Numerical Linear Algebra Group YouTube channel and link to them below.

The videos were announced at the recent conference Advances in Numerical Linear Algebra: Celebrating the Centenary of the Birth of James H. Wilkinson, held in Manchester May 29-30, 2019.

  • Inverse Iteration – James H. Wilkinson, Eigensystem Workshop, Argonne National Laboratory, Argonne, Illinois, USA, June 1973.
Posted in people | Tagged , | Leave a comment

Numerical Algorithms for High-Performance Computational Science: Highlights of the Meeting

wordle_nahpcs.png
Word cloud from abstracts.

The Royal Society discussion meeting Numerical Algorithms for High-Performance Computational Science, which I organized with Jack Dongarra and Laura Grigori, was held in London, April 8-9, 2019. The meeting had 16 invited speakers and 23 contributed posters, with a poster blitz preceding the lively poster session.

The meeting brought together around 150 people who develop and implement numerical algorithms and software libraries—numerical analysts, computer scientists, and high-performance computing researchers—with those who use them in some of today’s most challenging applications. The aim was to discuss the challenges brought about by evolving computer architectures and the characteristics and requirements of applications, both to produce greater awareness of algorithms that are currently available or under development and to steer research on future algorithms.

Several key themes emerged across multiple talks, all in the context of today’s high performance computing landscape in which processor clock speeds have stagnated (with the end of Moore’s law) and exascale machine are just two or three years away.

  • An important way of accelerating computations is through the use of low precision floating-point arithmetic—in particular by exploiting a hierarchy of precisions.
  • We must exploit low rank matrix structure where it exists, for example in hierarchical (H-matrix) form, combining it with randomized approximations.
  • Minimizing data movement (communication) is crucial, because of its increasing costs relative to the costs of floating-point arithmetic.
  • Co-design (the collaborative and concurrent development of hardware, software, and numerical algorithms, with knowledge of applications) is increasingly important for numerical computing.

Applications treated included deep learning, climate modeling, computational biology, and the Square Kilometre Array (SKA) radio telescope project. We learned that the latter project needs faster FFTs in order to deal with the petabytes of data it will generate each year.

Papers from the meeting will be published in a future issue of Philosophical Transactions of the Royal Society A.

Here are the talks (in order of presentation), with links to the slides.

For a brief summary of the talks see George Constantinides’s insightful thoughts on the meeting.

Here are some photos from the meeting.

Posted in conferences | 1 Comment