What is Applied Mathematics For?

Those of us working in applied mathematics are well aware that our field has many important uses in the real world. But if we are put on the spot during a conversation and asked to give some examples it can be difficult to conjure up a convincing list.

One response is to point people to The Princeton Companion to Applied Mathematics. Its 186 articles contains a large number of examples of how applied mathematics is put to work in fields such as sport, engineering, economics, physics, biology, computer science, and finance.

Another way to convince people of the value of applied mathematics is to get them to watch the 1-minute SIAM video below. It was constructed from interviews conducted at a variety of SIAM conferences and comprises snippets of 25 mathematicians saying what they use mathematics for.

Well done to Karthika Swamy Cohen and Michelle Montgomery at SIAM, Adam Bauser and his team at Bauser Media Group, and Sonja Stark at PilotGirl Productions, for producing this great advertisement for applied mathematics!

Anderson Acceleration

Anderson acceleration, also known in quantum chemistry as Pulay mixing or direct inversion in the iterative subspace (DIIS), is a technique for accelerating the convergence of a fixed-point iteration. It has been widely used in electronic structure computations, but does not seem to be well known to numerical analysts.

Anderson’s original paper is from 1965 and is well cited, as Google Scholar shows: I learned about Anderson acceleration in the minisymposium Anderson Acceleration and Applications organized by Tim Kelley at the SIAM Conference on Computational Science and Engineering in Salt Lake City in March 2015. Tim gave an excellent overview of the topic in the opening talk. The slides for that talk are available on Tim’s website.

PhD student Nataša Strabić and I have shown that Anderson acceleration works very well for speeding up the alternating projections method for computing the nearest correlation matrix. It typically gives a reduction in the number of iterations by a factor at least 2 for the standard nearest correlation matrix problem and by at least a factor 3 when additional constraints are imposed on the matrix (specified elements fixed and a lower bound on the smallest eigenvalue). In some cases the reduction is by a factor of as much as 25. Since the overhead of Anderson acceleration is small, significant speedups are obtained.

In my 2013 post The Nearest Correlation Matrix I included a MATLAB code nearcorr.m. In place of this I now recommend our new accelerated code nearcorr_aa.m, which is available from the repository anderson-accel-ncm on GitHub. Our paper describing this work is available on MIMS EPrints.

For me this project is an excellent illustration of the importance of going to conferences in order to learn of new ideas and new developments.

Top Five Tips on Book Writing

Snoopy writing

I’ve written four books, and am currently writing and editing a fifth (The Princeton Companion to Applied Mathematics). I am also an editor of two SIAM book series and chair the SIAM Book Committee. Based on this experience here are my top five tips about writing an (academic) book. These cover high level issues. In a subsequent post I will give some more specific tips relating to writing and typesetting a book or thesis.

1. Identify Your Audience

Book publishers ask prospective authors to complete a proposal form, one part of which asks who is the audience for the book. This is a crucial question that should be answered before a book is written, as the answer will influence the book in many ways.

As an example, you might be contemplating writing a book about the numerical solution of a certain class of equations and intend to include computer code. Your audience might be

readers in mathematics or a related subject who wish to learn about numerical methods for solving the equations and are most concerned with the theory or algorithms,
readers whose primary interest is in solving the equations and who wish to have lots of sample code that they can run,
readers in the previous class who also need to learn the language in which the examples are written.

The choice of content, and how the book is presented, will depend very much on which audience you are writing for.

2. Revise, Revise, Revise

Just like a paper, a book draft needs to go through multiple revisions, and you must not be afraid to make major changes at any stage. You may receive constructive criticisms from reviewers of your book proposal, but reviewers may not have time to read the complete manuscript carefully and you should not assume that they have found all errors, typos, and areas for improvement.

3. Take Time to Choose Your Publisher

Given the huge effort that goes into writing a book you should take the time to find the right publisher. Discuss your book with several publishers and compare what they can offer in the way of

format (hardback, paperback, electronic) and, if more than one format, the timescale in which each is made available,
if the publisher has branches in more than one country, how price and publication schedule will differ between the countries,
whether you are allowed to make a PDF version of the book freely available on your website, if this interests you,
willingness to allow you to choose the book design (page size, font, cover, etc.),
use of colour (which increases the cost),
royalties (including a possible advance),
pricing,
the publisher’s policy on translations,
copy editing (see the next section),
time from delivering a completed manuscript to publication,
marketing (will the book be advertised at all, and if so how?), and
how long your book is guaranteed to stay in print.

It is perfectly acceptable to submit a proposal to several publishers and see what they are willing to offer. However, it is only fair and proper to make clear to a publisher that you are talking to other publishers and, once you have set the wheels of a publisher’s review process in motion, to wait for an offer before making a decision to go with another publisher.

I am always surprised when I hear of authors who approach only one publisher, or who go with the first publisher to express an interest in the book. As in many contexts, it is best to make an informed choice from among the available options.

4. Ensure Your Book is Copy Edited

If you are an inexperienced writer, or your first language is not English, the benefits of copy editing are obvious. But even an experienced author finds it virtually impossible to think about all the little details that a copy editor will check for, such as correctness and consistency of spelling, notation, punctuation (notably the serial comma), citations, and references. For example, I sometimes mix US and UK spellings and don’t want to have to worry about finding and correcting my occasional lapses. A good copy editor will also suggest minor improvements of the text that might escape even the best writers.

Unfortunately, not all publishers copy edit all books nowadays. Notable exceptions that always do copy edit (and, as I know from experience, work to the highest standards in every respect) are Princeton University Press and SIAM.

If your publisher has a Style Manual it obviously makes sense to follow its guidelines in order to minimize changes at the copy editing stage. Here is a link to the SIAM Style Manual.

5. Think Twice Before Co-Authoring a Book

It might seem an attractive proposition to share authorship of a book: surely having $n$ co-authors reduces the work by a factor $1/n$ ? Unfortunately it often does not work out like that, despite best intentions. In fact, $n$ co-authors can easily take $n$ times as long to write a book as any one of them would. One of the biggest difficulties is timescale: one author may be willing and able to finish a book in a year but another may need twice that period to make their contribution. Indeed it is rare for the co-authors to be matched in the amount of effort they can put into the book; this is clearly problematic if initial expectations are not realized. Other potential problems are potentially differing opinions on content, notation, level, length, and almost anything else associated with a book.

Successful authorship teams often have a track record of co-authoring papers together. Although it is no guarantee that a much larger book project will run smoothly, experience with writing papers together will at least have given a good indication of where disagreements are likely to lie.

Videos of Lectures from Gene Golub SIAM Summer School 2013

Videos of lectures given by four of the five lecturers at the 2013 Gene Golub SIAM summer school at Fudan University in Shanghai are now available on the summer school website.

These include the five 2-hour lectures from my course on Functions of Matrices. Here is a summary of the contents of my lectures, with direct links to the videos hosted on YouTube.

Lecture 1: History, definitions and some applications of matrix functions. Quiz.
Lecture 2: Properties, more applications, Fréchet derivative, and condition number.
Lecture 3: Exponential integrator application. Problem classification. Methods for $f(A)$ : Schur-Parlett method, iterative methods for sign function and matrix square root.
Lecture 4: Convergence and stability of iterative methods for sign function and square root. The $f(A)b$ problem. Software for matrix functions.
Lecture 5: The method of Al-Mohy and Higham (2011) for the $\exp(A)b$ problem. Discussion of how to do research, reproducible research, workflow.

A written summary of the course is available as Matrix Functions: A Short Course (MIMS EPrint 2013.73).

The video team, visible in the photo below that I took of my audience, have done a great job. The music over the opening sequence is reminiscent of the theme from the film Titanic!

As a reminder, other relevant links are

a report on the School by Sam Relton,
post by me describing my experience of the summer school,
a post by me about the written summary of my course.

Matrix Functions Course at Gene Golub SIAM Summer School 2013

As described in a previous blog post, I gave a course on matrix functions at the Gene Golub SIAM Summer School in Shanghai last July. Summer Schools are regularly held in Shanghai and it has been traditional to produce a booklet with summaries of the courses delivered. The organizers therefore asked the speakers at the Golub Summer School to provide a summary of their courses.

I have written a summary, jointly with my postdoctoral research associate Lijing Lin, who acted as TA for the course. It is available as Matrix Functions: A Short Course (MIMS EPrint 2013.73). You can also access the other materials for the course.

In 2005 I interviewed Gene Golub when he visited Manchester. A transcription of the interview is available as An Interview with Gene Golub (MIMS EPrint 2008.8).

If you didn’t have the chance to meet Gene the interview will give you some insight into his career and the early history of numerical linear algebra. Here is a photo of Gene that I took after the interview.

The sketch below is by John de Pillis, a Professor of Mathematics at the University of California, Riverside. John is a talented sketcher and cartoonist and his 777 Mathematical Conversation Starters is full of cartoons, stories, and quotes. It includes a quote from Gene:

Most problems in scientific computing eventually lead to solving a
matrix equation.

Gene Golub SIAM Summer School 2013

A two week Gene Golub SIAM summer school on matrix functions and matrix equations was held at Fudan University, Shanghai from July 22 to August 2, 2013, in conjunction with the 3rd International Summer School on Numerical Linear Algebra and the 9th Shanghai Summer School on Analysis and Numerics in Modern Sciences. This was the fourth Golub Summer School and the second devoted to numerical linear algebra.

45 PhD students attended, coming from institutions in 15 countries. The lecturers were me and Marlis Hochbruck (Karlsruhe Institute of Technology, Germany) in week 1 and Peter Benner (Max Planck Institute, Magdeburg), Ren-Cang Li (University of Texas, Arlington) and Xiaoye (Sherry) Li (Lawrence Berkeley National Laboratory, USA) in week 2.

My 10 hours of lectures were on matrix functions. The slides and exercises can be downloaded from my website.

The lectures were held in the mornings in the GuangHua Twin Tower – an impressive, marbled 30-storey building on the Fudan campus. Attendees were grateful that the lecture room was air conditioned, as the Shanghai summer was at its peak of temperature and humidity, and on the Friday of the first week a record temperature of 40.6 degrees Celsius (105 degrees Fahrenheit) was reached in the city.

The GuangHua Twin Tower from my room in the Fuxuan Hotel.

Afternoons contained exercise sessions, 10-minute presentations by the students on their thesis work, and a guest seminar by Hongguo Xu (University of Kansas) in week 1 and Heike Fassbender (TU Braunschweig) in week 2. These were fully attended and it was great to see the students working so enthusiastically together and interacting with the lecturers.

For the lunches and dinners the students and lecturers sat together in randomized positions – an excellent idea on the part of the local organizers which helped ensure that people got to know each other.

Group photos at conferences can be rather shambolic. This one was the most professional I’ve ever seen. When we arrived at the designated spot the photographer had already set up three rows of metal staging and the photo was quickly taken (just as well given the scorching heat even at 8.15 am). Laminated photos were delivered to participants the same afternoon.

The local organizers are to be congratulated on an excellent job. In particular, Weiguo Gao and Yangfeng Su (Fudan University) and Zhaojun Bai (UC Davis) were busy every day making sure that the event ran smoothly. Daniel Szyld (Temple University) must also be mentioned for his excellent work over the last 5 years in chairing the SIAM committee that manages the Gene Golub SIAM Summer School program.

The school was generously supported by the SIAM Gene Golub Summer School fund, the Shanghai Center for the Mathematical Sciences, ISFMA (Sino French Institute of Applied Mathematics), the NSF (USA) and NAG.

Three things will stand out in my memory from the School. First, the enthusiasm of the students, among whom will no doubt be some of the future leaders of our field. (See the blog post by my PhD student Sam Relton.) Second, sitting in the plush 15th floor cafe of the GuangHua Tower chatting to other participants over a cafe latte with smooth jazz coming over the speakers. Third, the Chinese (motor) cyclists, who carry a wondrous variety of goods on their bikes and ride without any attention to the traffic signals but miraculously seem to avoid accidents. See the photos below taken on the short walk from the hotel to the department!

In summing up I can do no better than to endorse Charlie Van Loan’s words in describing the first Gene Golub Summer School in 2010: “The idea of a summer school for graduate students from around the world is the perfect way to honor Gene’s memory. It is exactly the kind of activity that Gene loved to promote.”

Professor Tatsien Li making welcoming remarks.

Morning callisthenics in front of the GuangHua Tower.

Notes on SIAM Annual Meeting Minisymposium on Professional Use of Social Media

In my recent post I publicized the upcoming minisymposium Establishing a Professional Presence in the Online World: Unraveling the Mysteries of Social Media and More organized by Tammy Kolda and I at the 2013 SIAM Annual Meeting in San Diego.

We had an enjoyable session. Despite being in the most far flung and hard to find room on the site, we had a good-sized audience who contributed useful questions and thoughts.

The slides for the four talks are downloadable from the previous post. Here, I summarize a few key points from each talk.

Tammy Kolda (Sandia National Labs) described how to export BibTeX entries for journal articles to html via the JabRef reference manager. The resulting html includes an abstract, keywords, and hyperlinks to the DOI, a PDF file, an expurgated BibTeX entry and a preprint version (assuming all this information is present in the entry). The idea is that the html can be used for lists of publications on a web page. I haven’t used Jabref for a while, but intend to try this export filter out. Tammy also gave a flowchart answering the question of how and where to post a publication list.

David Gleich (Purdue University) gave his presentation using Prezi, a cloud-based presentation tool that produces “multiscale” slides that zoom in and out. He surveyed the main social media tools and classified them into categories 1-1, 1-many and many-many. He then explained how he keeps on top of information using Flipboard, Feedly and Instapaper daily.

I described reasons for mathematicians to blog or tweet and the features that characterize a good blog. I also gave tips for using WordPress and Twitter and described SIAM’s plans for a SIAM blog.

Finally, Karthika Muthukumaraswamy (SIAM Public Awareness Officer) gave a compelling explanation of why mathematicians and scientists should blog and how the web is changing science communication. She also explained the benefits of blog networks, in which several people contribute to a blog, and the motivation for the planned SIAM blog.

More photos are available in my photo gallery.

Finally, I note that David Bindel has written some notes on the SIAM Annual meeting.

SIAM Annual Meeting Minisymposium on Professional Use of Social Media

Tammy Kolda and I are organizing a minisymposium Establishing a Professional Presence in the Online World: Unraveling the Mysteries of Social Media and More at the 2013 SIAM Annual Meeting in San Diego.

This page will act as a repository for the slides of the talks, related information, and a place for discussion. It will be updated as necessary from the date of first post. (Edit: the titles now link to the final versions of the talks in PDF form.)

MS89: Thursday, July 11, 10:30 AM-12:30 PM in Garden Salon I

10:30-10:55 Maintaining An Online Publication List, Tamara G. Kolda, Sandia National Laboratories, USA

Abstract: I will discuss the importance of making your publications easily
available online and various ways to maintain such a list. We’ll discuss
important information to include (like DOIs), various websites that
maintain the lists for you, and tools for tracking and exporting your own
lists.

11:00-11:25 Tools for Social Media (available as Prezi presentation or in PDF form), David F. Gleich, Purdue University, USA

Abstract: I’ll describe my experiences using social media from the past
few years and some lessons. This will include a brief survey of the tools
out there my reasons for using Twitter and WordPress.

11:30-11:55 Blogging and Tweeting for Mathematicians: Why and How, Nicholas J. Higham, University of Manchester, United Kingdom

Abstract. I will discuss how and why social media can be useful for a
researcher, both as a consumer and a contributor, drawing on my own
experiences of using Twitter and blogging with WordPress. I will also
discuss how SIAM is using social media.

12:00-12:25 On the Verge of a SIAM Community Blog, Karthika Muthukumaraswamy, SIAM, USA

Abstract. I will discuss the importance of blogging for scientific
communication in general, and more specifically, why SIAM may be ready
for a community blog. Based on surveys SIAM has conducted, I will
discuss how a shared blog space can help address the many needs of the
SIAM community in terms of networking, collaboration, scientific
discussion, funding and outreach. More broadly, I will discuss the
importance of direct communication between scientists and the general
public, and how blogs can help achieve this.

Fourth Edition (2013) of Golub and Van Loan’s Matrix Computations

Back in 1980 there were not many up to date books on numerical linear algebra. Stewart’s Introduction to Matrix Computations (1973) was a popular textbook, and was the text for the final year undergraduate course that I took on the subject. Parlett’s The Symmetric Eigenvalue Problem (1980) was a graduate level treatment of the symmetric eigenvalue problem. And Wilkinson’s The Algebraic Eigenvalue Problem (1965) was still the bible of numerical linear algebra, albeit already somewhat out of date due the fast moving research developments since it was published.

While an MSc student, I heard about the impending publication of a new book on matrix computations by Golub and Van Loan. I pre-ordered a copy and in spring 1983 received one of the first copies in the UK. The book was a revelation. It presented a completely fresh and up to date perspective on the subject. Some of the most exciting features were

extensive use of pseudocode, with MATLAB-style indexing notation, to describe algorithms,
the use of flops to measure computational cost,
emphasis on the use of the SVD,
modern presentation of rounding error analysis, with rounding error bounds given for each algorithm,
systematic treatment of the conjugate gradient and Lanczos methods,
coverage of topics not found in earlier books, such as condition estimation, generalized SVD, and total least squares,
very lively writing style.

I studied the book in great detail and learned a huge amount from it.

file://d:/dropbox/org/images/gova13.jpg — Covers of first to fourth editions.

A second edition was published in 1989. It was written while Charlie Van Loan was in the UK on sabbatical and I was spending a year at Cornell (Charlie’s home university). I had the opportunity to read and comment on draft chapters. The second edition maintained all the material from the first and added new chapters on matrix multiplication (and the relevant machine architecture considerations) and parallel algorithms, and it was typeset in LaTeX for the first time. The term flop was redefined so that a+b*c represents two flops (as it does today) instead of one as in the first edition. A number of other changes were introduced to address a criticism in some reviews of the first edition that the book was rather terse and fast-paced for use as a course textbook.

A third edition followed in 1996. After a 17 year gap the fourth edition has just been published. Work on this edition began following the untimely death of Gene Golub in 2007. Some statistics indicate the development of the book:

Edition	Year	Number of pages	Pages of master bibliography
First	1983	472	25
Second	1989	642	34
Third	1996	694	50
Fourth	2013	756	$65^\dagger$

$\dagger$ The master bibliography of the fourth edition is not printed in the book but is downloadable from the book’s web page.

What is Different About the Fourth Edition?

The new edition is physically larger than its predecessors, with a text width of 13 cm versus 11.5 cm in the last edition, so the content is increased by more than the page count would suggest. Moreover, the paper is extremely high quality, and this makes the book bigger and heavier than you would expect. I bought the hardback, because I know from experience that the softback of all three previous editions did not stand up well to heavy use. The image shows the third and fourth editions along with Horn and Johnson’s Matrix Analysis (second edition, 2013) and my Accuracy and Stability of Numerical Algorithms (second edition, 2002).

file://d:/dropbox/org/images/mc4-bookpile.jpg

A number of new topics are included, of which I would pick out

fast transforms
Hamiltonian and product eigenvalue problems
large-scale SVD
multigrid
tensor computations

I like the statement in the preface that “References that are historically important have been retained because old ideas have a way of resurrecting themselves.” This is of course particularly true as regards methods suitable for high-performance computing.

Lists of relevant LAPACK codes at the start of each chapter have been removed, as have many of the small, illustrative numerical examples, which are replaced by MATLAB codes to be made available on the book’s web page.

The fourth edition remains the best general reference on matrix computations and a must-have for any serious researcher in the field. A big difference from 1983, when the first edition appeared, is that now a separate research monograph is available covering almost every topic in the book (and due reference is made to 28 such “Global References”). But Matrix Computations brings together and unifies a wide variety of topics in one place.

2013 has been a good year for books on matrices and approximation, with the publication of a second edition of Horn and Johnson’s Matrix Analysis, Trefethen’s Approximation Theory and Approximation Practice, and now this very welcome fourth edition of Golub and Van Loan. It is available from the usual sources as well as from SIAM. Consider the Kindle edition to save your back. You can still have it signed!

file://d:/dropbox/org/images/mc4-sign.jpg

SIAM Conference on Computational Science and Engineering 2013

As predicted in my my preview post, this conference, held on the Boston waterfront, proved to be SIAM’s largest ever, with 1378 attendees. Over 1000 presentations were given in up to 20 parallel minisymposia at a time, but this did mean that there was at least one talk (and usually several) of interest to me in almost every time slot.

One thing I learned from the conference is how widely Python is being used in computational science, especially for solving real world problems involving large amounts of data. This is partly due to its ability to act as the glue between codes written in other languages and web applications. The IPython environment, with its notebook interface, was featured in a number of talks, in some of which the slides were displayed using the notebook.

The following highly selective photos will give a flavour of the conference.

file://d:/dropbox/org/images/130225-0726-35-1605.jpg The conference venue. Note the residual snow, which fortunately did not fall in any serious amounts during the conference.

file://d:/dropbox/org/images/130226-2124-20-1630.jpg The poster session of about 65 posters was preceded by a poster blitz (1 minute presentations) and was accompanied by an excellent dessert. This photo shows Edvin Deadman (University of Manchester and NAG Ltd.) discussing his poster on Matrix Functions and the NAG Library with Cleve Moler and Charlie Van Loan (authors of the classic Nineteen Dubious Ways to Compute the Exponential of a Matrix paper). For some thoughts on poster sessions by one of the conference attendees see Please, no posters! by David Gleich.

file://d:/dropbox/org/images/130227-1327-55-1690.jpg Josh Bloom’s (UC Berkeley) invited presentation Automated Astrophysics in the Big Data Era contained a fascinating mix of observational astronomy, machine learning, robotic telescopes, numerical linear algebra, and Python, with a focus on classifying stars.

file://d:/dropbox/org/images/130228-1629-26-1706.jpg It was interesting to see MapReduce being used to implement numerical algorithms, notably in the minisymposium Is MapReduce Good for Science and Simulation Data? organized by Paul Constantine (Stanford; standing) and David Gleich (Purdue; sitting, with pointer).

file://d:/dropbox/org/images/130227-1157-03-1678.jpg Here is the lunchtime panel Big Data Meets Big Models being videod. Highlights from this panel and some of the invited plenary talks will be available in due course on the SIAM Presents YouTube channel.

If you weren’t at the conference perhaps you can make it to the next one in two year’s time (date and location to be announced). In the meantime a good way to keep up with events is to join the SIAM Activity Group on Computational Science and Engineering, which organizes the conference.

Share this:

Share this:

1. Identify Your Audience

2. Revise, Revise, Revise

3. Take Time to Choose Your Publisher

4. Ensure Your Book is Copy Edited

5. Think Twice Before Co-Authoring a Book

Share this:

Share this:

Share this:

Share this:

Share this:

Share this:

What is Different About the Fourth Edition?

Share this:

Share this: