“Algorithms for Modern Hardware” is an upcoming high-performance computing book by Sergey Slotin, aimed at a wide range of readers from performance engineers to undergraduate students. The book, hosted entirely on GitHub, focuses on using modern hardware’s parallelism capabilities for practical algorithm speed improvements. It is divided into various parts, with themes ranging from performance engineering to distributed computing, and encourages community contributions and feedback.
Main Points
Performance Engineering
The book covers the basics of computer architecture and optimization of single-threaded algorithms, detailing CPU optimization topics like caching, SIMD, and pipelining.
Parallel Algorithms
Addresses concurrency, cache coherence, synchronization primitives, OpenMP, and more, focusing on models of parallelism and concurrent algorithms.
Distributed Computing
Explores distributed computing topics like networking, message passing, distributed databases, and cloud computing.
Software & Hardware
Covers software and hardware aspects like LLVM IR, compiler optimizations, FPGAs, ASICs, TPUs, and other AI accelerators.
Insights
Performance engineering shifts the focus from sole asymptotic complexity to practical approaches utilizing hardware parallelism.
In modern practical algorithm design, you choose the approach that makes better use of different types of parallelism available in the hardware over the one that theoretically does fewer raw operations on galaxy-scale inputs.
Open-source publication paves the way for a new educational paradigm in algorithm design and performance engineering.
What I really want to achieve is that performance engineering becomes taught right after introduction to algorithms. Writing the first comprehensive textbook on the subject is a large part of it, and this is why I rush to finish it by the summer so that the colleges can pick it up in the next academic year.
Links
URL
https://en.algorithmica.org/hpc/