The Art of Computer Programming
The Art of Computer Programming, Volume 1: Fundamental Algorithms
 
Author  Donald Knuth 

Country  United States 
Language  English 
Genre 
Nonfiction Monograph 
Publisher  AddisonWesley 
Publication date 
1968– (the book is still incomplete) 
Media type  Print (Hardcover) 
ISBN  0201038013 
519  
LC Class  QA76.75 
The Art of Computer Programming (TAOCP) is a comprehensive monograph written by computer scientist Donald Knuth that covers many kinds of programming algorithms and their analysis.
Knuth began the project, originally conceived as a single book with twelve chapters, in 1962. The first three volumes of what was then expected to be a sevenvolume set were published in 1968, 1969, and 1973. The first published installment of Volume 4 appeared in paperback as Fascicle 2 in 2005. The hardback Volume 4A, combining Volume 4, Fascicles 0–4, was published in 2011. Volume 4, Fascicle 6 ("Satisfiability") was released in December 2015; Volume 4, Fascicle 5 ("Mathematical Preliminaries Redux; Backtracking; Dancing Links") is currently estimated for release in November 2019.
The estimated release date for Fascicle 5 has slipped several times, due to lastminute inclusion of additional material. ^{[1]} Recent additions include an accelerated version of the Dancing Links algorithm based on the Binary Decision Diagrams of Volume 4A §7.1.4 enabling the efficient use of memo cache to store and reuse partial solutions.^{[2]}
Fascicles 5 and 6 are expected to comprise the first twothirds of Volume 4B. Knuth has not announced any estimated date for release of Volume 4B, although his method used for Volume 4A is to release the hardback volume some time after release of the paperback fascicles that comprise it. Nearterm publisher estimates put the release date at May or June of 2019, which proved to be incorrect.^{[3]}^{[4]}
Contents
History
After winning a Westinghouse Talent Search scholarship, Knuth enrolled at the Case Institute of Technology (now Case Western Reserve University), where his performance was so outstanding that the faculty voted to award him a master of science upon his completion of the baccalaureate degree. During his summer vacations, Knuth was hired by the Burroughs Corporation to write compilers, earning more in his summer months than full professors did for an entire year.^{[5]} Such exploits made Knuth a topic of discussion among the mathematics department, which included Richard S. Varga.
Knuth started to write a book about compiler design in 1962, and soon realized that the scope of the book needed to be much larger. In June 1965, Knuth finished the first draft of what was originally planned to be a single volume of twelve chapters. His handwritten firstdraft manuscript (completed in 1966) was 3000 pages long: he had assumed that about five handwritten pages would translate into one printed page, but his publisher said instead that about 1 ^{1}⁄_{2} handwritten pages translated to one printed page. This meant the book would be approximately 2000 pages in length. The publisher was nervous about accepting such a project from a graduate student. At this point, Knuth received support from Richard S. Varga, who was the scientific adviser to the publisher. Varga was visiting Olga TausskyTodd and John Todd at Caltech. With Varga's enthusiastic endorsement, the publisher accepted Knuth's expanded plans. In its expanded version, the book would be published in seven volumes, each with just one or two chapters.^{[6]} Due to the growth in the material, the plan for Volume 4 has since expanded to include Volumes 4A, 4B, 4C, 4D, and possibly more.
In 1976, Knuth prepared a second edition of Volume 2, requiring it to be typeset again, but the style of type used in the first edition (called hot type) was no longer available. In 1977, he decided to spend some time creating something more suitable. Eight years later, he returned with T_{E}X, which is currently used for all volumes.
The offer of a socalled Knuth reward check worth "one hexadecimal dollar" (100_{HEX} base 16 cents, in decimal, is $2.56) for any errors found, and the correction of these errors in subsequent printings, has contributed to the highly polished and stillauthoritative nature of the work, long after its first publication. Another characteristic of the volumes is the variation in the difficulty of the exercises. The level of difficulty ranges from "warmup" exercises to unsolved research problems.
Knuth's dedication reads:
This series of books is affectionately dedicated
to the Type 650 computer once installed at
Case Institute of Technology,
with whom I have spent many pleasant evenings.^{[a]}
Assembly language in the book
All examples in the books use a language called "MIX assembly language", which runs on the hypothetical MIX computer. Currently, the MIX computer is being replaced by the MMIX computer, which is a RISC version. Software such as GNU MDK exists to provide emulation of the MIX architecture. Knuth considers the use of assembly language necessary for the speed and memory usage of algorithms to be judged.
Critical response
Knuth was awarded the 1974 Turing Award "for his major contributions to the analysis of algorithms […], and in particular for his contributions to the 'art of computer programming' through his wellknown books in a continuous series by this title."^{[7]} American Scientist has included this work among "100 or so Books that shaped a Century of Science", referring to the twentieth century,^{[8]} and within the computer science community it is regarded as the first and still the best comprehensive treatment of its subject. Covers of the third edition of Volume 1 quote Bill Gates as saying, "If you think you're a really good programmer… read (Knuth's) Art of Computer Programming… You should definitely send me a résumé if you can read the whole thing."^{[9]} The New York Times referred to it as "the profession's defining treatise".^{[10]}
Volumes
Completed
 Volume 1 – Fundamental Algorithms
 Chapter 1 – Basic concepts
 Chapter 2 – Information structures
 Volume 2 – Seminumerical Algorithms
 Chapter 3 – Random numbers
 Chapter 4 – Arithmetic
 Volume 4A – Combinatorial Algorithms
 Chapter 7 – Combinatorial searching (part 1)
Planned
 Volume 4B... – Combinatorial Algorithms (chapters 7 & 8 released in several subvolumes)
 Chapter 7 – Combinatorial searching (continued)
 Chapter 8 – Recursion
 Volume 5 – Syntactic Algorithms (as of 2017^{[update]}, estimated for release in 2025)
 Chapter 9 – Lexical scanning (also includes string search and data compression)
 Chapter 10 – Parsing techniques
 Volume 6 – The Theory of ContextFree Languages
 Volume 7 – Compiler Techniques
Chapter outlines
Completed
Volume 1 – Fundamental Algorithms
 Chapter 1 – Basic concepts
 1.1. Algorithms
 1.2. Mathematical Preliminaries
 1.2.1. Mathematical Induction
 1.2.2. Numbers, Powers, and Logarithms
 1.2.3. Sums and Products
 1.2.4. Integer Functions and Elementary Number Theory
 1.2.5. Permutations and Factorials
 1.2.6. Binomial Coefficients
 1.2.7. Harmonic Numbers
 1.2.8. Fibonacci Numbers
 1.2.9. Generating Functions
 1.2.10. Analysis of an Algorithm
 1.2.11. Asymptotic Representations
 1.2.11.1. The Onotation
 1.2.11.2. Euler's summation formula
 1.2.11.3. Some asymptotic calculations
 1.3 MMIX (MIX in the hardback copy but updated by fascicle 1)
 1.3.1. Description of MMIX
 1.3.2. The MMIX Assembly Language
 1.3.3. Applications to Permutations
 1.4. Some Fundamental Programming Techniques
 1.4.1. Subroutines
 1.4.2. Coroutines
 1.4.3. Interpretive Routines
 1.4.3.1. A MIX simulator
 1.4.3.2. Trace routines
 1.4.4. Input and Output
 1.4.5. History and Bibliography
 Chapter 2 – Information Structures
 2.1. Introduction
 2.2. Linear Lists
 2.2.1. Stacks, Queues, and Deques
 2.2.2. Sequential Allocation
 2.2.3. Linked Allocation
 2.2.4. Circular Lists
 2.2.5. Doubly Linked Lists
 2.2.6. Arrays and Orthogonal Lists
 2.3. Trees
 2.3.1. Traversing Binary Trees
 2.3.2. Binary Tree Representation of Trees
 2.3.3. Other Representations of Trees
 2.3.4. Basic Mathematical Properties of Trees
 2.3.4.1. Free trees
 2.3.4.2. Oriented trees
 2.3.4.3. The "infinity lemma"
 2.3.4.4. Enumeration of trees
 2.3.4.5. Path length
 2.3.4.6. History and bibliography
 2.3.5. Lists and Garbage Collection
 2.4. Multilinked Structures
 2.5. Dynamic Storage Allocation
 2.6. History and Bibliography
 Chapter 1 – Basic concepts
Volume 2 – Seminumerical Algorithms
 Chapter 3 – Random Numbers
 3.1. Introduction
 3.2. Generating Uniform Random Numbers
 3.2.1. The Linear Congruential Method
 3.2.1.1. Choice of modulus
 3.2.1.2. Choice of multiplier
 3.2.1.3. Potency
 3.2.2. Other Methods
 3.2.1. The Linear Congruential Method
 3.3. Statistical Tests
 3.3.1. General Test Procedures for Studying Random Data
 3.3.2. Empirical Tests
 3.3.3. Theoretical Tests
 3.3.4. The Spectral Test
 3.4. Other Types of Random Quantities
 3.4.1. Numerical Distributions
 3.4.2. Random Sampling and Shuffling
 3.5. What Is a Random Sequence?
 3.6. Summary
 Chapter 4 – Arithmetic
 4.1. Positional Number Systems
 4.2. Floating Point Arithmetic
 4.2.1. SinglePrecision Calculations
 4.2.2. Accuracy of Floating Point Arithmetic
 4.2.3. DoublePrecision Calculations
 4.2.4. Distribution of Floating Point Numbers
 4.3. Multiple Precision Arithmetic
 4.3.1. The Classical Algorithms
 4.3.2. Modular Arithmetic
 4.3.3. How Fast Can We Multiply?
 4.4. Radix Conversion
 4.5. Rational Arithmetic
 4.5.1. Fractions
 4.5.2. The Greatest Common Divisor
 4.5.3. Analysis of Euclid's Algorithm
 4.5.4. Factoring into Primes
 4.6. Polynomial Arithmetic
 4.6.1. Division of Polynomials
 4.6.2. Factorization of Polynomials
 4.6.3. Evaluation of Powers
 4.6.4. Evaluation of Polynomials
 4.7. Manipulation of Power Series
 Chapter 3 – Random Numbers
Volume 3 – Sorting and Searching
 Chapter 5 – Sorting
 5.1. Combinatorial Properties of Permutations
 5.1.1. Inversions
 5.1.2. Permutations of a Multiset
 5.1.3. Runs
 5.1.4. Tableaux and Involutions
 5.2. Internal sorting
 5.2.1. Sorting by Insertion
 5.2.2. Sorting by Exchanging
 5.2.3. Sorting by Selection
 5.2.4. Sorting by Merging
 5.2.5. Sorting by Distribution
 5.3. Optimum Sorting
 5.3.1. MinimumComparison Sorting
 5.3.2. MinimumComparison Merging
 5.3.3. MinimumComparison Selection
 5.3.4. Networks for Sorting
 5.4. External Sorting
 5.4.1. Multiway Merging and Replacement Selection
 5.4.2. The Polyphase Merge
 5.4.3. The Cascade Merge
 5.4.4. Reading Tape Backwards
 5.4.5. The Oscillating Sort
 5.4.6. Practical Considerations for Tape Merging
 5.4.7. External Radix Sorting
 5.4.8. TwoTape Sorting
 5.4.9. Disks and Drums
 5.5. Summary, History, and Bibliography
 5.1. Combinatorial Properties of Permutations
 Chapter 6 – Searching
 Chapter 5 – Sorting
Volume 4A – Combinatorial Algorithms, Part 1
 Chapter 7 – Combinatorial Searching
 7.1. Zeros and Ones
 7.1.1. Boolean Basics
 7.1.2. Boolean Evaluation
 7.1.3. Bitwise Tricks and Techniques
 7.1.4. Binary Decision Diagrams
 7.2. Generating All Possibilities
 7.2.1. Generating Basic Combinatorial Patterns
 7.2.1.1. Generating all ntuples
 7.2.1.2. Generating all permutations
 7.2.1.3. Generating all combinations
 7.2.1.4. Generating all partitions
 7.2.1.5. Generating all set partitions
 7.2.1.6. Generating all trees
 7.2.1.7. History and further references
 7.2.1. Generating Basic Combinatorial Patterns
 7.1. Zeros and Ones
 Chapter 7 – Combinatorial Searching
Planned
Volume 4B, 4C, 4D – Combinatorial Algorithms
 Chapter 7 – Combinatorial Searching (continued)
 7.2. Generating all possibilities (continued)
 7.2.2. Backtrack programming (online as prefascicle 5b)
 7.2.2.1. Dancing links (online as prefascicle 5c)
 7.2.2.2. Satisfiability (published in Fascicle 6)
 7.2.2.3. Constraint satisfaction
 7.2.2.4. Hamiltonian paths
 7.2.2.5. Cliques
 7.2.2.6. Covers (Vertex cover, Set cover problem, Exact cover, Clique cover)
 7.2.2.7. Squares
 7.2.2.8. A potpourri of puzzles
 7.2.2.9. Estimating backtrack costs (chapter 6 of "Selected Papers on Analysis of Algorithms", and prefascicle 5b in Section 7.2.2 under the heading "Running time estimates")
 7.2.3. Generating inequivalent patterns (includes discussion of Pólya enumeration theorem)
 7.2.2. Backtrack programming (online as prefascicle 5b)
 7.3. Shortest paths
 7.4. Graph algorithms
 7.4.1. Components and traversal
 7.4.2. Special classes of graphs
 7.4.3. Expander graphs
 7.4.4. Random graphs
 7.5. Network algorithms
 7.5.1. Distinct representatives
 7.5.2. The assignment problem
 7.5.3. Network flows
 7.5.4. Optimum subtrees
 7.5.5. Optimum matching
 7.5.6. Optimum orderings
 7.6. Independence theory
 7.6.1. Independence structures
 7.6.2. Efficient matroid algorithms
 7.7. Discrete dynamic programming (see also Transfermatrix method)
 7.8. Branchandbound techniques
 7.9. Herculean tasks (aka NPhard problems)
 7.10. Nearoptimization
 7.2. Generating all possibilities (continued)
 Chapter 8 – Recursion (chapter 22 of "Selected Papers on Analysis of Algorithms")
 Chapter 7 – Combinatorial Searching (continued)
Volume 5 – Syntactic Algorithms

as of 2017^{[update]}, estimated for release in 2025
 Chapter 9 – Lexical scanning (includes also string search and data compression)
 Chapter 10 – Parsing techniques
Volume 6 – The Theory of Contextfree Languages^{[11]}
Volume 7 – Compiler Techniques
English editions
Current editions
These are the current editions in order by volume number:

The Art of Computer Programming, Volumes 14A Boxed Set. Third Edition (Reading, Massachusetts: AddisonWesley, 2011), 3168pp.
ISBN 9780321751041, 0321751043
 Volume 1: Fundamental Algorithms. Third Edition (Reading, Massachusetts: AddisonWesley, 1997), xx+650pp. ISBN 9780201896831, 0201896834. Errata: [1] (20110108), [2] (20170918, 27th printing). Addenda: [3] (2011).
 Volume 2: Seminumerical Algorithms. Third Edition (Reading, Massachusetts: AddisonWesley, 1997), xiv+762pp. ISBN 9780201896848, 0201896842. Errata: [4] (20110108), [5] (20170918, 26th printing). Addenda: [6] (2011).
 Volume 3: Sorting and Searching. Second Edition (Reading, Massachusetts: AddisonWesley, 1998), xiv+780pp.+foldout. ISBN 9780201896855, 0201896850. Errata: [7] (20110108), [8] (20170918, 27th printing). Addenda: [9] (2011).
 Volume 4A: Combinatorial Algorithms, Part 1. First Edition (Reading, Massachusetts: AddisonWesley, 2011), xv+883pp. ISBN 9780201038040, 0201038048. Errata: [10] (20170918, ? printing).
 Volume 1, Fascicle 1: MMIX – A RISC Computer for the New Millennium. (AddisonWesley, 20050214) ISBN 0201853922 (will be in the fourth edition of volume 1). Errata: [11] (20160802).
 Volume 4, Fascicle 5: Mathematical Preliminaries Redux; Backtracking; Dancing Links. (AddisonWesley, 20190914) 350pp, ISBN 9780134671796 (will become part of volume 4B)
 Volume 4, Fascicle 6: Satisfiability. (AddisonWesley, 20151208) xiii+310pp, ISBN 9780134397603. Errata: [12] (20170601) (will become part of volume 4B)
Previous editions
Complete volumes
These volumes were superseded by newer editions and are in order by date.
 Volume 1: Fundamental Algorithms. First edition, 1968, xxi+634pp, ISBN 0201038013.^{[12]}
 Volume 2: Seminumerical Algorithms. First edition, 1969, xi+624pp, ISBN 0201038021.^{[12]}
 Volume 3: Sorting and Searching. First edition, 1973, xi+723pp+foldout, ISBN 020103803X. Errata: [13].
 Volume 1: Fundamental Algorithms. Second edition, 1973, xxi+634pp, ISBN 0201038099. Errata: [14].
 Volume 2: Seminumerical Algorithms. Second edition, 1981, xiii+ 688pp, ISBN 0201038226. Errata: [15].
 The Art of Computer Programming, Volumes 13 Boxed Set. Second Edition (Reading, Massachusetts: AddisonWesley, 1998), pp. ISBN 9780201485417, 0201485419
Fascicles
Volume 4's fascicles 0–4 were revised and published as Volume 4A.
 Volume 4, Fascicle 0: Introduction to Combinatorial Algorithms and Boolean Functions. (AddisonWesley Professional, 20080428) vi+240pp, ISBN 0321534964. Errata: [16] (20110101).
 Volume 4, Fascicle 1: Bitwise Tricks & Techniques; Binary Decision Diagrams. (AddisonWesley Professional, 20090327) viii+260pp, ISBN 0321580508. Errata: [17] (20110101).
 Volume 4, Fascicle 2: Generating All Tuples and Permutations. (AddisonWesley, 20050214) v+127pp, ISBN 0201853930. Errata: [18] (20110101).
 Volume 4, Fascicle 3: Generating All Combinations and Partitions. (AddisonWesley, 20050726) vi+150pp, ISBN 0201853949. Errata: [19] (20110101).
 Volume 4, Fascicle 4: Generating All Trees; History of Combinatorial Generation. (AddisonWesley, 20060206) vi+120pp, ISBN 0321335708. Errata: [20] (20110101).
 Volume 4, Fascicle 6: Satisfiability. (AddisonWesley, 20151218) xiii+310pp, ISBN 9780134397603. Errata: [21] (20170601)
Prefascicles
Volume 4's prefascicle 6A was revised and published as fascicle 6.
 Volume 4B, Prefascicle 5A: Mathematical Preliminaries Redux (available for download; 55 pp July 19, 2019)
 Volume 4B, Prefascicle 5B: Introduction to Backtracking (available for download; 58 pp July 19, 2019)
 Volume 4B, Prefascicle 5C: Dancing Links (available for download; 270 pp July 19, 2019)
See also
References
Notes
 ^ The dedication was worded slightly differently in the first edition.
Citations
 ^ https://web.archive.org/web/20190711045601/https://www.nytimes.com/2018/12/17/science/donaldknuthcomputersalgorithmsprogramming.html The Yoda of Silicon Valley; by Siobhan Roberts, New York Times, December 17, 2018
 ^ https://aaai.org/ocs/index.php/AAAI/AAAI17/paper/download/14907/13855 Dancing with Decision Diagrams: A Combined Approach to Exact Cover; by Masaaki Nishino, Norihito Yasuda, Shinichi Minato, Masaaki Nagata1; Proceedings of the ThirtyFirst AAAI Conference on Artificial Intelligence (2017)
 ^ AddisonWesley Pearson webpage
 ^ Pearson Educational
 ^ Frana, Philip L. (20011108). "An Interview with Donald E. Knuth". hdl:11299/107413.
 ^ Albers, Donald J. (2008). "Donald Knuth". In Albers, Donald J.; Alexanderson, Gerald L. (eds.). Mathematical People: Profiles and Interviews (2 ed.). A K Peters. ISBN 1568813406.
 ^ "Donald E. Knuth – A. M. Turing Award Winner". AM Turing. Retrieved 20170125.

^ Morrison, Philip; Morrison, Phylis (November–December 1999). "100 or so Books that shaped a Century of Science". American Scientist. Sigma Xi, The Scientific Research Society. 87 (6). Archived from the original on 20080820. Retrieved 20080111. Cite uses deprecated parameter
deadurl=
(help)  ^ Weinberger, Matt. "Bill Gates once said 'definitely send me a résumé' if you finish this fiendishly difficult book". Business Insider. Retrieved 20160613.
 ^ Lohr, Steve (20011217). "Frances E. Holberton, 84, Early Computer Programmer". The New York Times. Retrieved 20100517.
 ^ "TAOCP – Future plans".
 ^ ^{a} ^{b} Wells, Mark B. (1973). "Review: The Art of Computer Programming, Volume 1. Fundamental Algorithms and Volume 2. Seminumerical Algorithms by Donald E. Knuth" (PDF). Bulletin of the American Mathematical Society. 79 (3): 501–509. doi:10.1090/s000299041973131738.
Sources
 Slater, Robert (1987). Portraits in Silicon. MIT Press. ISBN 0262192624.
 Shasha, Dennis; Lazere, Cathy (1995). Out of Their Minds: The Lives and Discoveries of 15 Great Computer Scientists. Copernicus. ISBN 0387979921.
External links
 Overview of topics (Knuth's personal homepage)
 Oral history interview with Donald E. Knuth at Charles Babbage Institute, University of Minnesota, Minneapolis. Knuth discusses software patenting, structured programming, collaboration and his development of TeX. The oral history discusses the writing of The Art of Computer Programming.
 "Robert W Floyd, In Memoriam", by Donald E. Knuth  (on the influence of Bob Floyd)
 TAoCP and its Influence of Computer Science (Softpanorama)