The quote comes from Franz Alt’s recollection of a talk given by von Neumann at the very first meeting of the Association for Computing Machinery in 1947:

Several versions of background wiring and their corresponding source languages were under discussion, each having a vocabulary between 50 and 100 instruction types. Their implementation and testing began in 1948. They were still only on paper at the end of 1947, when the Association for Computing Machinery was founded and held its first national meeting at Aberdeen Proving Ground. The attendance at that meeting was 300; the program consisted of about a dozen papers. We had succeeded in obtaining John yon Neumann as keynote speaker. He discussed the need for, and likely impact of, electronic computing. He mentioned the “new programming method” for ENIAC and explained that its seemingly small vocabulary was in fact ample: that future computers, then in the design stage, would get along on a dozen instruction types, and this was known to be adequate for expressing all of mathematics. (Parenthetically, it is as true today as it was then that “programming” a problem means giving it a mathematical formulation. Source languages which use “plain English” or other appealing vocabularies are only mnemonic disguises for mathematics.) Von Neumann went on to say that one need not be surprised at this small number, since about 1,000 words were known to be adequate for most situations of real life, and mathematics was only a small part of life, and a very simple part at that. This caused some hilarity in the audience, which provoked von Neumann to say: “If people do not believe that mathematics is simple, it is only because they do not realize how complicated life is.”

_{Franz Alt (1972). ‘Archaeology of computers: reminiscences, 1945–1947’. Communications of the ACM 15:7, p. 694.}

So von Neumann had been explaining to the audience that a very small number of machine instruction types was sufficient to express any desired mathematical calculation. This is a corollary of the Church–Turing thesis (roughly speaking, that all models of computing have equivalent power), something that every computer science undergraduate learns in their first course on computer architecture today (and moreover, that a single instruction is sufficiently expressive!). But the concepts were new at the time and his audience had some trouble taking it in.

Von Neumann then made an analogy with “plain English”—by which, I think he means to refer to Basic English, a project started in 1925 by philosopher Charles Kay Ogden to create an international auxiliary language based on English but using a small (roughly 1,000-word) vocabulary. He says that since “life” (meaning all human knowledge) can be described in Basic English with only 1,000 words, and mathematics is only a small part of human knowledge, it should not be surprising that mathematics can be described using an even smaller vocabulary (that is, the “dozen instruction types” that he proposed for future computers). The apparent complexity of mathematics arises from the infinite combinations of the vocabulary, and not from the size of the vocabulary.

The quote comes from Franz Alt’s recollection of a talk given by von Neumann at the very first meeting of the Association for Computing Machinery in 1947:

Several versions of background wiring and their corresponding source languages were under discussion, each having a vocabulary between 50 and 100 instruction types. Their implementation and testing began in 1948. They were still only on paper at the end of 1947, when the Association for Computing Machinery was founded and held its first national meeting at Aberdeen Proving Ground. The attendance at that meeting was 300; the program consisted of about a dozen papers. We had succeeded in obtaining John yon Neumann as keynote speaker. He discussed the need for, and likely impact of, electronic computing. He mentioned the “new programming method” for ENIAC and explained that its seemingly small vocabulary was in fact ample: that future computers, then in the design stage, would get along on a dozen instruction types, and this was known to be adequate for expressing all of mathematics. (Parenthetically, it is as true today as it was then that “programming” a problem means giving it a mathematical formulation. Source languages which use “plain English” or other appealing vocabularies are only mnemonic disguises for mathematics.) Von Neumann went on to say that one need not be surprised at this small number, since about 1,000 words were known to be adequate for most situations of real life, and mathematics was only a small part of life, and a very simple part at that. This caused some hilarity in the audience, which provoked von Neumann to say: “If people do not believe that mathematics is simple, it is only because they do not realize how complicated life is.”

_{Franz Alt (1972). ‘Archaeology of computers: reminiscences, 1945–1947’. Communications of the ACM 15:7, p. 694.}

So von Neumann had been explaining to the audience that a very small number of machine instruction types was sufficient to express any desired mathematical calculation. This is a corollary of the Church–Turing thesis (roughly speaking, that all models of computing have equivalent power), something that every computer science undergraduate learns in their first course on computer architecture today (and moreover, that a single instruction type is sufficiently expressive!). But the concepts were new at the time and perhaps not yet fully intuitive to his audience.

Von Neumann then made an analogy between computer instruction sets and human languages with restricted vocabularies. I guess that he had Basic English in mind: this was a project started in 1925 by philosopher Charles Kay Ogden to create an international auxiliary language based on English but using a small (roughly 1,000-word) vocabulary. Von Neumann said that since “real life” (meaning all human knowledge) can be described in Basic English with only about 1,000 words, and mathematics is only a small part of human knowledge, it should not be surprising that mathematics can be described using an even smaller vocabulary (that is, the “dozen instruction types” that he proposed for future computers). The apparent complexity of mathematics arises from the infinite combinations of the vocabulary, not from its size.