tl;dr
Iambic meters are the most common of all classical and modern meters generally, and iambic pentameter is closest to the natural patterns of English speech. Plus, it was a successful conspiracy cooked up by Marlowe, Sidney, and the Great Vowel Shift.
Introduction
The word iamb comes from an ancient Greek genre of invective poetry. The metrical term takes its name from the genre with which it was associated. Deriving from this Greek origin, a foot of one short and one long syllable was called an iamb in Latin poetry as well. The Princeton Encyclopedia of Poetry and Poetics says:
Iambic was thought in antiquity to be the rhythm nearest to common speech. Throughout Gr. and Lat. poetry, iambic in trimeter and tetrameter is the standard meter for recitation forms, esp. dramatic dialogue.... The same claim has been made in mod. times about Eng. as was made by the ancients about Gr. (Aristotle, Rhetoric, bk. 3)—namely, that iambic is the rhythm of common speech.
Interestingly, even though English prosody is accentual-syllabic, i.e., based on stressed and unstressed syllables, whereas classical prosody is quantitative, i.e., based on long and short syllables, the tendency to see iambic rhythms as nearest to common speech prevails in both.
With regard to English poetry specifically, we could begin by asking why spoken English falls into iambic patterns. We could then explore how poets exploit these patterns, and ask why pentameter is the most common meter.
Iambic as the natural pattern of English speech
In A Prosody Handbook, Karl Shapiro and Robert Beum write that in English:
intensity of expression is nearly always accompanied by an abundance of stress. Now, in English prose the ratio of nonstresses to stresses ranges from about 2:1 to 3:1. This means that in the rather more intense language of poetry, one should expect the ratio to be closer to 1:1, and we find that in fact 9:7 is a common figure. Clearly, then, a balance or near balance between troughs and crests is called for. (p. 34)
Shapiro and Beum argue that iambs are common in English speech because stressed monosyllables are typically preceded by unstressed articles, prepositions, or conjunctions. In words of two syllables, the stress more commonly falls on the first syllable rather than the second, and since these too would be introduced by unstressed articles or the like, speech patterns tend to default to iambic.
An earlier edition of the Princeton Encyclopedia provides a couple of examples (slightly modified here):
| x ‘ | x ‘ | x ` | x ‘ | x ‘ |
| I'll take | a Whop- | per, and | a med- | ium Coke. |
| x ‘ | x ‘ | x ‘ x ` x ‘
| k, when | you find | the ans- | wer, let | me know. |
Iambs in English poetry
Of course poets do not merely transcribe ordinary speech. Transforming everyday language into poetry involves, inter alia, manipulating its stress patterns. How did poets turn to iambic meter as the default for English poetry?
John Thompson's The Founding of English Metre provides a synopsis of how the naturally occurring iambic patterns of English were standardized as the norm between the publication of the first printed anthology of English poetry, Tottel's Miscellany in 1567, and the circulation of Philip Sidney's Astrophil and Stella poems in the 1580s.
It is true that Chaucer and Wyatt, among others, had used iambic meter earlier. But the 15th and 16th centuries were a time of tremendous change in the English language. The Great Vowel Shift, which separates Middle English from its modern counterpart, was underway through this period. Chaucer's late 14th C. poetry was firmly in Middle English, and Sidney's late 16th C. poetry just as firmly in Modern English. The very different pronunciation the phrase pale knight would have between Chaucer and Sidney immediately calls into question any easy statement such as "Chaucer used iambs, so English poetry followed Chaucer in that regard". Chaucer would have pronounced pale knight as an amphimacer (‘ x ‘); Sidney as a single spondee ( ‘ ‘ ). As for Wyatt, nobody tf knows how to pronounce, let alone scan, Wyatt. There's a wonderful Lit SE question to be asked about that, as I've said before (see Note 2 in that answer).
Thompson's argument is that as modE pronunciation and the conventions of English poetry began to be more or less standardized, iambic meter began to predominate because iambs are the natural pattern of English. Thompson credits Sidney with establishing iambic pentameter as the default for English poetry because he exploited the principle of tension. Tension is the difference between the abstract structure of a metrical pattern, and the actual stresses that prevail when a line of verse is spoken aloud.
For example, a poem in iambic pentameter would by definition have this pattern:
| x ‘ | x ‘ | x ‘ | x ‘ | x ‘ |
Yet, few poems would conform unvaryingly to this pattern throughout. In fact, doing so would yield a rigid and stilted work. Sidney's achievement was to play natural speech rhythms against the abstract framework of iambic pentameter so that the tension between the two could be manipulated for poetic effect. Thompson discusses poem 5 of Astrophel and Stella:
| x x | ‘ ‘ | x ‘ | x ‘ | x ‘ |
| It is | most true | that eyes | are formed | to serve |
| x ‘ | x ‘ | x x | x ‘ | x ‘ |
| The in- | ward light, | and that | the heav'n- | ly part |
| ‘ x | x ‘ |
| Ought to | be King |
The deviations from the abstract pattern puts extra emphasis on phrases such as "most true". The deferred stress of the pyrrhic "It is" makes "most" emphatic in a way that reinforces Sidney's meaning. An equally long deferral of stress also highlights "heaven", as is appropriate. And the inverted first foot, trochaic rather than iambic, of "Ought to", emphasizes what Sidney is saying: "this OUGHT to be the case [but is not]". These effects are possible not only because the natural stress patterns of English would emphasize those words, but because the abstract metrical pattern is manipulated to generate tension that puts those words in greater relief.
This sort manipulation requires a flexible substructure. The reason iambic pentameter works so well is not simply that it mimics the natural rhythms of English speech; it's that iambic rhythms, at least in English, reassert themselves even when they're not rigidly adhered to. By contrast, patterns of trochees, anapests, and dactyls tend to be fragile. Take Blake's famous trochaic lines:
| ‘ x | ‘ x | ‘ x | ‘ |
| Tyger! | Tyger! | burning | bright |
| x x | ‘ x | x x | ‘ |
| In the | forests | of the | night |
That second line could so easily be scanned as anapestic:
| x x ‘ | x x ‘ |
| In the for'sts | of the night |
Only the fact that it's surrounded by fairly regular trochaic lines allows us to scan this line as trochaic as well. But this regularity is a necessity; a trochaic poem of any length that attempts the sort of tension that iambs allow would soon be very difficult to sustainedly hear as trochaic. As a result, iambs predominate in English poetry. Shapiro and Beum estimate that some 70% of English poetry is iambic.
One possible reason we hear alternating patterns of unstressed and stressed syllables as iambic rather trochaic is that generally, there is a brief pause after a stressed syllable. In trochaic meters, the pause after the stress works against the pattern of the whole line, as the pause occurs in the middle of the foot. In iambic meters, on the other hand, the pause of the stress works naturally with the pause between feet. This makes iambic meter supple and flexible in a way that other patterns aren't.
Why Pentameter? Part I: Strictly Formal Considerations
The first sonnet of Astrophil and Stella is in alexandrines. Yet the default meter of English poetry is pentameter rather than hexameter or tetrameter. Why? Because pentameter is long enough to be flexible, yet short enough not to fall naturally into divisions. Sidney's poem itself has strong caesuras in most lines:
Loving in truth || and fain in verse my love to show
That she, dear she || might take some pleasure of my pain
Iambic trimeters would be rather choppy. They can, of course, be used to great effect precisely for that reason, as in Theodore Roethke's small masterpiece:
The whiskey on your breath
Could make a small boy dizzy;
But I hung on like death:
Such waltzing was not easy.
But the trimeters work only because of the context: a waltz. The deviations from regular meter capture the drunken missteps well, as in the hypermetric endings "dizzy" or "easy". It's hard to imagine this working as a default to hold capacious variations the way pentameter does.
Iambic tetrameter does give iambic pentameter a run for its money. One of the most justly celebrated poems in the language is in iambic tetrameter:
Come live with me and be my love,
And we will all the pleasures prove,
That Valleys, groves, hills, and fields,
Woods, or steepy mountain yields.
Another iambic tetrameter is simply a Marvell:
Had we but world enough and time,
This coyness, lady, were no crime.
We would sit down, and think which way
To walk, and pass our long love’s day.
The question of why pentameter feels more natural than tetrameter is hard to answer definitively. The fact that pentameter is by definition more capacious probably has something to do with it. Both Marlowe's and Marvell's poems are urgent invitations to love, and pentameter would probably feel too languorous. But a relatively shorter line, while working great when the poet is eagerly soliciting the favors of his inamorata, is somewhat less effective as an all-purpose solution.
Why Pentameter? Part II: Historical Considerations, aka the Real Answer
The popularity of sonnet sequences in the 16th century also boosted pentameter over tetrameter, because the ten-syllable pentameter line more closely approximated Petrarch's hendecasyllables. And of course, Marlowe's mighty line firmly established blank verse, i.e., unrhymed iambic pentameter, as the default for the Elizabethan stage:
Was this the face that launched a thousand ships,
And burnt the topless towers of Ilium?
Sweet Helen, make me immortal with a kiss:
Her lips suck forth my soul, see where it flies!
Come Helen, come, give me my soul again.
Chaucer and Wyatt are, frankly, too remote and too hard to scan, respectively, to claim that their influence established iambic pentameter as the default for English poetry. But Sidney and Marlowe are not. They provided enduring exemplars of what English poetry should sound like, and they established iambic pentameter as the default.
The question of what English poetry should be like was a real one during the 16th C. Poets such as Thomas Campion argued that English should use quantitative meter. Poets such as Sidney (An Apology for Poetry) and Samuel Daniel (A Defense of Rhyme) countered that accentual-syllabic, rhymed lines were more natural to the language. It was not a given that English poetry would end up with iambic pentameter as the default. It just so happened that the 16th C. was a time that lent itself to the establishment of a default:
- It was a time of tremendous change in the language itself, thanks to the Great Vowel Shift.
- The establishment of printing allowed a certain degree of standardization as well.
- The great success of Spain in establishing New World colonies, alongside the rivalries between England and Spain, led to an upsurge of nationalist sentiment that made it imperative to establish a cultural poetics for England; this caused poets to explicitly discuss questions about what an English poetry and poetics would look like.
- When Sidney and Marlowe established successful examples of what English poetry could accomplish, the meter they used quickly became the standard that subsequent poets adopted.
It needs to be stated that poets didn't consciously set out to imitate Sidney and Marlowe in choosing iambic pentameter for poetry and drama respectively. Iambic pentameter in English has a well-documented history before those poets. The point here is that iambic pentameter has certain built-in advantages for poetic effects in English. These poets were writing when Modern English was in the late stages of its emergence as something drastically different from Middle English. As English poets sought appropriate vehicles for poetic expression in the wake of the Great Vowel Shift, Marlowe and Sidney showed that iambic pentameter was an assertive, flexible, and supple pattern that allowed poetic achievement of a high order in what was practically a new language. Their example made that meter the default for English poetry.
We tend to think of Shakespeare as the default poet for English. And indeed, his achievements in iambic pentameter are better-known (some might argue "better" tout court) than those of Sidney and Marlowe with that same meter in poetry and drama respectively. But Sidney and Marlowe were those who first showed the tremendous possibilities that iambic pentameter offered English poetry, and they established it as the default that endured until Ezra Pound: "To break the pentameter, that was the first heave".
Edit based on comments: Continuity between Middle English and Modern English meters
As @PeterShor points out, entirely correctly, in the comments, Middle English poets such as Chaucer and Gower used iambic pentameter extensively. The practice of iambic pentameter could have continued throughout the Great Vowel Shift that marks the transition from ME to ModE. Poets could have imitated what they saw Chaucer doing, even if Chaucer's own scansion became impenetrable to them as the shift occurred. This argument deserves more consideration than is practical in this already overlong answer, but there is another question on this site that takes up the issue.
References
- Brogan, T. V. F. "Iambic." The New Princeton Encyclopedia of Poetry and Poetics. Ed. Alex Preminger and T. V. F. Brogan. 3rd ed. New York: MJF Books, 1993.
- Brogan, T. V. F., and S. S. Bill. "Iambic". The Princeton Encyclopedia of Poetry and Poetics. Ed. R. Green, S. Cushman, and C. Cavanagh. 4th ed. Princeton: Princeton UP, 2012. https://search.credoreference.com/content/entry/prpoetry/iambic/.
- Shapiro, Karl, and Robert Beum. A Prosody Handbook. New York: Harper and Row, 1965.
- Thompson, John. The Founding of English Metre. London: Routledge, 1966.
