The Flesch-Kincaid grade level for each of the Presidential debates in 1960, 2008, 2012, 2016 and 2020. Things are getting worse, not better.
Word Analysis of 2020 U.S. Presidential Debates
Trump vs. Biden / Harris vs. Pence
On these pages, I explore word usage in the 2020
U.S. Presidential debates between Donald Trump and Joe Biden and in the Vice-Presidential debate between Mike Pence and Kamala Harris.
Impatient? Skip to the full word analysis of the 1st U.S. Presidential Debate between Donald Trump and Joe Biden.
Formal debates present a unique opportunity to compare the speech patterns of candidates. The debate's format is controlled — though the debates have been thusfar unruly — and each speaker is subject to the same question (in principle) and is given the same amount of time to respond.
That being said, the dynamics of a debate can be greatly affected by one candidate, who can hijack the conversation and use interruptions to influence their opponent's natural style. Thus, the results of the debate analysis cannot be taken out of the context of the debate.
It's important to stress that this analysis is structural and not semantic. I look in detail of how things are said rather than what is said. However, there is a strong connection between the use of specific words (e.g. pronouns) and the speaker's inner dialogue (Your Use of Pronouns Reveals Your Personality).
I use transcripts from rev.com and explore themes such grade level, readability, sentence size, parts of speech usage, pronoun usage, unique and shared words and use of concepts. And I cannot help but draw some word clouds.
The analysis is fully automated and uses the Natural Language Toolkit for tokenizing, tagging and chunking. All data and word lists (tagged and chunked) are available for download in plain-text format — you are welcome to use these files in any manner.
Results from past years are available: 2008 debate analysis, 2012 debate analysis and 2016 debate analysis. Each year's analysis is a collection of stand-alone pages. For a given year, each of the three Presidential debates and the Vice-Presidential debate results are structured identically.
The analysis for the 2020 debates uses a different part of speech tagging engine (NLTK) than used in previous years (Brill tagger). Keep this in mind if you're comparing the 2020 results to past years.
Transcripts by the Washington Post for each debate were parsed to extract sections for each speaker, chunk the text into sentences and words, tag each word with its part of speech (tagging), and identify noun phrases (chunking).
The tagged and chunked transcripts are analyzed to determine
- • reading ease and grade level using the Flesch-Kincaid metric
- • word frequency distribution for each candidate
- • sentence size and proportion of unique words
- • words exclusive to a candidate and those shared by both candidates
- • pronoun use by person, gender and count
- • frequency of concepts, as defined by part of speech pairings (e.g. noun/verb)
- • complexity of noun phrases
- • word clouds for a variety of word lists extracted from the transcripts (e.g. all nouns unique to Trump)
I attempt to quantify the overall complexity and repetition by a metric
I call the Windbag Index, which is a product of 8 terms
each measuring uniqueness in different aspects of speech (more about Windbag Index).
A full description of each of the steps in the analysis is
available in the detailed methods section.
The analysis has some limitations.
Results and Commentary
Each debate analysis report contains a lot of data but is shown in exactly the same format, which should help you with making comparisons between debates. To start, you may find these elements the most interesting
Results are shown in a tabular format. From each table you can download the word list used to generate it. This makes it easy to, for example, grab all the adjectives used by Biden or all the verbs that Trump used that Biden did not use.
detailed results – tables, word clouds and commentary
Analysis of Donald Trump vs. Joe Biden (1st debate)
Analysis of Donald Trump vs. Joe Biden (town halls)
Analysis of Donald Trump vs. Joe Biden (3nd debate)
Analysis of Donald Trump vs. Joe Biden (combined debates)
Analysis of Mike Pence vs. Kamala Harris
Visualizing the Debates
Each debate is visualized using tables and word clouds — there's obviously a ton more than can be done. The word clouds visually show the words and their frequency and tables provide detailed statistics. You can download each word list directly from the tables.
tables & basic word clouds
Word usage tables describe the structural characteristics of speech by frequency of words, sentence size, proportion of unique and exclusive words and breakdown of words by part-of-speech • see example
Word clouds for each candidate, categorized by parts of speech.
Word clouds, categorized by ownership.
Word clouds for concepts based on part-of-speech pairs.
Candidates's Word Usage Profiles
Below are a few of the tables available in the full results section.
Readability and Grade Level
The Flesch–Kincaid readability tests are designed to indicate how difficult a passage in English is to understand. There are two tests, the Flesch Reading Ease, and the Flesch–Kincaid Grade Level.
The Flesch-Kincaid grade level is actually a periodic quantity.
Sentence size with and without stop words.
Part of Speech
Total and unique nouns, verbs, adjectives and adverbs. The parts of speech are identified by their Penn Treebank tags.
English has many pronouns. Here is an accounting of pronoun use by 1st (e.g. I, we, our), 2nd (e.g. you, yours) or 3rd (e.g. he, she, his, them) person.
These tables break pronouns by interesting contrasts. For example, the ratio of singular to plural 1st person pronouns reveals the use of "I/my/myself" vs. "we/our/ours".
The Windbag Index is a compound measure that characterizes the complexity of speech. A low index is indicative of succinct speech with low degree of repetition and large number of independent concepts. A large number suggests a stream of repeating words.
Word clouds below are colored by part of speech:
Words exclusive to Joe Biden (not spoken by Donald Trump) in the first debate, colored by part of speech.
Words exclusive to Donald Trump (not spoken by Joe Biden) in the first debate, colored by part of speech.
Word clouds below are colored by speaker:
All nouns in debates, colored by contributing speaker (Trump: red, Biden: blue, spoken by both: grey).
All verbs in debates, colored by contributing speaker (Trump: red, Biden: blue, spoken by both: grey).
Let's hope that after things get worse, they get better.
Content of word list archive and data structure syntax is described in the methods section.
Donald Trump vs. Joe Biden (1st debate) transcript word lists and tag clouds data structure
Donald Trump vs. Joe Biden (town halls) transcript word lists and tag clouds data structure
Donald Trump vs. Joe Biden (3nd debate) transcript word lists and tag clouds data structure
Donald Trump vs. Joe Biden (combined debates) transcript word lists and tag clouds data structure
Mike Pence vs. Kamala Harris transcript word lists and tag clouds data structure
1 Oct 2020. Expanded pronoun analysis to include interesting contrasts.
30 Sep 2020. Results and commentary of 1st debate are posted.
29 Sep 2020. It starts now.
26 Sep 2020. Working on preparing the system for Tuesday's debate.