When I worked at the U.S. Library of Congress, one of my favorite activities was to wander through the stacks. “The Stacks” were where we kept the books; not all the books, as the Library has more than 25 million of them, but a certain percentage. It used to be that all Library of Congress employees had stack access; older colleagues would tell me stories of reading inside the stacks on their lunch breaks. That changed over time for security reasons. By the time I arrived, only a fraction of Library employees had such access. I was one of them.
I have many fond memories of those labyrinthine and cavernous corridors lined with books, basking in centuries of accumulated knowledge. I vividly remember one day stumbling into an entire section of books on Napoleon, bookcases as far as the eye could see overflowing in all directions in dozens of languages. As someone who once tutored high school students in AP European History, it was a sobering moment. Surely, I knew more about Napoleon than 95 percent of my fellow Americans. Yet, amid a temple of knowledge, I was awestruck by how much there still was to learn. Suddenly, I felt as though I knew nothing at all.
When people ask me about the “lessons” we can draw from history, my first answer is to say that when done right, history should teach us humility. There is an endless amount to know about the past, and even the most gifted and brilliant among us only knows a small fraction of it. History has the capacity to humble us about how little we understand about our world.
The value of studying history, then, comes from the process of discovery. Searching through an archive, researching old newspapers, or reading the books of those who came before us forces us to grapple with something we never knew. It is from that friction that we learn, grow and forge new connections in our brains. We call that process “education.”
The A.I. applications available to us today promise to remove that friction. Large Language Models, such as ChatGPT, and “Answer Engines,” such as Perplexity, execute the discoveries for us. They sift through billions of pieces of online text, extract relevant information, and synthesize it into an output that simulates what a human would create. We, the users, are separated from the process of finding unexpected connections or uncovering things we never knew. Retrieval becomes a proxy for education.
On a recent Saturday, then—with the books about Napoleon inside the Library of Congress in the back of my mind—I decided to experiment with Perplexity and Google Gemini and see what outputs they would generate about Napoleon. Napoleon had come up for me in other contexts recently; when I was in Latvia with the U.S. Department of State, many of the young men I met told me they were fascinated by Napoleon and regularly searched for content about him on social media. I could imagine those high school students now using A.I. applications for their searches.
For those unfamiliar, Perplexity is an “Answer Engine.” It combines the functionality of a search engine with the text generation of a large language model in order to produce real-time answers to user questions. Those answers come with citations, i.e., URLs that can be clicked on to learn more. Google Gemini is a “multimodal” A.I. model, meaning it functions like a large language model while extending beyond language to generate images, audio, video and code. Gemini can retrieve information from across the Web and synthesize it into an array of outputs, include reports, illustrations and webpages.
My question to Perplexity was, simply, how many books are there about Napoleon? The answer engine generated a response that stated the number of books to be 300,000 titles. According to the citations, that statistic derived from a thread on the social network Reddit. The Reddit thread, in turn, cited a website called UKEssays.com, an “essay mill” that allows British students to purchase essays on subjects they need for school exams. UKEssays.com, in turn, seemed to have taken the statistic from a book called Napoleon for Dummies, published in 2005 and written by J. David Markham, a high school and community college teacher in the U.S. Finding Napoleon for Dummies in Google Books, I saw that, indeed, Markham made his claim on page 1 of chapter 1, stating that more than 300,000 unique titles existed about Napoleon. Where he got that number from was unclear, as he cited no sources.
Perplexity also listed in its sources a 2023 article published in the online magazine The Conversation written by Ben McCann, an Associate Professor of French Studies at the University of Adelaide in Australia. In his article, Dr. McCann cited businesswoman-turned-Napoleon enthusiast Margaret Rodenberg who in 2021 claimed there were 60,000 books published about the Emperor General. It’s unclear where she got her statistic from, as she also did not include a source. Even though Perplexity listed The Conversation article among its sources, it did not integrate its information into its output. In other words, Perplexity’s output used the number 300,000 but did not include the possibility that the actual number could be 80% fewer.
I, then, went to the Library of Congress website and conducted a Library of Congress catalog search. That search returned 5,662 books with Napoleon in the title. Next, I went to the British Library website and searched the British Library catalog, which returned 1,852 books with Napoleon as the subject. I, then, ran an Amazon search for Napoleon in “Books,” which yielded over 40,000 results; however, that included authors with either the first or last name of Napoleon. It seemed, then, that there could be either 1,852 | 5,662 | 40,000 | 60,000 | or 300,000 published books about Napoleon, depending on the source you looked at and who you believed. That was a very wide discrepancy!
The value of studying history comes from the process of discovery. At every page-turn, we grapple with something we never knew. It is from that friction that we learn, grow and forge new connections.
A.I. applications have a penchant for generating inaccurate or fictitious information, so this exercise was a good reminder to treat their outputs with healthy skepticism. But the point of the exercise was not to impugn the output generated by Perplexity, rather to begin to understand how Perplexity arrived at such an output in the first place, an aspect of what I consider to be “A.I. Literacy”—understanding how A.I. models generate their information.
In this instance, the pathway seems to have been:
The book Napoleon for Dummies, published in 2005, makes an unsourced claim;
A website, UKEssays.com, repeats the claim in 2021;
A Reddit post from 2021 cites UKEssays.com as a credible source;
In 2025, Perplexity cites Reddit, UKEssays.com and Napoleon for Dummies in order to generate a response it determined to have a high probability of accuracy.
The notion of probability is significant, as large language models rely heavily on probability in order to function. LLM’s such as ChatGPT have been trained on billions of pieces of text—including from Reddit, Wikipedia, billions of web pages and millions of books—to predict how human language looks and sounds. In overly simplistic terms, these models use probability (math) to determine the likelihood that certain words should appear in a certain order. For example, based on its training, ChatGPT recognizes that the word “Elon” is likely to be followed by the word “Musk,” which would be followed by the acronym “CEO,” which would be followed by the word “Tesla.” Thus, if I were to prompt ChatGPT with the question “Who is Elon Musk?” it would output “Elon Musk is the CEO of Tesla” with a high probability of accuracy.
The models perform similarly for claims that reside within their training data or the data they can search online. If enough “credible” sources reinforce a claim, the model will assess that claim as having a high probability of accuracy and integrate it into its output. This can be thought of as a “logic of consensus,” and it is similar to the logic underpinning Wikipedia. As I explained in chapter three of History, Disrupted, Wikipedia’s guidelines presume that if enough credible sources assert something to be true, that consensus becomes part of the Wikipedia entry—regardless of its veracity. For example, if several credible sources said the sky was green in 1885, Wikipedia’s entry for 1885 would include the sky being green—even if it was, in fact, blue. A consensus of credible sources becomes the foundation for making truth claims.
Hopefully it is clear why such an approach can be problematic. In the Perplexity example, the “answer engine” did not have the Library of Congress catalogue among its training data. As the largest library in the world, in existence for 225 years, with more than 175 million items, and with a mandate to collect books in all languages from across the globe, the Library of Congress likely has the nearest accurate count of the number of books published about Napoleon. Napoleon died in 1821, meaning it has only been 204 years since his passing. For there to be 300,000 published books about him would mean that nearly 1,500 books each year would have had to be published since his death—which seems unlikely. Even 60,000 total books would mean 294 Napoleon books published each year, on average, since his death—conceivable, but also not likely. That two Napoleon enthusiasts are so widely discrepant in their math also raises questions; presumably two experts on the same topic would be in closer alignment. Both their statistics seem apocryphal.
By relying on Reddit, UKEssays.com and Napoleon for Dummies to generate an output—and not the Library of Congress or the British Library—Perplexity illustrates the limitations of a model trained on the lowest hanging fruit of the Internet. It also illustrates how a consensus of sources can collapse under scrutiny. Sometimes a single source is more authoritative than a wide array of sources. It also demonstrates the circularity of how Internet claims become accepted truths; it is often the case that companies or those with commercial or political agendas purposefully start threads on Reddit that link back to their products or services as a way to drive traffic. It’s conceivable that the Reddit post that linked to UKEssay.com was, in fact, created by someone associated with UKEssay.com. The model interpreted the Reddit post and the UKEssay.com site as two independent sources without factoring in the possibility that they could be the same. Finally, Perplexity seemed to accept that a claim published in a book had a certain weight of “truthiness” to it, even though it had no evidence to back it up. In its entirety, this small example offers a window into how various actors can flood the public sphere with false or misleading information by planting claims inside books, websites and social media posts—which are then scraped, aggregated and disseminated by A.I. models. The expectation is that users would accept the A.I. outputs at face-value without digging deeper.
A.I. Literacy is part of our defense against such blanket, non-critical acceptance.
Nefarious actors can flood the public sphere with false information—which is then aggregated and disseminated by A.I. models. “A.I. literacy” is part of our defense against blanket, non-critical acceptance.
Next, I asked Google Gemini to “Tell me about Napoleon” using its “Deep Research” mode. In response, Gemini generated the following plan:
Research Websites
(1) Research the early life of Napoleon Bonaparte, including his Corsican origins, military education, and his rapid ascent during the French Revolution.
(2) Summarize Napoleon’s major military campaigns and strategic innovations, identifying his most significant victories and decisive defeats.
(3) Investigate the political and administrative reforms enacted during the Consulate and the First French Empire, focusing specifically on the creation and lasting impact of the Napoleonic Code.
(4) Detail the transition from the French Republic to the First French Empire, including the political rationale and public reception of his self-coronation as Emperor.
(5) Analyze the primary factors leading to Napoleon’s downfall, including the Russian campaign, the Peninsular War, and the final sequence of events involving Elba, the Hundred Days, and St. Helena.
(6) Explore the long-term legacy of Napoleon on European politics, warfare, and national identity, particularly concerning the spread of revolutionary ideals and the rise of nationalism.
(7) Find contemporary and modern historical evaluations and critiques of Napoleon’s leadership, character, and overall impact on global history.
Analyze Results
Create Report
I could have refined the plan (i.e., fine-tuned the model’s weights and parameters) but for this non-scientific exercise I chose not to. Gemini then began its output generation, which took several minutes and resulted in a 13-page report, titled “Napoleon: A Comprehensive Study.” The report is available in my Google Drive.
To inform its report, Gemini cited 21 websites:
1. Fondation Napoléon, France, 2008;
2. National Gallery of Victoria, Australia, 2012;
3. Encyclopedia Britannica, United States, 2025;
4. History Hit, United Kingdom, 2021;
5. The Cultural Experience, a tourism company in the United Kingdom, 2018;
6. Encyclopedia Britannica, United States, 2025;
7. A European clothing store;
8. Shannon Selin, a Canadian public official turned amateur historian, 2020;
9. Lumen Learning, from a course called “History of Western Civilization II”;
10. An English-language Wikipedia page about Napoleon, created in 2001, last updated in 2025, with a long edit history, including from suspended accounts;
11. Fiveable, an online AP study guide, AP European History Review, United States;
12. An English-language Wikipedia page on the Concordat of 1801, created in 2003, last updated in 2025;
13. Warfare History Network, a part of Sovereign Media and Homestead Communications, United States, 2000;
14. EBSCO research starters, 2023;
15. EBSCO research starters, 2023;
16. Cambridge University Press, a one paragraph summary of 470-page book, United Kingdom, 2022;
17. National Army Museum, United Kingdom;
18. MyTutor, an hourly tutoring service for students, United Kingdom;
19. Age of Revolution, a project of Waterloo 200 Ltd, United Kingdom, 2000;
20. HISTORY.com, 2022;
21. Study.com, an online resource for students, United States, 2025.
Continuing on the theme of A.I. literacy, after reviewing the report I was struck by several observations:
Regardless of how many thousands (or tens-of-thousands) of books on Napoleon have been published, Google Gemini, in this instance, did not retrieve or cite information from any of them directly. It pulled information from a summary of one Cambridge University Press book, and drew on two Wikipedia entries, which themselves cited books, but no text from any books was scraped directly.
The e-history it did cite, for the most part, did not, themselves, cite any sources. Presumably Study.com, HISTORY.com and History Hit relied on information about Napoleon from Wikipedia or Britannica, as well as published books or articles. But barely any of these e-history sites used footnotes. Thus, while Gemini deemed its sources as “credible,” the lack of citations undermined their credibility.
Gemini seemed to assume that I was either a student or a history buff, based on the sources it pulled and the manner by which it generated its report. In other words, there seemed to be a bias in the model itself for who would ask such a question about Napoleon. The model seemed to assume as a baseline, without any fine-tuning, that such a question would likely be asked by a student or a history buff—even though a journalist, an elected official, a genealogist, a filmmaker, a military officer, or an attorney might also want information about Napoleon. Napoleon as a subject was assumed to be for either students studying for an exam, or enthusiasts who want to go on heritage tours with other enthusiasts.
All the sources were “Western,” Anglo-American, and white. It is a verifiable fact that people of color in countries beyond the U.S., Canada, Australia, France and U.K.—for example Egypt—have written about Napoleon; however, they were not reflected in the citations. As well, none of the world’s top scholars on Napoleon and French History (or military history) were included. Instead, the sources were amateur historians, military enthusiasts, tutoring sites, edutainment sites, encyclopedia authors, EdTech contributors and popular writers.
It is also a verifiable fact that The New York Times, Washington Post, NPR, BBC, Oxford University Press, Foreign Policy and numerous university websites have articles and resources online about Napoleon. Yet, none were cited for this report. This, again, suggests an assumption in the model for who would be asking such a question and the grade level / sophistication of an appropriate response.
The source materials were all text: no videos, podcasts, or artwork (there are many paintings, drawings and visual depictions of Napoleon).
The output largely ignored the 150 years of knowledge produced about Napoleon prior to this century. It also ignored the context of why certain information about Napoleon was created during certain periods of time. For example, in 2023 there was a Hollywood film about Napoleon directed by Ridley Scott. The film was announced in 2020 and for the next several years inspired numerous articles, books, and op-eds, including The Conversation article. Five years earlier, the 200th anniversary of the Battle of Waterloo in 2015 also prompted an array of articles, exhibits and webpages on Napoleon, especially in the United Kingdom. Gemini seemed to treat its sources as evergreen, whereas a more sophisticated analysis would recognize them as context-dependent, in conversation with cultural moments such as a film release or major anniversary.
The model weighted sources and information about Napoleon’s exploits in war and politics more heavily than it did questions around gender, domestic life, identity, etc. As such, its claim to be “comprehensive” was, in fact, quite selective.
It’s worth re-stating that this Saturday afternoon exercise was not a substitute for a thorough, longitudinal study. It was a thought experiment, meant to tease out some ideas for wrestling with these emerging technologies:
As technological achievements, Perplexity and Gemini function remarkably well. In a very short amount of time, the applications search the Web, retrieve information from sources, make decisions about relevancy, synthesize information, and generate long-form responses that are structured and coherent. Over time they will become more accurate and increasingly customizable. Users will be able to fine-tune the citations, style, sophistication and tone in order to achieve better outputs.
Despite those improvements, as producers of original analysis and scholarship, the applications fall short. This is partially because the sources they rely on are of mixed quality; partially because they do not conduct any interpretation of those sources; and partially because they do not generate incisive responses that dig below the surface of what exists on the Web and social media. They do not identify gaps and do not express awareness of what they do not know. If I was back in the classroom, and a student submitted the essay generated by Google Gemini, s/he would receive an ‘F’ – not because they used the application, but because they failed to demonstrate any critical thinking about the application.
This creates an opportunity for new types of A.I literacy lessons. How do A.I. models generate their outputs? What types of sources do they rely on? What types of sources do they omit? How does historical consensus get formed? How can our current information ecosystem be exploited to propagate disinformation? Engaging students in these types of conversations creates the friction that is essential to learning. It reconnects them to the purpose of historical study: making surprising connections and grappling with new information. If done well, such lessons could, potentially, lead to positive educational outcomes, particularly for 21st century learners.
Ultimately, knowing the number of books published about Napoleon is a largely inconsequential statistic that makes little material difference to the world. But understanding why our A.I. technologies cannot generate a good answer to that question? That feels like something worth knowing.
Have an enlightening week,
-JS
Yeah. As someone who has a PhD from the olden days, I spent a lot of time in the stacks at Columbia and Cornell. They were both comforting and scary, because there were SO MANY BOOKS! Now that I use AI I am pretty constantly coming up against its limits, which are the limits of its training data. Reddit sources are a perfect example of that. How long will it take us to reach a decent level of AI Literacy?
This is an excellent piece, Jason! I’ll be reading it with my middle schoolers. They’re already grappling with how much their peers use ChatGPT and other A.I. for schoolwork.