#190 AI Historian
Often when we think of histories, we think of big, sweeping things like the Napoleonic Wars or the Roman Empire. There are a lot of interesting histories that are much smaller and deal with more recent history, whose participants are still alive. In the business world, examples include Hatching Twitter and Bad Blood.
As I understand these things to work, an absolutely essential part of this is interviewing many, many people. Yes, the actual act of writing is significant work, but the research that goes into e.g. Bad Blood’s 400 pages probably took years. The investment is huge and only happens in a very small number of cases.
This cost means there are interesting stories that won’t ever get a book. I spent 12 good years at Indeed. Nobody’s going to write a book about the history of that company. No editor is going to believe it would have the audience and sales to justify paying someone for two years (and their expenses) to do the necessary research. I can think of a couple hundred people who would buy that book (maybe just to see if they were in it), but that’s a few orders of magnitude short of what’s needed. And nobody’s going after the movie rights either.
Now imagine this for any substantial company, non-profit (e.g. a history of the ACLU), university, government agency, and any similar organization. Maybe 0.01% of them have their stories written, but a lot of more of them have stories that are worth recording and interesting to someone.
Recent technology makes this much more feasible. LLM-based bots can conduct interviews, synthesize them, and write the text. In some ways they might be better at it because they can scale better. A really thorough human researcher might interview a couple of hundred people; I think often it’s more in the mid double digits. A bot, on the other hand, could interview everyone who’s willing.
The “willing” part could be a bit of sticking point. A bot would probably get lower response rates, and respondents might have more misgivings about providing sensitive information. How can you be sure the bot understands and will properly handle off the record information, especially if that information is sensitive or NDA-ed? I don’t have the answers, but I’m optimistic that this is answerable.
Today these books need audiences of hundreds of thousands or more to justify the effort needed to create them. What if your minimum viable audience was just hundreds?