LLMs In Search of A Problem

Recently, I was listening to a podcast where the host was talking about their new AI side project. Their workflow was to take all their podcast episodes, run them through an AI to transcribe the audio, then feed those into an LLM so that it could could find topics that come up often. The LLM would then generate a blog post expressing the host's thoughts on the subject.

I don't understand why he wanted to do this, and I'm pretty sure he didn't either.

If the goal here was to take existing material and produce low effort content for his blog, I've got some bad news. This low effort is also going to be low quality, most likely defeating any purpose for actually having content on your site in the first place.

Even the best possible articles generated aren't going to be worth reading for a few reasons. First, despite all the advances in technology, the old rule of "garbage in, garbage out" still applies. The starting point here is going to be shit that you just said, off the cuff, on a podcast. So let's be real, it's not like the inputs here are going to be the highest quality, most clear, concise, well structured or well thought out.

I'm not even going to get into the fact that transcriptions are not always accurate, that it won't know the difference between sarcasm, jokes, and your genuine thoughts. I have no clue if the model will accurately discern things that you're saying and your guest or co-host was saying.

But at the end of the day, why are you trying to make a post about this in the first place? Presumably, because you have something to say. Maybe it's something you just want to get off your chest. Maybe it's something that you see coming up again and again and want to formulate and share your thoughts about it.

The whole magic of writing is that it forces you to engage with your thoughts, craft your arguments, distill your ideas, and really figure out what you're trying to say. The post is really just a byproduct of that. The value for the reader is that you're providing them with artifact so they can benefit from the results of this process.

Maybe there's value to having all those transcripts so you can search through and find all the times you talked about a given topic. Maybe you just want to make sure to include the various hot takes you've shared.

But you should be writing the article because something is on your mind, because you have a desire to share it. If you need an LLM to go through and summarize things you've said about it, that's probably just a sign that it's not worth you posting an article about in the first place.

To me, this looks like a case of trying to find something to throw AI at, instead of having a real need and using AI as a tool to fulfill it.