The Worst AI Metric

The “how many r’s in strawberry” test for AI intelligence is dumb.

As a writer to write a quality sentence for the book they’re working on, and as they’re writing—or typing—suddenly scream at them mid-sentence:

HOW MANY VOWELS IN THAT?!?

First, they’ll be very annoyed. But more importantly, you will have stopped them from creating their sentence.

Human’s can’t output at the same time they’re thinking about how to do so.

Ask them—in the middle of a sentence—how many words they’re using have an even number of characters. Or how many rhyme with “cow”. Or how many r’s the sentence contains, and they’ll have no idea whatsoever. And you’ll have ruined what they were saying.

So the question is: Do you want a sentence, or do you want information about a sentence? You need to pick one.

When we hire a writer, or a speaker, or an AI, we’re hiring them for the content they produce, not for trivia about that content.

So let’s not judge AIs too harshly for something we somehow forgot humans can’t do either.

1. There’s also the fact that modern AIs work off of tokens, not words, so you’re basically asking an alien about ice cream. It’s not as important as people think it is.