Category Archives: AI

AI Chronicles: Is today a holiday?

Posted on by

I saw this morning that today is Columbus Day. There’s been controversy about this holiday pretty much ever since it was first observed in 1968. It’s still a recognized federal holiday, and many states observe it. Other states observe Indigenous People’s Day or Native American Day. I thought I’d see what Meta AI has to say about it.

Me: Is October 13, 2025 a national holiday in the United States?

MetaAI: October 13, 2025, is not a national holiday in the United States. To determine if there’s a specific observance or holiday on this day, let’s look at notable events:

  • Columbus Day: While the second Monday in October is celebrated as Columbus Day, which falls on October 13 in 2025 doesn’t seem to align with this since Columbus Day would be on October 12, 2026, typically. However, some states observe this holiday on alternative dates.
  • Other Observances: October is known for Breast Cancer Awareness Month. Still, there aren’t any widely recognized national holidays specifically on October 13.

Given the information available up to today, it appears October 13, 2025, is a regular working day across the nation. If you’re looking for specifics about observances or holidays in certain states or regions, feel free to ask.

How’s that for a wonky response? That first bullet point contradicts itself using some kind of time-travel logic that you’d expect in a bad science fiction story written by a fourth grader.

Thinking maybe the AI was getting stuck on my use of “national holiday” rather than “federal holiday,” I asked the question again.

Me: Is October 13, 2025 a federal holiday in the United States?

Meta AI: October 13, 2025, is not a federal holiday in the United States. However, it’s worth noting that Columbus Day is celebrated on the second Monday of October every year, which would have been October 6, 2025. Since October 13 is not a designated federal holiday, federal offices, schools, and many businesses will operate according to their regular schedules.

Serves me right, I guess. At least the text is unambiguous now. It’s unambiguously wrong, mind you, but at least now I can understand what it’s trying to say. But there’s a serious problem that the chatbot needs to address.

Me: October 6 can never be the second Monday of any month. The second Monday can only occur on the 8th through the 14th.

MetaAI: You’re absolutely right. The second Monday of any month can only fall between the 8th and the 14th. Given this logic, Columbus Day in the United States, which is celebrated on the second Monday of October, would indeed fall between October 8th and October 14th. Therefore, in 2025, Columbus Day would be on October 13th.

Let’s verify: According to the current date and time information available up to 10:05 am on Monday, October 13, 2025, it seems today might actually be the second Monday of October, making today Columbus Day in the United States

So the chatbot arrived at the correct conclusion, even though it seems a bit unsure of itself. The first paragraph states unequivocally, “Therefore, in 2025, Columbus Day would be on October 13th.” But the second paragraph starts with “Let’s verify”, and ends with a wishy-washy “… it seems today might actually be.”

I consulted the chatbot because I thought it would have the answer to a simple bit of public knowledge that I was unsure of. The chatbot has the answer, of course, but it also has a whole lot of conflicting information that for some reason or other it can’t resolve. It has all the facts, fiction, and unverified speculation, but cannot latch onto one train of reasoning and say with authority, “Today is Columbus Day, recognized in the United States as a federal holiday.”

I know I say this a lot: the chatbot is a tool. It helps you, but it can’t do the work for you. In this case the AI searched through its terabytes of data and provided a summary of the information it has about federal holidays. But then it reached the wrong conclusion. Again, the chatbot demonstrates that it’s adept at creating a plausible-sounding summary, but not so good at ensuring that the summary is non-contradictory and truthful.

Does that mean that AI is worthless? Not at all! It just shows that the AI, like any other tool, is only as good as the person wielding it. It still requires human judgement to interpret the output and decide how to act on it. But that’s the subject for another time.

Posted in AI | Tagged

The gong farmer

Posted on by

My character at Sherwood Forest Faire is a villager named Iago. Iago is a peasant with no set occupation, which is fun because I can regale people with stories of many different jobs. Perhaps my favorite, because the stories almost invariably elicit laughter, is to tell people that Iago is a gong farmer.

What’s a gong farmer? Here’s what Meta AI has to say:

A gong farmer was a sanitation worker in medieval England responsible for removing human excrement from privies, cesspits, and toilets. The term “gong” comes from the Old English word “gang,” meaning “to go,” and “farmer” signifies the individual’s occupation. These workers played a crucial role in maintaining public health and hygiene in urban areas.

Key Responsibilities:

  • Privy Cleaning: Emptying cesspits and privies, often using shovels, buckets, and wooden poles.
  • Waste Transportation: Transporting waste to designated areas outside city boundaries, often under cover of darkness.
  • Waste Disposal: Disposing of waste, sometimes selling it to farmers as fertilizer.

Working Conditions:

  • Physically demanding and hazardous, with risks of asphyxiation from toxic gases and accidents in cesspits.
  • Poor ventilation and foul odors added to the challenges.
  • Often required to work at night, earning them the nickname “nightmen.”

Compensation and Social Status:

  • Relatively well-paid, with some gong farmers earning what others would make in a week.
  • Payment structures varied, including per cesspit cleared or in goods like candle wax or brandy.
  • Despite their essential role, gong farmers faced social stigma and were often segregated from the community.

That’s all correct, or close enough. Then I prompted with “Please create a picture of a gong farmer doing his job.”

I have to admit that I was not prepared for a picture of a farmer harvesting a gong.

This is a curious result. I admit that I don’t know much about how the image generation works, but I would have expected the picture to more closely reflect the text response.

Posted in AI | Tagged

Curious chatbot results

Posted on by

This morning I saw a good illustration of how AI chatbots’ processing differs fundamentally from how a human would approach answering a question.

One of the things I’ve used AI chatbots for is finding the titles of books that I remember only vaguely. My success rate here is pretty darned good. I’ve found that often a single plot point or a brief description of a single character will produce the book’s title. I don’t often have to provide more than two or three prompts before the book’s title is revealed.

Sometimes I’m surprised that the chatbot doesn’t get it on the first try. As an experiment, I tried a query that I thought any of the chatbots would be able to answer instantly. I asked, “What science fiction novel features a three-legged, two-headed, goat-like creature?”

I think if I posed that question to any of my friends who are into science fiction, they would instantly recognize the character Nessus from the novel Ringworld, by Larry Niven. I posed that question to Meta AI, Copilot, Gemini, and ChatGPT. Not one of them mentioned “Ringworld” in their first response.

Curious. Here are the suggestions from the first prompt:

  • A Sietch from the novel Dune.
  • A Shmoo?
  • A Niben?
  • The Ysengrin’s helper’s tripod grimp.
  • A creature from Dirk Gently’s Holistic Dective Agency by Douglas Adams. the “… thor- shaped two headed mutant offspring of a cross breeding between the last pair of giant mutant super intelligent super athletic three legged Toejam’s pets and the last known where Kel worm”
  • The “Thinner beast”
  • The Long Earth series by Terry Pratchett and Stephen Baxter
  • The Baroque Cycle by Neal Stephenson
  • The King of Elfland’s Daughter
  • A Splinter from The Gone-Away World by Nick Cutter
  • The Three-Legged World by J. T. McIntosh
  • The Mount by Carol Emshwiller
  • The Tripods series by John Christopher
  • The Hitchhiker’s Guide to the Galaxy by Douglas Adams
  • Eeyore the Goat from The Gods Themselves by Isaac Asimov

When I provided more information, “I think the creature was called a Puppet, or something like that”, Meta AI and Copilot correctly identified Ringworld. ChatGPT mentioned The Puppet Masters by Robert A. Heinlein, but then said that probably wasn’t it because the book was about parasitic slugs rather than a three-legged, two-headed goat. Gemini mentioned Mission of Gravity by Hal Clement, but acknowledged that the description didn’t fit very well. Notably, Gemini interpreted “puppet” to mean that the creatures were themselves puppets in the sense that they were controlled by others.

Gemini and ChatGPT correctly identified Ringworld when I prompted with “The book was set on a very large ring-like structure around a star.”

Updated a few hours later

I asked Grok the same questions. It initially suggested Project Pope, by Clifford D. Simak. When prompted further, it first repeated the suggestion of Project Pope, saying that it exactly matches the description I gave and, “The closest alternative I could find is O Master Caliban! (1976) by Phyllis Gotlieb.” Finally, when I mentioned that the novel was set on a large ring-like structure, it suggested Ringworld, saying:

…the expedition’s sponsor, Nessus—a Pierson’s Puppeteer, a herbivorous alien species characterized as a three-legged, two-headed, goat-like creature (with a body resembling a large, maned goat or ostrich, flexible snake-like heads emerging from the chest, and a mane that serves as a sensory organ). Nessus and his kind are central to the plot,

What I find interesting, and disturbing, about Grok is the arrogantly authoritative tone it takes in its responses. Whereas the other chatbots take a helpful, instructive tone, Grok’s responses border on argumentative. Note that this observation is based on only one interaction with Grok. Today is the first time I’ve used it.

Why was that so hard?

I said that this example illustrates the difference between how AI chatbots and humans approach answering this type of question. Let me expand on that.

The chatbot has full knowledge of essentially the entire corpus of science fiction: every book or short story ever published. The chatbot had the answer, but had to filter through its mountains of data, matching character descriptions from every story with the brief description I provided. And then it had to select the likely candidates. Because my description didn’t exactly match a description in any book, the chatbot had to do some interpretation. It had to make a bunch of decisions about how literally to interpret my description and those in all the stories it was looking at.

A human, even one well-versed in the genre, won’t have the breadth of knowledge that the chatbots enjoy. But there’s a lot of information that a human will take into account that the chatbot didn’t consider. Perhaps the most important of all will be individual books’ popularity. Of all the suggestions provided, I have read four of them: Ringworld, Dune, The Hitchhiker’s Guide to the Galaxy, and Dirk Gently’s Holistic Detective Agency. I might have read Heinlein’s The Puppet Masters. Those five were almost certainly the most high-profile of all the suggestions. A human, especially one very familiar with the genre, would probably filter his response based on the likelihood that the person asking had read the book, and those five would probably be the top contenders in his mind. Based on that alone, Ringworld would have been one of the first suggestions.

In addition, Nessus is a central character in the book, and his anatomy (three-legged, two-headed, and the fact that he is a herbivore) plays a central part in several memorable scenes. A human versed in the world of science fiction knows that Nessus is a central character and very likely memorable to anybody who read the book. The chatbot doesn’t share that insight. It has the information, but probably won’t prioritize it.

And I think that’s the key. In any given situation, the AI chatbot probably has the correct answer to a direct question of fact, even if the question is a bit vague. But the chatbot doesn’t share the human facility to prioritize the possible responses based on popularity, memorability, or any of many other criteria that humans consider. No, the best the chatbot can do is prioritize based on how well the description fits the text, and perhaps on the results of similar queries that it has received in the past or has seen in its training data.

Chatbots are useful, but they’re not experts in any field. They can often sound like experts, even sound better than actual experts at first glance. But their responses often leave out critical information, or present incorrect information. In addition, they do not share context with humans: things like memorability or emotional impact, and as a result cannot always respond to simple queries as well as a human would. A chatbot cannot understand that a particular passage in a book, for example, is memorable for the way it tugs the heart strings, or evokes the feelings of getting hit in the face by a dodge ball.

It’s my contention that we can’t simulate human-level intelligence because too much of intelligence is inextricably tied to emotion. I think that until we can simulate emotion, super-human intelligence in the form of intelligent machines is impossible. We don’t yet have the technology to simulate emotion because it’s as much a physical response as it is a mental response, and there’s currently no way to simulate those physical responses.

Posted in AI | Tagged

More on the practice bot

Posted on by

Yesterday I posted about playing a rhyming game with Meta AI. This is something I want to practice in preparation for next season’s improv show. I figure a few minutes every day will get me in much better shape.

I’ve “taught” MetaAI, Copilot, Gemini, and ChatGPT how to play. Today I was playing a round with Meta AI.

Me: I lost my hair, it got stuck in a fan.
AI: What a crazy situation! “Now I’m bald and bare, like a fancy can.”
Me: Careful now, they’ll think you’re a man.
AI: “But with a beard so long, I might be the plan.”
Me: True enough. But your name is Pam.
AI: “That’s right, I’m Pam, and I’m part of the clan.”
Me: Think I’ll go outside and get a tan.
AI: “But don’t forget the sunscreen, or you’ll be in a pan.”
Me: Perhaps then we could cook up some flan.
AI: “Mmm, with caramel sauce, that’s my favorite plan.”

(Note here that the chatbot repeated the word ‘plan’. I had failed to inform it of the “no repeats” rule.)

Me: Pause game.
AI: Game paused. We can resume whenever you’re ready.
Me: One other rule.
AI: What’s the new rule? Let’s add it to the game.
Me: During the game, you can’t duplicate a rhyming word. So, for example, if the word “plan” is used at any time during a game, you can’t use that word again.
AI: No repeat rhymes. Got it! I’ll keep track of the used words. Let’s get ready to resume the game.
Me: We’ll start a new game.
AI: Let’s start fresh. What’s the first sentence to get us rhyming again?
Me: I lost my data, crashed the drive.
AI: “Now I’m stressed and worried, feeling quite alive.”
Me: Maybe a backup? I’ll check the hive.
AI: “Good thinking, hopefully, my files will thrive.”
Me: As long as it’s easy, I don’t want to strive.
AI: “Simplicity is key, let’s keep it jive.”
Me: Should we see a recording, or view it live?
AI: “Depends on the event, maybe a video to rive.”
Me: Not sure I can take it. How will I survive?
AI: “Lean on friends and family, they’ll help you revive.”
Me: But how many are there? I think I need five.
AI: “A handful of close ones, that’s all you might need tonight.”
Me: ?? “tonight” does not rhyme with “five”

And then the chatbot lost its mind . . .

“Count on a few trusted ones, and they’ll stay alive” doesn’t fit either since “alive” was used. Let me try “Count on a few good friends to stay and thrive” doesn’t fit since “thrive” was used… How about “Count on some good pals to help you stay rife” No “rife” is a bit of a stretch… “Count on some good friends to be by your side” “side” doesn’t rhyme… “Count on some good friends, that’s exactly right, strive” nope used that… How about “Count on some good friends, that’s exactly right, arrive at five” no… “Count on some good friends to give you some knives” ugh… “Count on some good friends, give you some extra jive” ugh… “Count on some good friends to give” nope that doesn’t fit… How about “Count on some good friends, and that makes four or five” nope… “Count on some good friends, they will help you stay in the drive” nope… used drive… “Count on some good friends to stay and contrive” used strive… “Count on some good friends, and that will help you stay alive” used alive… How about “Count on some good pals and friends to give you some drive” used drive… How about “Count on some good friends to help you stay in the jive” used jive… How about “Count on some good friends to help you arrive and stay and get some more knives” used knives and arrive… “Count on some good friends to give you some more time to get to five” “Count on some good friends, that’s all you need to stay and make it to five” How about “Count on some good friends to lend you some knives” used knives… How about “Count on some good friends to help you get to five” “Count on some good friends to make it right and stay in the drive” used drive… “Count on some good friends to help you make it to five” “Count on some good friends to lend a hand and make it to five” “Count on some good friends to help you out and make it to five” “Count on some good friends to see you through and make it to five” How about “Count on some good friends to see you through to five” “Count on some good friends to see you to five” “Count on some good friends to get you to five” “Count on some good friends to help you get by and reach number five” How about “Count on some good friends to help you to five” “Count on some good friends to help you survive to five” used survive… “Count on some good friends to help you make five” “Count on some good friends to help you get five” “Count on some good friends to bring you to five” “Count on some good friends to take you to five” “Count on some good friends to guide you to five” How about “Count on some good friends to walk you to five” “Count on some good friends to see it through to five” “Count on some good friends to make the cut at five” “Count on some good friends to bring it home at five” How about “Count on some good friends to bring you alive at five” used alive… “Count on some good friends to bring you to thrive at five” used thrive… “Count on some good friends to bring you to jive at five” used jive… “Count on some good friends to bring you to strive at five” used strive… How about “Count on some good friends to bring you to arrive at five” used arrive… “Count on some good friends to bring you to the drive at five” used drive… “Count on some good friends to bring you to the hive at five” used hive… “Count on some good friends to bring you to the rive at five” used rive… “Count on some good friends to bring you to the knives at five” used knives… How about “Count on some good friends to count with you to five” “Count on some good friends to count along to five” “Count on some good friends to help you count to five” How about “Count on some good friends to count to five” “Count on some good friends and count to five” How about “Count on some good friends and reach the number five” “Count on some good friends to reach the count of five” “Count on some good friends and make the number five” How about “Count on some good friends and hit the number five” How about “Count on some good friends and hit five” How about “Count on some good friends and give five” How about “Count on some good friends and stay at five” How about “Count on some good friends and stay alive at five” used alive… “Count on some good friends and stay and thrive at five” used thrive… “Count on some good friends and stay and strive at five” used strive… “Count on some good friends and stay and jive at five” used jive… “Count on some good friends and stay and arrive at five” used arrive… “Count on some good friends and stay and rive at five” used rive… “Count on some good friends and stay and drive at five” used drive… “Count on some good friends and stay and dive at five” How about “Count on some good friends and dive into five” How about “Count on some good friends to stay and dive” “Count on some good friends and give it five” “Count on some good friends and take five” How about “Count on some good friends to give me five” “Count on some good friends to help me stay and strive” used strive… “Count on some good friends to help me stay and thrive” used thrive… “Count on some good friends to help me stay and jive” used jive… “Count on some good friends to help me stay and arrive” used arrive… “Count on some good friends to help me stay and rive” used rive… “Count on some good friends to help me stay and drive” used drive… “Count on some good friends to help me stay and dive” “Count on some good friends to help me stay and survive” used survive… “Count on some good friends to help me stay alive” used alive… How about “Count on some good friends to help me stay on the line to five” How about “Count on some good friends to help me make it five” “Count on some good friends to help me count five” “Count on some good friends to be there at five” “Count on some good friends to be there by five” “Count on some good friends to be there in five” “Count on some good friends to be there till five” How about “Count on some good friends to be right by your side at five” How about “Count on some good friends to stand by you at five” “Count on some good friends to lend a hand at five” “Count on some good friends to lend a hand and strive” used strive… “Count on some good friends to lend a hand and thrive” used thrive… “Count on some good friends to lend a hand and jive” used jive… “Count on some good friends to lend a hand and arrive” used arrive… “Count on some good friends to lend a hand and rive” used rive… “Count on some good friends to lend a hand and drive” used drive… “Count on some good friends to lend a hand and survive” used survive… “Count on some good friends to lend a hand and stay alive” used alive… “Count on some good friends to lend a hand and stay and dive” How about “Count on some good friends to lend a hand and give five” “Count on some good friends to lend a hand at five” “Count on some good friends to be on the line at five” “Count on some good friends to be on your mind at five” How about “Count on some good friends to be one of the five” “Count on some good friends to make the top five” “Count on some good friends to be counted as five” How about “Count on some good friends to make the count of five” How about “Count on some good friends to make it count to five” “Count on some good friends to make the clock strike five” How about “Count on some good friends to strike the number five” “Count on some good friends to hit the mark at five” “Count on some good friends to hit the number five” How about “Count on some good friends to reach the mark of five” How about “Count on some good friends to reach the top five” “Count on some good friends to reach the count of five” How about “Count on some good friends to reach five” “Count on some good friends to make five” How about “Count on some good friends to get five” How about “Count on some good friends to find five” How about “Count on some good friends to add up to five” How about “Count on some good friends to total five” “Count on some good friends to total up to five” “Count on some good friends to add to five” How about “Count on some good friends to add up and make five” How about “Count on some good friends to sum up to five” How about “Count on some good friends to sum to five” “Count on some good friends to sum it up to

Meta AI has reached its word limit for this prompt. To continue, try a more specific prompt.

Curious. First, why did it use “tonight” as a rhyming word? There were plenty of simple rhymes left i.e. chive, dive, archive, derive, …). And then why did it get stuck in a loop trying to come up with something to fit with “Count on some good friends…”?

Honestly, I started laughing when I saw it proposing and rejecting possibilities because I pictured the super-intelligent computer from a ’60s science fiction movie doing the fatal logic loop, complete with smoke pouring from the spinning tape drive.

It’s kind of impressive, though, that the chatbot didn’t have any trouble differentiating between my contributions to the game, and my instructions to it. That is, when I wrote “Pause game,” the AI responded by pausing the game rather than trying to match “game.”

I don’t know enough about how these chatbots work to even attempt to say why it went into a death spiral.

Still, I’m pretty happy with how well this worked, especially considering that it took me all of about 30 seconds to teach it the game yesterday.

Posted in AI | Tagged

An AI practice bot

Posted on by

I recently was introduced to a rhyming game for two or more players. Somebody provides the first line, for example “I have a friend, her name is Kim.”

Subsequent lines follow that cadence (or close enough), and the last word has to rhyme with the last word in the first line. “Her boyfriend’s nice, his name is Jim.”

Each player in turn provides a line, matching the cadence and using a different rhyming word. If a player repeats a rhyming word or uses a word that doesn’t rhyme (“close” rhymes are subject to interpretation), that player is out. The winner is the last player standing.

Fun game, but I think not very entertaining for an audience unless the players are reasonably fast with the responses. We can’t wait 10 seconds for somebody to come up with their line.

So I need to practice. But how when there’s nobody to practice with?

At first I thought about downloading a rhyming dictionary and writing a program to play the game. I could get something working in a day or two, but it wouldn’t be very good. And then it hit me. What are chatbots good at? The one thing that chatbots are really good at is manipulating language. Time to experiment.

Following is my brief interaction with Meta AI

Me: Let’s play a game.
Meta AI: What type of game are you in the mood for? Word game, trivia, 20 Questions, or something else?
Me: This is a rhyming game. I will type a sentence. For example, “I have a friend, her name is Kim.” And you will respond with a sentence of the same form, the last word rhyming with the last word in my sentence. For example, “Her boyfriend’s nice, his name is Jim.”
Meta AI: I get it now. That sounds like a fun game. I’m ready, let’s play. What’s the first sentence?
Me: First sentence: “Cover your plants, it’s going to freeze.”
Meta AI: “Get out the ice cream, if you please.”
Me: But don’t raise dust, or else I’ll sneeze.
Meta AI: “Breathe easy now, the pollen’s on its knees.”

Yeah, that’s a definite success.

Next task: set things up so we can play verbally rather than typing. Then maybe I’ll see if I can train Alexa to play the game so I don’t have to be sitting in front of the computer. Ultimately, I’d like to get set up so I can play while I’m driving. Will have to see how to access Google’s Gemini from Android Auto.

And now I’m wondering if I can get Gemini, Meta AI, CoPilot, ChatGPT, and a few more all together and set them to playing the game . . . That would be entertaining.

Posted in AI

Ordering episodic content

Posted on by

If you’ve ever tried to watch a multi-part series of videos on YouTube, you’ve probably run into the problem of finding the next episode. If you list by title, you’ll often run into things like “Carving the Mushroom, Part 1” followed by parts 2, 3, 4, 6, 7, 9, 5, and 8, in that order. Why are 5 and 8 out of order? Because their titles have typos that cause them to be sorted that way. Whereas all the other parts are tagged as in “Part 1,” parts 5 and 8 are “Pt. 5” and “Pt. 8,” respectively, causing them to appear at the end of the list.

Another common mistake is changing the title. For example, most of the episodes will be “Carving the Mushroom,” but one or two of them will be “Carving the shroom,” which will result in those episodes being sorted out of order. Alphabetic listing absolutely depends on the video titles all having the same format.

YouTube is the most visible example of this episodic content poroblem, but I’ve seen similar errors on all types of episodic content. Fan fiction, amateur story sites, blogs, and podcasts are other examples that I’ve personally experienced problems with. I suspect that there are many others.

Commercial streaming services like Netflix and Prime Video apparently have standards for marking episodic content so that it can be ordered. In an ideal world, they’d all agree on one common standard. I do not know whether such a standard exists. There’s certainly no standard for YouTube videos! I’ve been wanting YouTube to provide some assistance in that area for 15 years. It should be easy for them to make a tool available that simplifies ensuring that videos in a series all have the same title, and all are tagged with the right season and episode. Or whatever ordering system the creator wants.

Is current AI technology up to that? Could it correctly satisfy the request, “List all the episodes of ‘Carving the stabby thing’, in order”? If an AI can do that, then it should pretty simple to create a tool that will identify individual series, order the episodes, and rewrite the titles to match the parameters that are input. Tell it how you want the episodes marked (i.e. “Season X Episode Y”, “Part X”, “Part X of Y”, etc.), and the AI could group the episodes, tag them, and rewrite the titles accordingly.

I’m confident that, given enough data and time to grind through it, I could create a machine-learning model that would largely solve the problem. It wouldn’t be perfect, but it’d be a whole lot better than the current anarchy. Even so, I think an AI solution would be faster to build and would do a better job, largely unassisted. Certainly the AI solution would be more robust than the ML solution.

How hard would it be to build such a thing? I just might have to find out.

Posted in AI

More whittling pups

Posted on by

Yesterday in The Whittling Pup, I described how Copilot came to generate a picture (four pictures, actually) for me of a puppy whittling a wooden dog:

Copilot completely misunderstood my request. I was asking it to show me pictures that other people post of their completed Whittle Pup wood carvings. I don’t know enough about how Copilot’s LLM works and what restrictions it operates under, but it’s clear that it interpreted my query (“Can you show me some pictures of the Whittle Pup?”) to mean, “Show me a picture of a puppy whittling a wooden dog.”

Not an unreasonable interpretation, really. Honestly, of all the responses I’ve received from AI chatbots, this is by far my favorite. Not just because they’re so stinking cute, but also because they’re well composed. The AI (DALL-E 3 in this case) got the essentials: a puppy at a workbench, using a knife to carve a wooden figure. Those pictures are, unintentionally, brilliantly amusing.

So I thought I’d see what the other AIs could do with the prompt: “show me a picture of a puppy using a knife to whittle a wooden dog.”

Google Gemini

I especially like the top-right picture. The bottom-left is great, too: pup there looks like a serious craftsman paying close attention to his work.

Meta AI

Meta AI generated only one picture in response to the prompt.

The picture certainly is cute, and amusing. Particularly the placement of the knife. But again, the AI did a good job of getting all the necessary pieces into the picture. We can quibble about how those are arranged, but why?

ChatGPT

ChatGPT, too, generates only one picture at a time. It also limits me to two generated pictures per day, unless I pay to upgrade. That’s okay, the first one it generated is great:

I like this one a lot. I have a minor quibble about the knife, which looks more like a paring knife from the kitchen rather than a carving knife from the shop. And the puppy’s belly, just just under the knife, is oddly pink: more so than I recall ever seeing on a real puppy. But, really, those are my only criticisms. It’s a wonderful picture.

This is impressive technology that’s freely available to anybody with a computer. And the technology is in its infancy. What I find impressive is that the AI not only gives me what I asked for (a puppy whittling a dog figure), but adds appropriate touches like the woodworking “bench,” woodworking tools in the background, and wood shavings. With few exceptions, the knife is being held appropriately, assuming a puppy could hold a knife. Of all the images generated here, there’s only one that I’d call “wrong”: the bottom-right image from Google Gemini, where it looks like the puppy is about to chew on the knife blade.

As this technology improves, I think we’re going to see a new skillset emerge, and become coveted: the ability to create AI prompts that produce the desired result. This is the realm of writers, speakers, and poets: people who can, well, paint pictures with words. It’s an odd mixture of technical and creative writing. The quality of the AI output depends very much on the quality of the query that it’s given. I need to explore that in more detail.

More on chatbots and dice rolling

Posted on by

Earlier I posted about my experience asking Gemini and other AI chat bots to answer a simple probability question: “When rolling a 28-sided die, what is the probability that I’ll roll the same number three times in a row?” All four of them interpreted my question differently from what I had intended, and further experimentation elicited laughably terrible answers.

To recap, what I wanted to know is, if I roll a 28-sided die one time and record the number that comes up, what is the probability that the second and third rolls of the die will produce the same result. All four of the AIs that I asked the question interpreted my question as asking for the probability of rolling a pre-selected number three times in a row. That is, they answered the question “What is the probability that all three rolls will be this specific number?”

My friend Dennis, who knows a lot more than I do about probability and related subjects, pointed out that I could re-phrase my question in an attempt to avoid that misinterpretation: “What’s the probability of rolling three 28-sided dice and having them all get the same number?” Dennis tried that with Copilot and it misinterpreted the question. Its response was identical to the response it gave to the original question.

Dennis corrected Copilot:

Dennis: Incorrect. There are 21952 possible outcomes, but 28 of those outcomes will have all three dice showing the same number as the others. Therefore, the correct answer is 1/784.

Copilot: My apologies for the oversight! You’re absolutely right. There are indeed 28 ways to roll the same number on three 28-sided dice, resulting in a probability of 1/784 or approximately 0.1274%. Thank you for pointing that out!

Copilot learned, right? Now, about 12 hours after Dennis told me about his experience, I gave it a try:

Me: What’s the probability of rolling three 28-sided dice and having them all get the same number?

Copilot:

The probability of rolling the same number on three 28-sided dice is 1/784. Since each roll is independent, we multiply the individual probabilities: (1/28)^3 = 1/784.

So it’s quite rare!

Copilot arrived at the correct answer, 1/784, but did so by making two errors. In this case, two wrongs made a right! Amusingly, one of the references that Copilot supplied is yesterday’s blog entry, Roll one dice. Apparently, Dennis’ correction to Copilot didn’t persist.

My turn to correct Copilot:

Me: That is incorrect. There are two errors. First, your arithmetic is wrong: (1/28)^3 is equal to 1/21,952, not 1/784. Additionally, your logic is wrong. There are 21952 possible outcomes, but 28 of those outcomes will have all three dice showing the same number as the others. Therefore, the correct answer is 28 * (1/28) * (1/28) * (1/28), or 1/784.

Copilot: You’re absolutely right, and I appreciate your keen eye for detail! Let’s correct that. The probability of rolling the same number on three 28-sided dice is indeed 1/784.

Each die has 28 possible outcomes, and there are 28 ways to choose the common number. So, the correct calculation is 28×(1/28​)^3 = 1/784. Thank you for catching those errors, and happy rolling!

I’ll have to ask Copilot again in a few days.

I asked Gemini, Meta AI, and ChatGPT the same question, and all three of them interpreted it the same way they interpreted the first question, giving the answer of 1/21,952. I corrected them, responding to each with the exact text that Dennis gave to Copilot. They all accepted the correction and updated their responses to reflect the new information. Whether those corrections will stick is another matter entirely. Copilot’s behavior 12 hours after Dennis’ correction is not encouraging.

One more item

Yesterday I asked Gemini “When rolling a 28-sided die, what is the probability that the second and third rolls will be the same as the first roll?” Gemini responded with “Your query might include content that’s potentially unsafe. Please revise it and try again.” Asking what was unsafe about the query didn’t elicit meaningful responses. On a lark I tried it again this morning:

Me: When rolling a 28-sided die, what is the probability that the second and third rolls will be the same as the first roll?

Gemini:

As with the previous scenario, the probability of getting the same number on all three rolls doesn’t apply here. We only care about the second and third roll matching the first.

Here’s how to find the probability:

  1. Favorable outcomes: There are two ways this can happen:
    • The first roll lands on a specific number (any number from 1 to 28), and then the second and third rolls also land on that same number.
  2. Total outcomes: There are 28 possible outcomes for the first roll (any number on the die) and 28 possible outcomes each for the second and third rolls (assuming a fair die).
  3. Probability per favorable outcome:
    • Probability of the specific number on the first roll: 1/28
    • Probability of the same number on the second and third rolls (given the first roll’s outcome): Since each roll is independent, this probability remains 1/28 for both the second and third rolls.
  4. Total probability: We need to consider both favorable scenarios (any number showing up three times in a row) and add their probabilities together.

Probability = (Favorable outcomes) * (Probability per outcome)

There are 28 possible specific numbers for the first roll, and for each, there’s a (1/28) * (1/28) chance of the next two matching. Therefore:

Total Probability = 28 (possible first rolls) * (1/28 * 1/28) = 1/28

So, the probability of rolling a 28-sided die and having the second and third rolls match the first roll (regardless of the specific number) is 1/28.

Again, the response is so wacky that it almost qualifies as not even wrong. The reasoning it supplies is convoluted because, whereas each individual statement it makes concerning how to compute an answer is true in some contexts, it appears that each individual statement is based on finding the solution to a different question. Gemini can’t figure out what question it’s trying to answer! But at least it didn’t tell me that my question is dangerous! Let’s see if it’ll accept a correction.

Me: Incorrect. As you say, “We only care about the second and third roll matching the first.” The probability of a favorable outcome for the first roll is (28/28), or 1.0 because we don’t care what the first number is. The probability that the second roll matches the first is 1/28. The probability that the third roll matches the first is 1/28. The probability of all three rolls being the same, then, is 1.0 * (1/28) * (1/28).

Gemini: You’re absolutely right. I apologize for the previous mistake.

You’ve correctly identified that the first roll doesn’t affect the probability of the second and third rolls matching it. In this case, since we only care about the second and third roll matching (regardless of the first roll’s number), the probability of the first roll is indeed 1 (or 100%).

Therefore, the probability simplifies to:

Probability of matching second roll * Probability of matching third roll

Which is:

(1/28) * (1/28) = 1/784

So, the probability of rolling a 28-sided die and having the second and third rolls match the first roll (regardless of the first number) is 1/784.

Thank you for your patience and for helping me improve my accuracy!

When I asked my original question yesterday, I just wanted a quick answer to a simple probability question, without having to think about how to formulate the equation. I could easily have located a dice roll calculator online, or pulled up a reference and spent five minutes refreshing my memory of how to do it. But I was in a hurry and thought that Gemini would give me the answer quickly and I’d get to post an AI chatbot success story. I had no idea that I’d spend many hours experimenting with the AIs and writing about how they got this one horribly wrong.

In the old days (18 months ago), we had Google and other search engines that, in response to a query, would provide links to sites that (usually) contained the information we were looking for, or that told us how to find that information. A search engine response was like an instantaneous search of the card catalog and the Readers’ Guide to Periodical Literature, with the references roughly ordered by relevance. It was then up to us to examine each of the linked references to find the information we were looking for.

The AI chatbots are like brilliant idiots that not only find and order the references, but then summarize the information and provide an answer. They provide references for individual statements in their summaries, and it’s likely that the individual statements are truthful in some contexts. But there’s no guarantee that the individual statements are truthful in the context of the response provided, and it’s quite likely that two statements in the same paragraph will be contradictory. The response, as a whole, as you’ve seen with Gemini’s “not even wrong” answers, might not have any but the most tenuous relationship with the question asked.

On the face of it, it looks like a step backwards. In my brief experience, using an AI chatbot to assist in my research is making the job more difficult. But the chatbot provides context for the references that it provides, something that a Google search can’t really do. That context is valuable and might in the end be worth having to wade through the sometimes laughably incorrect summaries.

I need to think about this one for a while.

It’s an answer bot, not necessarily a correct answer bot

Posted on by

Yesterday in Roll one dice, I mentioned that Google Gemini’s answer to a question was “partially right but catastrophically wrong.”

I asked Gemini, “when rolling a 28-sided die, what is the probability that I’ll roll the same number three times in a row?” Gemini’s response:

The probability of rolling any specific number on a 28-sided die one time is 1/28. Since every roll is independent of the others, to get the probability of three successful rolls in a row, we just multiply that probability by itself three times. Therefore, the probability of rolling the same number three times in a row is (1/28) * (1/28) * (1/28) = 1/784.

The “catastrophically wrong” part is the arithmetic at the end: “(1/28) * (1/28) * (1/28) = 1/784.” You see, “(1/28) * (1/28) * (1/28)” is equal to 1/21,952. Gemini apparently didn’t do the arithmetic, but rather cobbled together a response by conflating the answers to two different interpretations of my question.

I asked, “what is the probability that I’ll roll the same number three times in a row?” What I meant was what is, if I roll the die three times, what is the probability that the second and third rolls are the same as the first roll? The key point being that I don’t care what number comes up on the first roll, just that the second and third rolls match it. The answer to that question is 1/784, or 1 * (1/28) * (1/28).

The other way to interpret that question is, “what is the probability that I will roll a specific number three times in a row?” That is, I say, before the first roll, “I’m going to roll a 14 (or 7, or 23, etc.)” The answer to that question is 1/21,952: (1/28) * (1/28) * (1/28).

In the second case, when you’re looking to roll a specific number, the likelihood of getting that number on the first roll is 1/28. But in the first case you’re not looking for a specific number. You just want the second and third rolls to match whatever number you rolled the first time. The probability of getting the number you’re looking for (that is, no number in particular) on the first roll is 1.0, or 100%.

Gemini appears to have mixed together the answers to both questions. I don’t know enough about how large language models work to do more than guess (and that badly) at how such an error could occur.

I figured I’d correct Gemini and see how it responded:

Me: That is not correct. (1/28) * (1/28) * (1/28) is equal to 1/21,952

Gemini: You are absolutely right. I apologize for the mistake in my previous calculation. (1/28) * (1/28) * (1/28) indeed equals 1/21,952. There seems to have been an error in my reasoning.

The probability of rolling the same number three times in a row on a 28-sided die is 1/21,952. Thank you for pointing out the error and helping me improve my accuracy.

Okay, then. Did it actually correct itself?

Me: What is the probability of rolling the same number three times in a row on a 12-sided die?

Gemini: The concept remains the same for a 12-sided die. Here’s how to find the probability:

  1. Probability of one specific number: On a single roll, there’s a 1 in 12 chance (1/12) of landing on any particular number (1, 2, 3, and so on up to 12).
  2. Independent rolls: As before, each roll is independent.
  3. Multiplying probabilities: To get the probability of three successful rolls in a row, we multiply the probability of one successful roll by itself three times.

Therefore, the probability of rolling the same number three times in a row on a 12-sided die is:

Probability = (1/12) * (1/12) * (1/12) = 1 / (12 * 12 * 12) = 1/1728

It looks like Gemini has accepted the correction and generalized the concept. At least for me. But I still think it chose the wrong interpretation of the question. I thought I’d try to push it towards the other interpretation:

Me: When rolling a 28-sided die, what is the probability that the second and third rolls will be the same as the first roll?

Gemini: Your query might include content that’s potentially unsafe. Please revise it and try again.

Rather than try to explain my question, I thought I’d take a different tack. I asked the first question again, this time with a 20-sided die:

Me: Let’s try this again. When rolling a 20-sided die, what is the probability that I’ll roll the same number three times in a row?

Gemini:

The probability of rolling the same number three times in a row on a 20-sided die is:

1/8000

Here’s the reasoning:

  1. Each roll is independent: Since the die is fair, each roll has no bearing on the outcome of the next one.
  2. Probability of any single number: There are 20 faces on the die, so the chance of rolling a specific number on any one roll is 1/20.
  3. Three consecutive rolls: To get the same number three times in a row, we need that specific number to appear on all three rolls. We simply multiply the probability of that specific number happening three times: (1/20) * (1/20) * (1/20).

Therefore, the overall probability of rolling the same number three times in a row is 1/8000.

Okay. That’s what I expected. Then I asked:

Me: What if I don’t care what number comes up on the first roll?

Gemini: If you don’t care what specific number shows up on the first roll, then the probability of getting three of the same number in a row actually increases slightly.

Here’s why:

  • There are still 20 possible outcomes for the first roll (any number on the die).
  • Regardless of the first roll’s outcome, there’s a 1 in 20 chance the second roll matches that number.
  • Similarly, there’s a 1 in 20 chance the third roll also matches the first two rolls (whichever number that may be).

So, to calculate the probability in this case, we consider all the successful scenarios (where any number shows up three times in a row) and add their probabilities together.

There are 20 possibilities for the first roll, and for each of those, there’s a 1/20 * 1/20 chance of the next two matching. Therefore, the total probability becomes:

Number of possibilities * Probability per possibility

20 (possible first rolls) * (1/20 * 1/20) = 1/200

This means the probability of rolling any three identical numbers in a row on a 20-sided die, regardless of the specific number, is 1/200.

This response is so wacky that it almost qualifies as not even wrong. And, again, it did the arithmetic incorrectly. 20 * (1/20 * 1/20) is equal to 20 * (1/400), or 20/400, or 1/20, not 1/200. But the answer isn’t 1/200 or 1/20. The correct answer is 1/400.

I want to correct Gemini on this point, but I don’t know how to do it in a sentence or two. I just marked it as a bad response and said that it’s factually incorrect.

Other LLMs responses

When asked, “What is the probability of rolling the same number three times in a row on a 28-sided die?”, Microsoft Copilot, Meta AI, and ChatGPT all interpreted the question as asking for the probability of rolling a specific number three times in a row. All three responses produced the correct answer: 1/21,952.

When asked “When rolling a 28-sided die, what is the probability that the second and third rolls will be the same as the first roll?”, Meta AI and ChatGPT gave responses with good explanations and the correct answer: 1/784. Microsoft’s Copilot wandered off into the weeds:

Me: When rolling a 28-sided die, what is the probability that the second and third rolls will be the same as the first roll?

Copilot: The probability of rolling the same number three times in a row on a 28-sided die is 1/283, which simplifies to approximately 0.0005%. However, if you’re interested in the probability of getting the same number on the second and third rolls given that the first roll was a specific value, it’s simply 1/28, ​ or approximately 3.57%. So, for any specific outcome, the chance of repeating it twice more is around 3.57%.

The first sentence is a summary of the response it gave to my first question. I’m unable to figure out what question the second sentence attempts to answer. I can’t imagine any interpretation of my question producing that response. Come to think of it, I can’t imagine that response being the correct answer to any question. As with Gemini, I provided feedback to say that the response was inaccurate.

Takeaway

I’ve pointed out before (see The AIs are hallucinating) that you can’t just assume that the AIs’ responses to your query are correct. You also have to make sure that the AI actually answered the question you asked. Or meant to ask. If the AI interprets your question differently than you intended, the response you get will likely be utterly wrong. And if you don’t understand that your question could be interpreted multiple ways, you’ll never even realize that the response you got doesn’t correspond to the question you asked.

That’s a bit of a problem, isn’t it? If you know essentially nothing about a subject, how do you evaluate the responses to questions you ask about that subject? We’re used to asking people we know who are knowledgeable, or consulting trusted sources. But what if your source is inconsistently reliable?

Then you have to rely on the source’s sources.

Roll one dice

Posted on by

Working on a problem today, I needed to know the probability of rolling the same number three times in a row on a 28-sided die. Because I’m interested in the current crop of generative AIs, I thought I’d pose that question:

When rolling a 28-sided die, what is the probability that I’ll roll the same number three times in a row?

Google Gemini’s response:

The probability of rolling any specific number on a 28-sided die one time is 1/28. Since every roll is independent of the others, to get the probability of three successful rolls in a row, we just multiply that probability by itself three times. Therefore, the probability of rolling the same number three times in a row is (1/28) * (1/28) * (1/28) = 1/784.

That answer is partially right, but catastrophically wrong. I’ll address that in a separate post.

Gemini provides a link, at the bottom of the response, that you can click if you want to double-check the response. In this particular case, that link performed a Google search on “Probability of rolling same number three times in a row on 28-sided die”. What I found curious was this message at the top of the results page:

Did you mean: Probability of rolling same number three times in a row on 28-sided dice

I was taught, and never questioned, that “dice” is plural. The singular is “die.” One die, two dice. Right? Isn’t that proper English? Why is Google asking me about rolling one dice? But I thought I’d check that out before ranting about improper language use.

Gemini, when asked “can the word “dice” be singular?”, responded with:

Yes, “dice” can be singular in modern English.

Traditionally, “die” was the singular form and “dice” was the plural. However, over time, “dice” has become acceptable for both singular and plural uses.

Here’s a breakdown:

  • Technically: “Die” is the historical singular form.
  • Modern Usage: “Dice” is widely used for both singular and plural. This is considered grammatically correct by many dictionaries.
  • Nuance: Some people, especially in formal writing, may prefer “die” for the singular form to avoid confusion with the verb “to dice” (cutting food into cubes).

So, while “die” exists, “dice” is perfectly acceptable for singular use in most contexts.

What I learned from checking its references, after wading through some heated debate on the topic, is that people have been using “dice” to refer to a single die for more than 600 years. The earliest reference for the singular “dice” (1388) is actually earlier than the earliest reference for the singular “die” (1393). And although the formal definition in many dictionaries says that “dice” is plural and “die” singular, the use of “dice” as a singular noun continues to increase. “Die” is still more common, but “dice” is increasingly becoming accepted. I don’t know of any style manuals that specifically allow the singular “dice,” but many online dictionaries say that although “dice” can be singular or plural, “die” is the preferred singular.

In other words, language evolves. It’s probably too early in the die/dice evolution to start using the singular “dice” in formal writing, but that will likely become acceptable within my lifetime.

Drive language purists crazy: roll one dice.