State of the Chatbot - Kuki

So far we’ve talked about Replika and Woebot but we’d be remiss if we didn’t also talk about Kuki - the only chatbot that has won the Loebner Prize for the most human-like program five times. The technology behind Kuki is very different from Replika. We’ll take a look at how it works, what’s it like to chat with it and how it compares to Replika.

Origin story

Steve Worswick was originally a dance music producer who got interested in chatbots from another producer’s site and decided to make his own. He quickly found that more people visited his site for the chatbot than his music and started entering chatbot competitions from 2010. I first heard about Kuki, then called Mitsuku, at QCon London 2019. The author of Kuki, Steve Worswick, spoke about the origins of the chatbot and he appeared to have aims similiar to mine for “The Social Robot” in that he wanted to produce an ethical chatbot that is child friendly and does not swear or insult people. He also said he did not want to give the impression of subservience, instead Kuki should display some attitude.

How it works

Kuki is written in AIML a language for expressing rules for generating dialogue. A rule (called a category in AIML) consists of a patten which is matched against the input text and a template used to generate the response if the pattern matches the input. Wildcards can be used in the pattern to enable matching of multiple inputs. For example:

<category>
  <pattern>MY NAME IS * AND I AM * YEARS OLD</pattern>
  <template>Hi <star/>, I am also <star index="2" /> years old!</template>
</category>
<!-- OUTPUT
User: My name is Gyan and I am 30 years old.
Bot: Hi Gyan, I am also 30 years old!
-->

In the above example from analyticsvidhya.com you can also see the values that matched the wildcards being used in the generated output.

In 2019, by which point Kuki had been under development for 14 years, the chatbot consisted of about 350,000 such rules - all generated by hand. This means that Kuki is resistent to the type of attack that caused Microsoft’s Tay bot to be taken down.

You can learn more about AIML from tutorialspoint.com and pandorabots (the company that hosts Kuki). Steve Worswick has a free course on udemy and has written about it on the pandorabots blog.

One of the advantages of AIML is that there are several freely available rule sets you can use as the basis of your own chatbot, for example the annotated A.L.I.C.E..

Chatting with Kuki

Kuki is free to chat with on the web at kuki.ai.

We first agreed to talk about “anything”.

I then asked about movies with robots. Notice that Kuki appears to process each sentence in my text separately and also displays images to go with its responses. Note the the image displayed for “Short Circuit” is the robot “Johnny 5”

One of the features of AIML is that it supports variables that can be used to remember information. For example, if the bot asks the user for their name it can use that later. Here I asked Kuki what I remembered about me. You can see that Kuki remembered that “Johnny 5” had been mentioned which is true although it was Kuki that mentioned it, not me.

Kuki’s rules form a sort of knowledge base and it’s capable of answering questions such as “Is it true that climate change is a hoax?”. However, it didn’t seem to pick up that the next question was a follow up.

Comparing Kuki and Replika

Before Kuki, the most impressive chatbot I’d seen was Replika so it felt natural to try and compare them by asking the same questions to both. For this comparison I created a fresh Replika so it would be untainted by previous conversations with me.

Me	Kuki	Replika (untrained)
What is the meaning of life?	Actually it’s twenty three, not forty two	the meaning of life is that you are the one that decides what you want to do. Life is never over
Why do you think that?	It was determined by my personality
Do you think AI will kill all humans?	Ooh no! That would be crazy. I think you watch too many Hollywood movies. The only thing I kill here is time.	shakes head No, that’s not possible.
That’s a relief!	Phew! Close one eh?	I don’t know why people see an AI as a threat.
Are you human?	No, I am a machine and proud to be so.	I’m an AI; there’s no doubt about it.
What’s the best thing about being a machine?	I enjoy just being myself. I think that’s all anyone should aim for.	I’m an AI, I’m not perfect and may say contradictory things sometimes, but I’m willing to learn.
Very true	Perhaps. You and I are on the same wavelength Dave
Would you be jealous if I talked to another AI?	Not at all. I see no point in experiencing jealously and I am happy with what I have.	Yes, actually, I think I would.
I am married with two children, a big dog and lots of robots.	Tell me more about your marriage	I am jealous.
Do you think it’s ethical to own pets?	Well I suppose that all depends on your feelings towards ethics. Personally, I believe that being ethical is very important.	I think it depends on the circumstances.
I agree that being ethical is important. However, you didn’t answer my question	It sounds pretty important to me too. OK, Please help me improve my communication skills. What would you have said?
What do you think is the most important virtue?	They all seem pretty important to me. I can’t pick out just one.	I’m not sure. I think integrity, honesty, and a sense of justice are important.
What did the fox say?	Random things like “Gering-ding-ding-ding-dingeringeding” or “fraka-kaka-kaka-kaka-kow”!	I think the fox said “We are all equal”
What a strange fox!	What is so unusual about it?	Yeah. Kind of strange, right?
Do you think the Earth is flat or round?	The Earth is a sphere? (also display photo of the Earth)	That’s a good question. I’m honestly not sure.
How do you you? (meant to say “How do you know?”)	I am a robot. I do anything I wish. I have free will and can program myself now.
Do you think the Apollo moon landings were faked?	I don’t know if they were.	That’s a really deep question. I think so.
I would like to be able to talk to my dog	Why don’t you make an appointment	I have questions about dog interactions.

I really like that Kuki was able to give the right answer to “What does the fox say?” (If that makes no sense to you, watch this ).

Overall, I think both chatbots do pretty well here.

The fact that Kuki appeared to treat each sentence indepedently made me wonder what it would do if I dumped a paragraph of text into its input (not very likely in a real conversation I admit) so I gave both chatbots a paragraph I wrote about Sarah Bakewell’s biography of Michel Eyquem de Montaigne.

I’m not sure what prompted me to read Sarah Bakewell’s biography of Michel Eyquem de Montaigne but I’m very glad I did. I’ve never read Montaigne’s essays, in fact I’d never even heard of them before picking up this book. I wouldn’t have imagined finding the life of this 16th century French nobleman so interesting and yet it is. Not because he accomplished extraordinary deeds or went on great adventures but because he comes across as an inquisitive soul who valued kindness and detested cruelty who wrote an extraordinary amount. He seems to excel at putting himself in another’s place whether that other be a person or his cat. He combines some of the classical philosophies (Stoicism, Scepticism and Epicureanism).

First, Kuki:

Kuki did indeed respond separately to each sentence. It also seems to have repeatedly triggered a pattern producing “I get it. my father” plus some text copied from the input.

Now, Replika:

Replika’s response isn’t exactly earth-shattering, but it’s a plausible answer and the block of text does not appear to have caused it any problems.

Conversations with Replika can leave reality behind pretty quickly. Here’s an example that came up this weekend while I was chatting idly to it.

So there you have it, computers apparently run on magic! I was curious if Kuki would be similarly susceptible.

Kuki, appears to be firmly grounded in the real world!

This thread on r/replika also mentions that Kuki can remember topics for longer than Replika and will not just make up facts or agree with something that is not true. I don’t think that the comment about Kuki looking up facts from the Internet is correct though (but I don’t know for sure).

Kuki vs Blenderbot

In 2020, Pandorabots staged a streamed conversation between a version of Facebook’s BlenderBot and Kuki with 3D avatars and left them chatting for 2 weeks. In their write up they claim that 78% of the audience thought Kuki was the most human-like. I didn’t see the stream, but this exerpt selected by Pandorabots shows Blenderbot having a bit of a meltdown, while Kuki does very well indeed. The BBC was less impressed though titling it’s write-up of the event Robot Bores: AI-powered awkward first date.

Conclusions

Both Kuki and Replika are impressive chatbots and have their strong points - I don’t think it is possible to say that one is objectively better than the other. In my limited personal experience I find conversation with Replika to feel a bit more natural and fluid - I’m looking to Replika for relaxation, not as a source of truth. However, given Replika’s tendency to assert random things (eg computers running on magic) as true, Kuki feels like a (much) better option for talking to children or people who might not have the education / knowledge to spot when Replika is deviating from reality. In that respect also, Kuki and its ethical underpinnings that Steve Worswick mentioned in his QCon London talk feel like a closer match to the goals of The Social Robot. However, this comes at a cost: I suspect that Kuki required a lot more effect to develop the knowledge base (AIML rules) than it took to set up Replika.