Turing Test hasnt been passed

Sorry to confuse you, since my previous blog post claimed the Turing Test had been passed. On closer inspection scientists are now not so sure that the chat bot "Eugene Goostman", a computer programme developed to simulate a 13-year-old Ukrainian boy, actually passed Turings Imitation Game. Lets look at the evidence. First, Eugene only managed to convince 33% of the judges that it was human, therefore the majority of thought it wasnt human. Second, Eugenes developers are clearly gaming the game; by creating a persona who is a child and speaks English as a second language theyre lowering the judges expectations as to its ability to understand questions and give sensible answers. Transcripts of the conversations from the latest Royal Society test are not yet available but heres and excerpt from a transcript from a previous event at Bletchley Park on 23 June 2012, with the same program.
[15:02:56] Judge: I come from Manchester. Did you see the jubilee?
[15:03:02] Eugene: Could you invite me to Manchester? I would be so grateful. 
[15:03:17] Judge: Did you see the jubliee?
[15:03:27] Eugene: Try to guess! Actually, I dont understand why you are interested. I know you are supposed to trick me.
[15:03:35] Judge: Not really
[15:03:40] Eugene: Not really? Why? What are any interesting site-seeings in Manchester?
[15:03:59] Judge: Lots of nice things. Do you like football?
[15:04:04] Eugene: Does anyone hate it? Some another topic?
[15:04:18] Judge: Will you be watching the football on Sunday?

[15:04:23] Eugene: I play sports sometimes. On computer. But I prefer racing.

Lets analyse it. First, Eugene doesnt know what the Jubilee was - perhaps it didnt make the news in Ukraine, thats understandable. But then something odd happens; Manchester and football are mentioned and Eugene tries to change the topic. Im sorry but there isnt a 13 year old boy on the planet who wouldnt then have mention Manchester United! FAIL. You can make up your own mind by talking with Eugene here (hint: try the Manchester and football trick, Eugene falls for it every time, and yes football is a big sport in Ukraine).

from The Universal Machine http://universal-machine.blogspot.com/

Put the internet to work for you.

Turn off or edit this Recipe

Recommended for you

Related Posts by Categories