Jump to content

Talk:Loebner Prize

Page contents not supported in other languages.
From Wikipedia, the free encyclopedia

not 5 minutes any more!

[edit]

In 2010 the rules changed - 25 minutes instead of 5. I will not edit the article because I'm not native english speaker/writer, so if anybody... The info is confirmed on the official Loebner Price webpage --Ravyr 22:11, 09 February 2011 (UTC)[reply]

made this an ai stub

[edit]

This article could really use fleshing out with descriptions of the state of the art and progress over the years. Since judges do occassionally get fooled here, there is a case that the Turing Test has been passed by some systems, which is significant to the debate about Strong AI --Jaibe 20:39, 14 July 2006 (UTC)[reply]

fleshed out the requirements for the $25,000 prize

[edit]

Although it has been traditional to state the requirements for the $25,000 prize (and by extension the $100,000 prize) as being merely to convince judges that a computer is a human, the structure of the competition makes this a misleadingly incomplete requirement. The judge knows that one entity is a computer and the other a human. Therefore in order to declare the computer to be the human the judge must also declare the human to be the computer. Stating this implicit requirement explicitly gives a clearer picture of what contestants in the competition are really up against, and raises the important question as to whether the competition can be won even in principle. This second requirement obviously being the harder of the two, leaving that requirement implicit misleads by omission. It also suggests that the Loebner Prize is not for passing the Turing Test but rather the very much harder Loebner Test. Vaughan Pratt 19:14, 20 November 2006 (UTC)[reply]

Turing actually originally phrases his test as a thought experiment where you are trying to determine which of two people (over a terminal) is a man and which is a woman. the same problem of misattribution holds --- you have to both believe the deceiver and disbelieve the honest person. I've seen a group of AI graduate students run this test as a part of a competition, and indeed this sort of failure was rare. One time two people were pretending to be male they were asked their tux measurements, and as it happened the woman knew hers and the man had never even owned a suit. The other time two people were pretending to be women, and they were asked if they were ready to have a baby, and the woman said "yeah sure, why not, I'm ready." No one believed any women present thought that, so she lost. So my point is, it's not impossible, but it is way less probable. You'd have to ask a question the human happened to have a very unlikely answer for, while the computer would most likely have the average answer down.--Jaibe 20:32, 28 November 2006 (UTC)[reply]

==

I removed "First Turing Test" because (see Jaibe's comments above) others have run what claim to be Turing tests, so this gets into complicated arguments over definitions. At the least, it would need attribution. Also, clarified that the contest decides among chatterbots entered in the competition, not all those in the world--for the latter, the organizers would have to actively recruit as many bots as possible, not just call for entries. Vicki Rosenzweig 01:03, 26 December 2006 (UTC)[reply]

misunderstanding the test circumstances

[edit]

In the test one member of the jury uses one computer screen and one keyboard. There is one person (or program) on the other side to which the judge poses questions and the other side responses. So, there is no 2 screens at the same time for asking two competitors! (anyone can check the test conditions at the official homepage). Misibacsi 08:08, 9 July 2007 (UTC)[reply]

You are wrong, in the 2006 Prize at least, there were two boxes, left-hand & right-hand to each screen available to judges. Each side was linked, through Loebner's communication protocol, to an entity. Hence the machine was paired with a human - judge deciding which was which. —Preceding unsigned comment added by 86.138.133.54 (talk) 15:07, 6 December 2007 (UTC)[reply]

The contests since 2006 have have presented the judges with two identical screens, one controlled by a human, the other by a computer. In this respect the contest fully complies with Turing's description.Loebner (talk) 19:09, 2 December 2009 (UTC)User:Loebner[reply]

Is this a joke?

[edit]

I don't understand how anyone can be fooled. I've tried using Alice, elbot, etc. and they are all stunningly not human. Simple questions like "How fast is a train?" fool them all. Someone please explain how anyone could be fooled. Were the judges not allowed to choose the questions? 155.198.65.29 (talk) 13:08, 15 October 2008 (UTC)[reply]

Interacting with a system in isolation is not the same as textually engaging two unseen / unheard entities and using their responses to determine which is human and which is machine. Judges could ask whatever they wanted. See the BBC news video clip here: [1] —Preceding unsigned comment added by Filosofee (talk • contribs) 10:47, 21 October 2008 (UTC)[reply]

also, the web-versions are simplified versions of the corresponding bot. 85.149.120.16 (talk) 23:52, 7 November 2008 (UTC)[reply]

[edit]

The automatic link to "Thomas Whalen", winner of the 1994 Lobener Prize Competition is directed to the wrong page. Thomas Whalen, the researcher who won the competition is not the same Thomas Whalen who was mayor of Albany. The Loebner Prize Winner does not have an entry in Wikipedia —Preceding unsigned comment added by 142.92.60.20 (talk) 14:38, 16 March 2009 (UTC)[reply]

The article fails to mention flaws in the 2009 Loebner Prize

[edit]

Following links on the Web, I found the Loebner Prize website. I downloaded the player and the scripts for the 2009 event. I played all the scripts and read all of the conversations of all three judges with all of the contestants. Then I examined the reported score sheet.

Observations

[edit]

None of the programs responded in a human way for more than a sentence or two at a time. All of them attempted to control the conversation instead of giving answers one would expect from a human. All of them made errors in which they repeated words that had been used by the judge in ungrammatical ways. In fact, one of the programs stated its age as a bit over one year, which, while undoubtedly true, is not what a human would say. That same program elsewhere suggested a "help" question, and, when the judge asked the question, responded with a lengthy list of all of its specific capabilities (such as items like "I can answer the question 'what is two plus five'")!

All of the humans responded in a completely human way, chatting with context and intelligence about subjects ranging from speech processing to rock and roll music.

Although my evaluation of the score sheet was hampered by ambiguity in the terse headings of the spreadsheet-like results, it appeared to be full of errors. It reported zero (0) success for every judge for every contestant. Even though it separately reported that all combinations of contestants and judges resulted in correct evaluations on the part of the judges, it nevertheless apparently picked a winner from among the submitted programs.

Conclusions

[edit]
  • I feel sure that it would be obvious to anyone reading the transcripts that all of the programs did a terrible job. None sounded to me even remotely human. No wonder all the judges could tell the programs apart from the humans; it wasn't at all difficult.
  • The supplied score sheet for the 2009 competition draws erroneous conclusions from its own data. Furthermore, it does not indicate any success by any program.
  • I would agree with and confirm Minsky's opinion, especially since Loebner is president of Crown Industries, the event's sponsor (entries for 2010 are even to be submitted to Loebner in care of Crown Industries, showing little separation between the man and his company).
  • Although the contest itself appears to have been conducted properly, its complete failure to demonstrate artificial intelligence (as evaluated by its version of the Turing Test) is apparently mentioned nowhere. In fact, I found a claim elsewhere on the Web (Kurzweil News Report) that the 2008 version of this contest fooled 25% of the judges; that certainly was not true in 2009. By apparently not reporting the true results, this contest would appear to be a scam put on for the sake of publicity. I wish I could come to some other conclusion, but the facts do not appear to permit of any other conclusion, at least for 2009. (Note: here are transcripts for previous years, including 2008.)
  • We need a reliable secondary source to add this information to the article, since my information presented here is a result of my own original research. Someone with more free time than I have should finish the job, especially to note the obvious failure of this contest in 2009 (at least) in demonstrating acceleration in the field of artificial intelligence. David spector (talk) 22:39, 11 December 2009 (UTC)[reply]
You seem to be misunderstanding the competition. The award each year is given to the computer program that seems most human, regardless of whether it fooled any judges or not. In the 2009 competition, the humans scored highest, so the award was given to the highest scoring computer program, even though it fooled no judges. There is a larger $25,000 prize that will be awarded to the first program that the judges can not distinguish from the humans, but this prize has not yet been awarded since the contest began. —Preceding unsigned comment added by 68.40.87.202 (talk) 00:19, 25 October 2010 (UTC)[reply]

2010 section has future tense

[edit]

I believe it already happened. —Preceding unsigned comment added by 203.49.232.252 (talk) 23:59, 24 October 2010 (UTC)[reply]


Russ Abbott, Erroneous Link?

[edit]

I'm not sure Computer Science Professor Russ Abbott who judged the 2007 contest will be familiar to fans of Miss Funnyfanny and other characters brought to life by English musician, comedian and actor Russ Abbot. —Preceding unsigned comment added by 109.158.25.77 (talk) 17:37, 26 February 2011 (UTC)[reply]

2016

[edit]

The 2016 prize was won on 2016-09-17 by Steve Worswick's Mitsuku. I am pretty sure that as usual the winner did not persuade any of the judges that it was human. I have not added this information to the article because the source of my information is that I happened to be present when it was awarded, and that would be original research. I'm including it here because it may (1) be useful to someone and/or (2) provoke someone into adding it to the article once a reliable source is available. Gareth McCaughan (talk) 11:03, 18 September 2016 (UTC)[reply]

[edit]

Hello fellow Wikipedians,

I have just modified 2 external links on Loebner Prize. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:

When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.

This message was posted before February 2018. After February 2018, "External links modified" talk page sections are no longer generated or monitored by InternetArchiveBot. No special action is required regarding these talk page notices, other than regular verification using the archive tool instructions below. Editors have permission to delete these "External links modified" talk page sections if they want to de-clutter talk pages, but see the RfC before doing mass systematic removals. This message is updated dynamically through the template {{source check}} (last update: 5 June 2024).

  • If you have discovered URLs which were erroneously considered dead by the bot, you can report them with this tool.
  • If you found an error with any archives or the URLs themselves, you can fix them with this tool.

Cheers.—InternetArchiveBot (Report bug) 07:23, 9 December 2017 (UTC)[reply]

[edit]

Hello fellow Wikipedians,

I have just modified one external link on Loebner Prize. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:

When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.

This message was posted before February 2018. After February 2018, "External links modified" talk page sections are no longer generated or monitored by InternetArchiveBot. No special action is required regarding these talk page notices, other than regular verification using the archive tool instructions below. Editors have permission to delete these "External links modified" talk page sections if they want to de-clutter talk pages, but see the RfC before doing mass systematic removals. This message is updated dynamically through the template {{source check}} (last update: 5 June 2024).

  • If you have discovered URLs which were erroneously considered dead by the bot, you can report them with this tool.
  • If you found an error with any archives or the URLs themselves, you can fix them with this tool.

Cheers.—InternetArchiveBot (Report bug) 04:28, 5 January 2018 (UTC)[reply]

Math

[edit]

Á 2001:8003:3198:C00:9507:126F:5296:E990 (talk) 05:49, 1 December 2022 (UTC)[reply]

Retrieved from "https://en.wikipedia.org/w/index.php?title=Talk:Loebner_Prize&oldid=1208552448"