Comments on "Human-Computer Conversation"
by Yorick Wilks and Roberta Catizone (1999)
Chris Culy
culy@csli.stanford.edu
Summary: Overview of Human-machine conversation (HMC) and sketch of one system
Some historical points
Turing Test (1950)
ELIZA (Weizenbaum 1965, ubiquitous clones)
SHRDLU (Winograd 1972)
PARRY (Colby 1973; now defunct)
RACTER (Chamberlain and Etter 1984): http://www.mcs.net/~jorn/html/ai/racterfaq.html
Loebner competition (1991-present): http://www.loebner.net/Prizef/loebner-prize.html
State of Play
Tension between science and engineering (< Shieber); doesn't talk much about engineering (e.g. Eliza, many other Loebner winners) except for CONVERSE
W&C give 8 trends; regrouped here
1. AI: Model agents' beliefs, knowledge
2. AI: Model dialog linguistically (focus, failure, repair...)
3. AI: Script/frame based with (top-down) inferencing (i.e. of situtations)
5. AI: Model discourse based on speech acts; (bottom up) inferencing (i.e. of utterances)
-----------
4. Sociology: qualitative conversational analysis [not clear that this is HMC, but see 6,7]
6. Other: Model of 4 using general frames (cf. text grammar)
7. Other: Statistical analysis of discourse turns (cf. 4, 6); pattern matching; no "dialogue grammar"
-----------
8. Nirvana (not obtained yet): automatic acquisition of dialogue acts with rich representation using NLP as base.
Restaurant guide from last week: 5+
Converse: 3, 7, some 5; wants to add 1
Structure of CONVERSE
cf. http://www.loebner.net/Prizef/converse_works.html
(not based on scientific research p. 5)
Main idea: control conversation as much as possible (not new)
Use scripts, interaction w. databases (e.g. dictionaries, WordNet, Personality, general knowledge) to make responses reasonable
Commercial parser (Prospero) adapted for dialogue (based on corpus)
Steps in processing:
1. Rough preprocessing (e.g. split into sentences, some spelling, tag names)
2. Tag and parse
3. Post-process (=PPP): simplify/fix parse
3a. MicroQueries: information gathering after (3) e.g. lexical lookup
4. Submit each clause to actions; "highest bidder" takes action
a. Topic: topic tracking/switching
b. Pisces: handles requests for information; uses databases for answers
c. Information scavenger: information extraction from statements (fall-back)
d. Special case: "rudeness and rubbish"; repetition; tricks, etc (last stage)
5. Decide what to do
6. Generate output
Miscellaneous:
Loebner Prize
controversial:
Shieber: http://www.eecs.harvard.edu/shieber/papers/loebner-rev-html/loebner-rev-html.html
Loebner's response:
http://www.loebner.net/Prizef/In-response.html
Whalen (Thoughts on losing Loebner from previous winner)
http://debra.dgbt.doc.ca/chat/story95.html
CONVERSE wasn't the first non-US winner: Canadian (1994) and Australian (1996) were
1996 winner (year before CONVERSE): http://ciips.ee.uwa.edu.au/~hutch/hal/HEX/
Eliza clone; also different entries in other years: http://ciips.ee.uwa.edu.au/~hutch/hal/
1997 (2nd): ??
1998 (2nd): Markov model(s)
1999 (4th): "fractal-like"
1998, 1999 winner (years after CONVERSE): http://home.att.net/~r.g.garner/
Frames; fuzzy logic, finite state parsing; database
Agents/chatterbots
Agents as help-desk/customer service (already exist)
A couple chatterbot pages
http://www.toptown.com/hp/sjlaven/
http://bots.internet.com/search/s-chat.htm
Ranking of Programs by Judges
1 Julie 2 Human 3 Barry Defacto 4 Bob OS 5 SEPO 6 CONVERSE
Judge 1 5 1 4 6 3 2
Judge 2 4 1 3 6 2 5
Judge 3 4 1 5 6 2 3
Judge 4 4 1 5 6 3 2
Judge 5 3 1 4 6 5 2
---------------------------------------------------------------------------
Median 4 1 4 6 3 2
Mean 4 1 4.2 6 3 2.8
Judge 2's initial conversation with CONVERSE (excerpt in paper is from Judge 4):
PROGRAM[10:39:20]Hi, my name is Catherine, what's yours?
PROGRAM[10:39:27].
JUDGE02[10:39:36]My name is Nathan.
PROGRAM[10:39:37]Well, it's nice to meet you, Nathan.
PROGRAM[10:39:45]Did you see that story on CNN last night about the lesbian
PROGRAM[10:40:03]couple who came out at a White House party on Sunday?
JUDGE02[10:40:45]No. I'm sorry I missed that. How did the Prez take it?
PROGRAM[10:40:47]Ellen Degeneres was one of them - she was kissing her lover
PROGRAM[10:41:08]in front of Bill Clinton. Do you think he was very
PROGRAM[10:41:21]embarrassed?
JUDGE02[10:41:37]You tell me. I didn't see it.
PROGRAM[10:41:38]
PROGRAM[10:41:38]Are you still a Friend of Bill Clinton?
JUDGE02[10:42:22]Yes. I would like to talk about something else. Do you know anything about sculpture.
PROGRAM[10:42:24]Well, I suppose he's doing his best like most of us, but he
PROGRAM[10:42:35]does seem to attract an UNUSUAL amount of bad publicity,
PROGRAM[10:42:51]wouldnít you say--can it all be bad luck?
JUDGE02[10:43:20]I asked you if you knew anything about sculpture.
PROGRAM[10:43:21]OK, but what happens now--does it all drift on, with
PROGRAM[10:43:36]everyone waiting to see how much his friends have said about
PROGRAM[10:43:47]him under threat of lots and lots of jail-or will something
PROGRAM[10:44:01]dramatic actually happen?
PROGRAM[10:44:06].
JUDGE02[10:44:19]Do you know what sculpture is?
PROGRAM[10:44:20]Well, youíre more a fan of his than I am I guess---though I
PROGRAM[10:44:34]cant SWEAR what Id have done in that locked hotel room, I
PROGRAM[10:44:51]must say.
JUDGE02[10:45:26]Have you ever been to the Salmagundi Club?
PROGRAM[10:45:29]Never. to go back to the original discussion Time to get
PROGRAM[10:45:43]off this one I think and onto something more down to earth!
JUDGE02[10:46:20]What is more down to earth than art?
PROGRAM[10:46:21]I don't know what -- to be honest. -
PROGRAM[10:46:30]I suppose it depends on where you're coming from, but as the
PROGRAM[10:46:46]song goes in My Fair Lady, "Why can't a woman be more like a
PROGRAM[10:47:03]man? "
JUDGE02[10:47:52]I liked that song but I prefer "Why can't children be more like we are"
JUDGE02[10:48:10]from another play the name of which I cannot remember.
PROGRAM[10:48:12]What, if anything, do you enjoy about being a man?
JUDGE02[10:49:01]I would enjoy all aspects of life no matter what my gender.
PROGRAM[10:49:02]Do you think of yourself as being attractive?
JUDGE02[10:49:49]@@01