Overview

Raw Data
Stats
Common Errors

Sheet 1: Raw Data
Task ID Text Hypothesis Truth Value Guess (Rajat) Score (Rajat) Threshold (Rajat) Guess (Aria) Score (Aria) Threshold (Aria) Type of Reasoning Type of Implication Coreference Quantities Acronyms Nominalizations Idioms Negation Comments
CD 678

TRUE Correct 1.35
Correct 0.139 0.590 More Plausible Yes No No No No No Difficult to understand what pronoun “it” refers to, although this is necessary to get hypothesis correct
CD 693

TRUE Incorrect 12.05
Correct 0.533 0.590 More Plausible Yes Yes No No No No Two part hypothesis
CD 699

TRUE Correct 2.48
Correct 0.178 0.590 Syntactic Plausible No No No No No No The same item used in the same way is in quotes in the hypothesis but not in the text
CD 702

TRUE Correct 5.79
Correct 0.065 0.590 More Plausible No No No No No No Need to know Budapest is in Hungary
CD 713

TRUE Correct 0
Correct 0.055 0.590 Syntactic Entailed Yes No No No No No
CD 719

TRUE Correct 4.28
Incorrect 1.000 0.590 Lexical Plausible No No No No No No Need to know “popularly known as” and “called” are synonyms
CD 723

FALSE Incorrect 7.12
Incorrect 0.453 0.590 Lexical
Yes No No No No No Hypothesis repeats part of the text, but then concludes something not supported by the text
CD 733

TRUE Correct 0.5
Correct 0.175 0.590 Lexical Entailed Yes No No No No No Need to pair words referring to the same thing
CD 735

TRUE Incorrect 20.84
Correct 0.565 0.590 More Plausible Yes No No No No No Need to correctly dereference “it”, also need to know what economic activities includes
CD 750

TRUE Correct 1.9
Correct 0.070 0.590 Syntactic Entailed No No No No No No Hypothesis is basically a part of the longer text repeated
CD 802

TRUE Incorrect 13.2
Correct 0.394 0.590 More Plausible No No No No Yes No Need to know “seriously wounded” can be equated with “almost died” and that someone who tries to assassinate someone is a “would-be assassin”
CD 803

TRUE Correct 6.24
Correct 0.356 0.590 Lexical Entailed No No No No Yes No Need to know huge budget deficits results from deficits ballooning
CD 804

TRUE Correct 0.72
Correct 0.071 0.590 Syntactic Entailed No No No No No No Hypothesis makes a more general statement from a more specific text
CD 806

TRUE Incorrect 15.46
Incorrect 1.000 0.590 Lexical Entailed Yes Yes No No Yes No Semantically insignificant quotes around certain adjectives, need to knows “about 30” is the same as “in his 30s”
CD 809

TRUE Correct 0
Correct 0.000 0.590 Lexical Plausible No No No No No No Need to know that what authorities say can be concluded as true (usually)
CD 810

TRUE Correct 6.34
Incorrect 0.688 0.590 Lexical Plausible No No No No Yes No Need to know “appeared” and “scored a bit part” can be equated
CD 816

TRUE Incorrect 11.04
Correct 0.406 0.590 Lexical Entailed Yes Yes No No No Yes Need to know that raising something from 1% means that currently that thing is at no more than 1%
CD 818

TRUE Incorrect 19.82
Incorrect 1.000 0.590 Lexical Entailed Yes N ? No No No No Quantity isn't an exact quantity but “odds of getting...cancer rose” versus “markedly higher risks for...cancer”, need synonyms, coreferences
CD 819

TRUE Incorrect 8.33
Incorrect 0.688 0.590 Lexical Plausible No No No Yes No No Need to know that increase is the same as no more likely to
CD 820

TRUE Incorrect 9.49
Correct 0.333 0.590 Lexical Entailed No Yes No Yes No No Need to know conducted interviews is the same as interviewed
CD 823

TRUE Correct 5.33
Correct 0.218 0.590 More Plausible Yes No No No No No Correct, but seems to involve incorrect reasoning that concludes something more specific from something more general (in this case, that killer is synonymous with attacker)
CD 826

TRUE Incorrect 10.37
Correct 0.157 0.590 More Plausible Yes No No No No No Text and hypothesis only differ in that the text refers to a location as “baji” and the hypothesis refers to the location as “beiji”
CD 827

TRUE Correct 0.28
Incorrect 1.000 0.590 Syntactic Entailed Yes No No No No No Need to know if someone is from a country, they can be termed “<country>'s <insert attribute/name>
CD 829

TRUE Incorrect 10.79
Correct 0.586 0.590 Lexical Entailed No Yes No No Yes Yes Text makes a statement about the last time something happened, hypothesis rephrases using the construction “not since <the last time>”
CD 1806

FALSE Correct 16.01
Correct 1.000 0.590 Lexical
Yes Yes No No No No Text and hypothesis are related but not dealing with the same action
CD 1953

FALSE Correct 17.09
Correct 0.679 0.590 Lexical
Yes No No No No No Similar words used, but with crucial differences
CD 1989

FALSE Correct 10.34
Incorrect 0.474 0.590 Lexical
Yes Yes No No Yes No Hypothesis and text have same subjects but different verbs/actions
CD 2115

FALSE Incorrect 19.51
Correct 0.638 0.590 Lexical
No No No No No No Both discuss terror, heathrow airport, but are otherwise very different
CD 2116

FALSE Correct 25.83
Correct 1.000 0.590 Syntactic
No No No No Yes No Both talking about terror attacks, but completely different people/incidents
CD 2118

FALSE Correct 26.24
Correct 0.732 0.590 Lexical
No Yes No No No No text/hypothesis both discuss troops, but are otherwise dissimilar
CD 2119

FALSE Correct 17.86
Correct 0.795 0.590 Lexical
Yes No No No Yes No Subjects of the sentences are the same, but completely different actions
CD 2120

FALSE Correct 24.65
Incorrect 0.498 0.590 Syntactic
No Yes No No No No General text, hypothesis makes assumptions about the text, applies extra adjectives to make things more specific without justification
CD 2147

FALSE Correct 18.22
Correct 0.680 0.590 Lexical
Yes No No No No No Text discusses a witness's statements while hypothesis discusses television footage
CD 2154

FALSE Correct 8.2
Incorrect 0.414 0.590 Lexical
No No No No No No Need to know antonyms
CD 2156

FALSE Correct 12.76
Correct 0.700 0.590 Syntactic
Yes No No No No No Hypothesis makes a statement about what the text discussed but states things not in the text
CD 2158

FALSE Correct 20.78
Correct 0.624 0.590 Lexical
Yes No No No No No Sentences involve similar words but constructed in ways to create different meanings
CD 2159

FALSE Incorrect 6.67
Incorrect 0.536 0.590 Lexical
Yes No No No No No Text is long, hypothesis is short so doesn't incur as much penalty
CD 2160

FALSE Correct 15.16
Correct 1.000 0.590 Lexical
Yes No No No No No Both discuss ship's captain, but otherwise very different
CD 2161

FALSE Correct 17.27
Correct 0.660 0.590 Syntactic
No No No No No No Hypothesis and text do not discuss the same action (one is about a captain being fired, the other is about the captain refusing to talk)
CD 2164

FALSE Correct 18.53
Correct 0.735 0.590 Syntactic
No Yes No No No No Same number and company used in both text/hypothesis, but otherwise very different
CD 2165

FALSE Correct 12.24
Correct 0.626 0.590 Lexical
Yes No No No Yes No Same subject, different reasons
CD 2166

FALSE Correct 14.72
Correct 0.638 0.590 More
No No Yes No Yes No
CD 2170

FALSE Correct 17.55
Correct 0.672 0.590 Syntactic
Yes No No No No No Same person described involved in two different actions
CD 2172

FALSE Correct 13.7
Correct 0.597 0.590 Lexical
Yes No No No No No Hypothesis introduces subjects not discussed in the text
CD 2174

FALSE Correct 9.9
Correct 0.732 0.590 Syntactic
No No No No No No Hypothesis and text do not discuss the same action (one is about broadcasting, the other about problems with authorities)
CD 2177

FALSE Correct 10.56
Correct 0.653 0.590 Syntactic
No No No No No No text/hypothesis have same subject but different objects
CD 2178

FALSE Correct 15.31
Correct 1.000 0.590 Lexical
No No No No No No Both discuss HIV and breastfeeding, but from two different angles
CD 2191

TRUE Correct 1.16
Correct 0.080 0.590 Lexical Plausible Yes No No No No No Need to figure out reference for “its”
IE 944

FALSE Incorrect 4.35
Incorrect 0.254 0.590 Lexical
Yes No No No No No Need to correctly attach modifier phrases to what they are modifying
IE 1467

FALSE Correct 10.54
Correct 1.000 0.590 Lexical
No No No No No No Hypothesis makes claim about the subject of the text that is not mentioned in the text
IE 1599

TRUE Correct 7.75
Correct 0.406 0.590 More Entailed No No No No No No Need to know that a takeover offer implies someone is trying to take over a company, also that one takes over a company (not a board or other aspect of the company)
IE 1600

TRUE Incorrect 11.89
Correct 0.492 0.590 More Plausible Yes No No No Yes No Need to know that offering to buy a company is taking it over
IE 1601

FALSE Incorrect 7.51
Incorrect 0.249 0.590 Lexical
No No No No Yes No “Tries to take over” does not follow from someone becoming the new owner
IE 1604

FALSE Incorrect 0.91
Incorrect 0.323 0.590 Syntactic
Yes No No No No No Need to know that the verbs “killed” and “kidnapped” refer to the brother of the education minister, not the education minister
IE 1605

TRUE Incorrect 12.91
Correct 0.250 0.590 Lexical Entailed No No No No No No Need to know that “Hostage taking” means someone was kidnapped
IE 1612

TRUE Incorrect 10.01
Correct 0.457 0.590 Lexical Plausible No No No No No No Need to correctly parse, identify that one's home state is probably where one was born
IE 1614

TRUE Correct 3.79
Correct 0.127 0.590 More Plausible Yes No No No No No Need to know that one's hometown is usually the place one was born
IE 1619

TRUE Correct 0.71
Correct 0.239 0.590 Syntactic Entailed No No No No No No Need to correctly attach modifier “illinois born” to charleton heston and decode
IE 1626

TRUE Correct 2.2
Correct 0.312 0.590 Lexical Plausible No No No No No No Need to know that “is from” probably means “was born in”
IE 1631

TRUE Correct 2.53
Correct 0.359 0.590 Syntactic Entailed Yes No No No No No Need to correctly determine the reference for “This”
IE 1635

TRUE Correct 2.21
Correct 0.293 0.590 Lexical Entailed No No No Yes No No Need to know that an assassin of someone is someone who killed that person
IE 1636

TRUE Correct 3.08
Correct 0.298 0.590 Lexical Plausible No No No No No No Need to know that if one is convicted of murdering someone, one probably killed that person
IE 1637

TRUE Correct 7.1
Correct 0.281 0.590 Lexical Plausible No No No No No No Need to know if someone is shot, then later dies in the hospital, the person who shot them killed them
IE 1638

TRUE Correct 6.25
Incorrect 1.000 0.590 More Entailed Yes No No No No No Need to understand “alias” and connect modifiers correctly
IE 1639

TRUE Correct 2.35
Correct 0.148 0.590 Lexical Entailed No No No No No No Need to know that someone who murders someone kills that person
IE 1652

TRUE Correct 3.36
Correct 0.460 0.590 Syntactic Entailed No No No No No No Hypothesis is mainly a repeat of part of the text
IE 1658

TRUE Correct 4.25
Correct 0.482 0.590 More Entailed No No No No No No Need to know “<city name>, <state name>” means that <city> is in <state>
IE 1659

TRUE Correct 5.72
Correct 0.318 0.590 Syntactic Entailed No No No No No No Need to correctly attach modifier phrases to what they are modifying
IE 1665

TRUE Correct 7.25
Correct 0.531 0.590 More Plausible No No No No No No Need to understand that if someone graduates from an organization in a specific place, the organization is probably based in that place
IE 1902

FALSE Incorrect 0.12
Incorrect 0.000 0.590 Lexical
No No No No No No Need to know that just because something “can” happen, it didn't necessarily happen
IE 1908

FALSE Correct 15.38
Incorrect 0.343 0.590 Syntactic
Yes No No No No No Need to correctly attach modifier phrases to what they are modifying
IE 1910

FALSE Incorrect 4.02
Incorrect 0.097 0.590 Syntactic
Yes No No No No No Need to figure out the proper reference for “He”
IE 1914

FALSE Incorrect 5.05
Incorrect 0.152 0.590 Syntactic
No No No No Yes No Need to figure out what killed has as subject/object
IE 1917

TRUE Correct 4.83
Correct 0.371 0.590 Lexical Entailed No Yes No No Yes No Need to connect the facts that only one election happened and that someone won an election in 1990 to deduce that the only election took place in 1990
IE 1918

TRUE Correct 5.65
Incorrect 1.000 0.590 More Entailed No No No No Yes No Need to know “entered” is synonymous with “Take part in”, also that the organizing committee of something is a good source for things about that thing
IE 1926

FALSE Incorrect 5.84
Incorrect 0.367 0.590 More
Yes No No No No No This sentence pair may be misclassified – my reading is that the hypothesis follows from the text, but that's not how it is marked
IE 1927

FALSE Incorrect 7.91
Incorrect 0.505 0.590 Lexical
Yes No No No No No Need to correctly attach modifier phrases to what they are modifying
IE 1929

TRUE Correct 4.46
Correct 0.518 0.590 More Entailed Yes No No No Yes No Need to do chains of reasoning about what it means for one team to be joined by another and what attributes of the first team are then taken on by the second team (e.g., both are participating in the same tournament)
IE 1930

TRUE Correct 7.32
Correct 0.383 0.590 Lexical Entailed No No No Y ? No No Need to identify that a leader of a country participating means that a country participates
IE 1931

TRUE Correct 5.75
Correct 0.372 0.590 Lexical Entailed Yes No No No No No Extract hypothesis from phrasal structure
IE 1933

FALSE Incorrect 6.34
Incorrect 0.273 0.590 Lexical Entailed No No No No Yes Yes Need to know that if a conference is “killed” it means it won't be happening
IE 1934

TRUE Correct 2.96
Correct 0.200 0.590 Syntactic Entailed Yes No No No No No Need to correctly match descriptive phrases with what they are describing
IE 1935

FALSE Incorrect 4.34
Incorrect 0.318 0.590 Syntactic
Yes No No No No No Need to correctly match descriptive phrases with what they are describing
IE 1939

FALSE Incorrect 5.67
Incorrect 0.291 0.590 Lexical
Yes No No No No No Need to correctly attach modifier phrases to what they are modifying
IE 1946

FALSE Incorrect 4.24
Incorrect 0.477 0.590 Lexical Entailed No No No No Yes No Need to know “ease back” means lowered and that lowered/rose are antonyms
IE 1949

FALSE Incorrect 7.35
Incorrect 0.302 0.590 Lexical Entailed No No No No Yes No Need to know “fall back” means lowered and that lowered is an antonym of rose
IE 1950

FALSE Incorrect 2.94
Correct 1.000 0.590 Lexical
No No No No Yes No Need to know “dips below” is an antonym for “rise”
IE 1951

TRUE Correct 3.76
Correct 0.412 0.590 Lexical Entailed No No No No Yes No Need to know that “shot up” is the same as rose is this context
IE 2001

FALSE Incorrect 4.42
Incorrect 0.460 0.590 Syntactic
No No No No No No Need to figure out what each modifier refers to
IE 2002

FALSE Incorrect 7.49
Incorrect 0.382 0.590 Lexical
No No No No No No Need to understand the concept of “liason offices” and that if someone establishes something somewhere, it does not necessarily mean their entire organization is based there
IE 2004

FALSE Incorrect 4.8
Incorrect 0.402 0.590 Lexical
No No No No No No Hypothesis has many of the same words as the text, but uses a different (non-synonymous) verb
IE 2007

FALSE Incorrect 7.95
Incorrect 0.387 0.590 Syntactic
Yes No No No No No Hypothesis uses many of the same words as the text, but incorrectly matches a modifying phrase and what it modifies
IE 2011

FALSE Incorrect 5.33
Incorrect 0.088 0.590 Lexical
No No No No Yes No Sentence is somewhat ungrammatical
IE 2013

TRUE Correct 5.38
Correct 0.361 0.590 Lexical Entailed Yes No No No No No Need to know that belonging to/being a member of are synonyms
IE 2015

FALSE Incorrect 4.25
Correct 0.632 0.590 Lexical
No No No No No No Need to do more than just have the same words in both text/hypothesis
IE 2075

FALSE Incorrect 3.79
Correct 0.638 0.590 Lexical
No Y ? No No No No Need to know “born”/”died” are antonyms, quantity is the year which is applied to both born and died – this should incur a cost
IE 2083

FALSE Incorrect 4.09
Incorrect 0.275 0.590 Lexical
Yes No No No No No Need to know that parts of organization can be established at different times than the entire organization
IE 2085

FALSE Incorrect 4.94
Incorrect 0.220 0.590 More
Yes No No No No No Need to know that conclusions about an entire company cannot be made from info about a small part of the company
IR 24

TRUE Incorrect 10.44
Correct 0.571 0.590 More Plausible Yes No No No No No Need to know that a physician is a good source of medical info
IR 36

TRUE Incorrect 17.44
Correct 0.535 0.590 More Plausible Yes No No No No No Need to know what surfing is (to know that injured surfers probably are surfing accident victims), also that “Scripps hospital” can refer to those who work at the hospital
IR 72

FALSE Incorrect 5.11
Correct 1.000 0.590 More
Yes No No No No No Lots of synonyms, soft negative “withdrew”
IR 100

TRUE Correct 5.24
Correct 0.551 0.590 Lexical Entailed Yes No No No No No Need to know that someone who sexually molests a child is a pedophile
IR 102

TRUE Correct 7.2
Corect 0.505 0.590 More Entailed Yes No No No No No “failed to act” is the same as “ignored” - soft negative
IR 105

TRUE Correct 3.84
Correct 0.183 0.590 Lexical Entailed No No No No No No Need to know that if something is voted to happen, it happens
IR 112

FALSE Incorrect 3.13
Incorrect 0.367 0.590 Lexical
Yes Yes No No No No Need to reason which of the many numbers in the text refer to what
IR 128

TRUE Incorrect 10.28
Correct 0.530 0.590 More Plausible No No No No Yes No Hypothesis is oddly worded – questionable item
IR 137

TRUE Incorrect 8.07
Correct 0.055 0.590 Syntactic Entailed No No No No Yes No Hypothesis rearranges part of text
IR 256

TRUE Incorrect 11.4
Correct 0.570 0.590 More Entailed Yes Yes No No No No Need to reason that 80% is greater than 50% and thus if 80% of people are something, most people are that thing
IR 258

FALSE Correct 10.86
Correct 0.599 0.590 More
Yes No No No No No Lots of “they” references, hypothesis suggests a they reference which is incorrect
IR 260

TRUE Correct 6.98
Correct 0.466 0.590 Lexical Entailed Yes No No No No No Need to know “social integration” is “integration into society”, very complex sentence – subject of sentence isn't well defined
IR 262

TRUE Incorrect 8.56
Correct 0.542 0.590 Lexical Entailed Yes No No No Yes No Have to know professors are scholars, synthesize the long text into the short hypothesis
IR 279

FALSE Correct 3.68
Incorrect 0.288 0.590 Syntactic
No No No No No No Need to correctly determine what the places in the verb predicate refer to – the passive voice seems to cause confusion
IR 286

FALSE Incorrect 7.78
Incorrect 0.323 0.590 Lexical
Yes No No No Yes Yes Soft negation, invoked in idiom, key to understanding sentence
IR 2137

FALSE Incorrect 6.53
Incorrect 0.135 0.590 Lexical
No No No No No No Need to correctly determine that prime minister is not a coreference for foreign minister
IR 2140

FALSE Incorrect 4.68
Incorrect 0.494 0.590 Lexical
No No No No Yes Yes Negation is soft “unlikely to become involved in” - need a better negation detector
IR 2141

FALSE Incorrect 0
Incorrect 0.055 0.590 Syntactic Entailed No No No No No Yes Hypothesis includes almost all the words from the text, but fails to include “not” - negation clearly handled badly because cost is 0.0
IR 2142

FALSE Incorrect 3.34
Incorrect 0.532 0.590 Lexical
Yes No No No No Yes Hypothesis includes almost all the words from the text, but fails to include “not”
IR 2143

FALSE Incorrect 0.67
Incorrect 0.447 0.590 More
Yes No No No No No Need to make chain of reasoning that if there are no weapons here, and there are as many WMDs in Iraq as weapons here, there must be no WMDs in iraq – this is the negation of the hypothesis. Thus reasoning may involve negation
MT 1148 The American television network "A.B.C." quoted undisclosed sources at the American Defense Department today, Thursday, saying that the intense bombing operations on Iraq may cease by the end of this week. The television network "CNN", quoting sources in the American Ministry of Defence, revealed that the bombing of Iraq might end this week. FALSE Correct 9.95 8.03 Incorrect 0.503 0.590 Lexical
Yes Yes Yes No No No Everything seems to be true, except for the name of the news agency, which could throw off our system. I labeled this as involving “quantity” because it invovles temporal reasoning. Hopefully, our system should not match the two different uses of “end”.
MT 1149 The American State Department announced that Russia recalled her ambassador to the United States "for consultation" due to the bombing operations on Iraq. The American Ministry of Foreign Affairs announced that Russia called the United States about the bombings on Iraq. FALSE Correct 14.38 8.03 Incorrect 0.275 0.590 Lexical
Yes No No No No No
MT 1150 The humane organization, Caritas, quoting its branch in Iraq, announced today, in Germany, that several Iraqi hospitals, especially Saddam Hospital, which is considered the biggest hospital in Iraq, were hit in the American-British bombing of the Iraqi capital. Caritas announced that several Iraqi hospitals, (excluding Saddam Hospital,) were hit in the shelling of Baghdad. FALSE Incorrect 7.15 8.03 Incorrect 0.351 0.590 Lexical
Yes No No No No No
MT 1152

TRUE Correct 5.88
Correct 0.431 0.590 Lexical Plausible Yes No No N ? No No Need to know that if someone's funeral is held, they are buried at that time, also that British and Anglo refer to the same thing
MT 1153

TRUE Incorrect 11.65
Correct 0.257 0.590 More Entailed Yes No Yes No Yes No Need to know that “children of Iraq” means people from Iraq, also understand “a.k.a” and do many synonyms
MT 1155

FALSE Correct 8.49
Incorrect 0.493 0.590 Lexical
Yes No No No No No Need to know that if something is recommended, it is not necessarily obligated
MT 1156

FALSE Correct 16.46
Incorrect 0.397 0.590 Lexical
Yes No No No No No Need to correctly determine tenses – text refers to past tense and hypothesis refers to future, concerning the same event
MT 1158 An official of the International Special Commission in charge of disarming Iraqi weapons (UNSCOM) announced yesterday, Saturday, that the Commission will keep its inspectors in Iraq despite Baghdad's decision to stop its cooperation with them. UNSCOM is to stay in Iraq. TRUE Correct 5.03 8.03 Correct 0.281 0.590 More Plausible Yes No Yes No No No Must recognize that UNSCOM stays in Iraq if its inspectors stay in Iraq
MT 1159

FALSE Correct 10.98
Incorrect 0.295 0.590 Lexical
Yes No No No Yes No Need to know that titles are exact (can't necessarily replace with synonyms), also uses more ambiguous senses of words that cannot be replaced by synonyms as the hypothesis tries
MT 1160

FALSE Incorrect 7.89
Correct 1.000 0.590 Lexical
Yes No No No Yes No Nedd to attach negative correctly – hypothesis concludes “want them to leave” from the text “will not ask them to depart” - it ignores the negation
MT 1162

FALSE Correct 9.12
Incorrect 0.383 0.590 Lexical
Yes Yes No No Yes No Hypothesis substitutes “Wilson” for “UNSCOM” in text
MT 1164 Ramadan told reporters at the opening ceremony of the Baghdad International Exposition, "No cooperation and no inspection or monitoring by the American Zionist espionage commission (the Special Commission for disarming Iraq's banned weapons - UNSCOM) before Iraq's demands are met.". Ramadan announced that Iraq will not cooperate with the inspectors. TRUE Incorrect 13.42 8.03 Incorrect 1.000 0.590 More Entailed Yes No No Yes Yes No Need to understand that it is Iraq that will not cooperate.
MT 1165 Aziz confirmed that Iraq's demands include "the lifting of the blockade and the restructuring of the espionage commission, including its Chairman, (Richard) Butler. Iraq demands the lifting of the embargo and changes in the committee. TRUE Correct 6.1 8.03 Correct 0.346 0.590 Lexical Entailed No No No Yes No No
MT 1172 The American House of Representatives is due to vote today, on a resolution submitted by the Chairman of the Judicial Committee, Henry Hyde, asking Congress to launch an investigation over charges brought forward by Independent Prosecutor-General Kenneth Star against President Clinton in the Monica Lewinsky affair. Voting will take place in the American House of Representatives today, on the President Clinton-Monica Lewinsky scandal. TRUE Incorrect 8.52 8.03 Correct 0.371 0.590 More Plausible No No No Yes Yes No Idiom: launch an investigation. Must recognize that the text refers to the same “President Clinton-Monica Lewinsky” scandal as the hypothesis.
MT 1173

FALSE Correct 8.6
Incorrect 0.312 0.590 Lexical
No No No No No Yes Need to correctly process negation “ought not to be removed” is an antonym of “should be removed”
MT 1178

TRUE Incorrect 9.26
Correct 0.370 0.590 Lexical Plausible Yes No No No Yes No Need to know that “gearing up” is synonymous with preparing, quotes in text around “Mitch” are not semantically significant, also that Latin American countries include South American countries
MT 1179 Security authorities have declared a state of maximum emergency in Guatemala, which is located directly in the path of the hurricane. There is a state of maximum emergency in Guatemala because of the hurricane. TRUE Correct 7.27 8.03 Correct 0.405 0.590 More Plausible No No No No No No Need cause and effect reasoning to determine that the hurricane is the cause of the state of emergency.
MT 1182

TRUE Incorrect 14.96
Correct 0.298 0.590 Lexical Entailed Yes No No No Yes No Need to parse information out of a quote, summarize the text using very different words
MT 1186

FALSE Correct 11.67
Correct 0.590 0.590 Lexical
Yes No No No No No Need to know that asserting is not the same as querying
MT 1190

TRUE Incorrect 11.87
Correct 0.389 0.590 Lexical Entailed Yes No Yes No Yes No Need to know what “human rights violations” includes, determine that ANC is African National Congress
MT 1191

FALSE Correct 17.05
Correct 0.602 0.590 Lexical
Yes No No Yes Yes No Need to know “prevent them” and “give them the opportunity” are antonyms
MT 1197

TRUE Incorrect 9.1
Incorrect 0.616 0.590 Lexical Entailed Yes No No No Yes No Need to summarize words in quotes, also do coreference between “parties” and “delegates”
MT 1199

TRUE Incorrect 10.38
Correct 0.497 0.590 Lexical Plausible Yes No No No Yes No Hypothesis substitutes “pressurizing” for “puts pressure on” in reference to people – this seems incorrect, although otherwise the two sentences are synonymous
MT 1205 Israel controls an area of about 850 square kilometres in the south of the Lebanese Republic, while Syria has about 30 thousand soldiers over two thirds of the Lebanese territory. Syria has 30,000 soldiers in 60% of Lebanon, and Israel controls an area in the south of 850 square kms. TRUE Incorrect 12.73 8.03 Correct 0.199 0.590 More Entailed Yes Yes No No No No Need to know “Lebanon” = “Lebanese territory”, “in the South” refers to the South of Lebanon”
MT 1207 Eddeh reasserted his opinion that President Al-Hrawi ought to be reinstated for another 3 years, claiming that the he "has considerable experience in the running of the state, but his resignation would be appropriate after the withdrawal of Israeli and Syrian forces.". Eddeh calls for an extension to Al-Hrawi's presidency, saying that he should resign after the withdrawal of the Israeli and Syrian forces.". TRUE Correct 7.49 8.03 Incorrect 1.000 0.590 Lexical Entailed Yes No No Yes Yes No Needs to recognize that “would be appropriate” in this context means “should”.
MT 1211 The Croatian will face competition, especially from the American of Chinese origin, Michael Chang, second-seeded in the tournament. Organizers hope that the tournament will contribute to advancing tennis in their countries. The Croatian and the Asian-American, Michael Chang (number two) will play eachother, and hopefully popularise the game in their native countries. TRUE Incorrect 18.95 8.03 Correct 0.556 0.590 More Entailed Yes No No No Yes No Needs to connect “face competition” and “play each other,” “American of Chinese origin” and “Asian-American”, and “advancing” and “popularize”, and recognize that “the game” refers to tennis.
MT 1215 A statement issued by Royal Moroccan Airlines, a copy of which was received by Agence France Presse today, said that the company decided to start two flights a week between Casablanca and Gaza as soon as the Palestinian airport is opened there. With the opening of the Palestinian airport, Royal Moroccan Airlines will fly between Casablanca and Gaza. TRUE Correct 5.14 8.03 Correct 0.256 0.590 More Plausible Yes No No Yes No No Need to recognize “the company” refers to “Royal Moroccan Airlines”, “two flights a week” = fly, “as soon as” = “with the”, etc.
MT 1219 The Tunisian Foreign Ministry announced that Libyan leader Colonel Moammar Al Kaddafy arrived in Tunis this morning, Monday, for a "brotherly and recreational" visit, at the invitation of Tunisian President Zine El Abidine Bin Ali. Libya's leader Colonel Muammar Qaddafi arrived in Tunisia at the invitation of President Zine El Abidine Bin Ali. TRUE Incorrect 12.5 8.03 Correct 0.266 0.590 More Plausible No No No No No No Only implied because organizations can announce things that are false. Also need to know Tunis is in Tunisia.
MT 1226

TRUE Incorrect 8.26
Correct 0.343 0.590 Syntactic Entailed Yes No No No Yes No Need to parse and rearrange clauses
MT 1228

TRUE Incorrect 12.99
Correct 0.424 0.590 More Plausible Yes No No No Yes No Complicated sentence with lots of coreference
MT 1230 The Yemeni Islamic Jihad group emerged in 1992, and it is comprised of old warriors from Afghanistan. Yemen's Islamic Jihad group was set up by old Afgan warriors in 1992. TRUE Incorrect 15.33 8.03 Correct 0.402 0.590 More Plausible No No No No No No Need to infer that the members who comprise the group probably founded it.
MT 1232

TRUE Incorrect 11.76
Correct 0.486 0.590 Lexical Entailed Yes No No No No No Need to know that “voters” in this context is a synonym for “Damascus public”
MT 1244

TRUE Incorrect 12.42
Correct 0.442 0.590 Lexical Entailed Yes No No No No No Hypothesis is grammatically bad
MT 1246 The Iraqi Foreign Minister, Mohammad Said Al-Sahaf, met today with the new ambassadors and briefed them "on the Iraqi diplomacy directives of escalating the move toward lifting the blockade imposed on Iraq and developing relations with various countries of the world.". Mohammed Said AL-SAHAF met with the ambassadors and briefed them on Iraq's diplomacy directives. TRUE Correct 3.48 8.03 Correct 0.049 0.590 Syntactic Entailed Yes No No No No No
MT 1777 Nevertheless, 58 percent of Americans expressed their satisfaction with the President's job performance. 44 percent supported Nixon at the very beginning of the investigation into Watergate. 58% of Americans are unsatisfied with Clinton's job performance. FALSE Correct 9.24 8.03 Incorrect 0.540 0.590 Lexical
Yes Yes No Yes No No
MT 2094 It was like hitting the jackpot. It was like removing the lottery. FALSE Correct 16.76 8.03 Correct 0.620 0.590 Lexical
No No No No No No
MT 2095

FALSE Incorrect 7.26
Incorrect 0.260 0.590 More
Yes No No No No No Hypothesis has bad grammar, incorrectly assumes “bursts” means explosions even though it has a separate scientific meaning
MT 2098 U.S. consumer spending dived in June. The cost of the consumer of the United States fell in June. FALSE Incorrect 7.2 8.03 Incorrect 0.304 0.590 Lexical
No No No No No No Only syntactic information is needed to categorize this as False.
MT 2099

FALSE Correct 22.75
Incorrect 0.298 0.590 Lexical
Yes No Yes No Yes No Need to determine that “consumer spending” is not a synonyn of “the cost of the consumer”
MT 2100 Greenspan's stance has underpinned predictions the Fed will raise rates steadily this year. The position of Greenspan has propped up the predictions that the Fed will gradually elevate the east interest rates year. FALSE Incorrect 6.53 8.03 Incorrect 0.363 0.590 (Ungrammatical)
No No No No Yes No Ungrammatical.
MT 2101 Both candidates are making a major push in Iowa. Both candidates are delivering a great attack in Iowa. FALSE Correct 13.83 8.03 Incorrect 0.261 0.590 Lexical
No No No No No No Push does not imply attack.
MT 2104

FALSE Correct 13.71
Incorrect 0.469 0.590 Lexical
No No No No Yes No Text is grammatically very bad
MT 2105

FALSE Correct 21.41
Incorrect 0.380 0.590 Lexical
Yes No Yes No No No Need to know that acronyms cannot be switched (tries to substitute OMC for WTO and UE for EU)
MT 2107

FALSE Correct 17.93
Correct 0.604 0.590 Syntactic
No No No No Yes No Hypothesis has completely incorrect grammar – could probably solve by parsing the hypothesis and noting how unlikely it is
MT 2110 Journalists of the concealed BBC, that was made happen through advisers of London industralists, asked Slavkov. BBC undercover reporters, posing as consultants acting for London businessmen, ask Slavkov. FALSE Correct 21.89 8.03 Incorrect 0.546 0.590 (Ungrammatical)
No No No No No No Ungrammatical.
MT 2111 Slavkov, accused in the Salt Lake scandal but later cleared, said he had been approached by Takatch. Slavkov, defendant in the scandal of Salt Lake but that later was exculpated, in fact said that Takatch. FALSE Correct 12.12 8.03 Incorrect 0.534 0.590 (Ungrammatical)
Yes No No No No No Ungrammatical.
PP 34 Saudi Arabia, the biggest oil producer in the world, was once a supporter of Osama bin Laden and his associates who led attacks against the United States. Saudi Arabia is a key U.S. ally. FALSE Correct 12.26 8.03 Incorrect 0.407 0.590 Syntactic
No No No No No No
PP 1429 Alcohol now fuels 44 percent of all violent crime and 70 percent of Accident and Emergency hospital admissions at peak times are due to booze. Alcohol has an effect on violent crimes. TRUE Incorrect 9.56 8.03 Correct 0.532 0.590 Lexical Entailed No Yes No No Yes No I say quantity because it matters that 44 percent is non-zero. “Has an effect” can be considered an idiom.
PP 1434 The report catalogues 10 missed opportunities within the CIA and FBI to uncover pieces of the September 11 plot. Ten missed opportunities within the CIA and FBI are uncovered in the report. TRUE Correct 3.98 8.03 Correct 0.221 0.590 Lexical Plausible No Yes No No No No I call this implication because it assumes the missed opportunities were previously unknown. Hopefully, the system should not pair the different uses of “uncover”.
PP 1435 Blair has sympathy for anyone who has lost their lives in Iraq. Blair is sympathetic to anyone who has lost their lives in Iraq. TRUE Correct 6.79 8.03 Correct 0.273 0.590 Lexical Entailed No No No No No No
PP 1435 Blair has sympathy for anyone who has lost their lives in Iraq. Blair is sympathetic to anyone who has lost their lives in Iraq. TRUE Correct 6.79 8.03 Correct 0.273 0.590 Syntactic Entailed No No No No Yes No I counted “has sympathy for” as an idiom, but it could alternatively be considered a nominalization.
PP 1436 Blair has sympathy for anyone who has lost their lives in Iraq. Blair is sorry for anyone who has lost their lives in Iraq. TRUE Correct 6.6 8.03 Correct 0.255 0.590 Lexical Entailed No No No No Yes No I counted “has sympathy for” as an idiom, but it could alternatively be considered a nominalization.
PP 1437 Iraq would have been an unsafe, unsecure country that did pose a threat to the world. Iraq would have constituted a threat to the world. TRUE Correct 3.74 8.03 Correct 0.220 0.590 Lexical Entailed No No No No No No
PP 1438 Iraq would have been an unsafe, unsecure country that did pose a threat to the world. A threat to the world would have arisen from Iraq. TRUE Incorrect 10.32 8.03 Correct 0.301 0.590 Lexical Entailed No No No No No No Needs to connect “X poses a threat” and “a threat arises from X”
PP 1439 The prosecutor told the court that the incident had caused "distress" to one of the children. The prosecutor told the court that "distress" in one of the children is associated with the incident. TRUE Correct 2.57 8.03 Correct 0.174 0.590 Lexical Entailed No No No No No No
PP 1959 Kerry hit Bush hard on his conduct on the war in Iraq. Kerry shot Bush. FALSE Incorrect 2.51 8.03 Incorrect 0.110 0.590 Lexical
No No No No Yes No
PP 1965 Rich and poor nations struck a historic deal on Sunday to slash billions of dollars in farm subsidies, create more open industrial markets and revive stalled world trade talks that could boost global growth. Rich and poor nations crashed into a historic deal on Sunday. FALSE Incorrect 1.71 8.03 Incorrect 0.335 0.590 Lexical
No No No No No No
PP 1966 Relatives of a missing woman and her husband said late Saturday that new information from the husband has prompted them to call off a volunteer search for the woman. Relatives of a missing woman and her husband cancelled a volunteer search for the woman. TRUE Correct 1.81 8.03 Correct 0.284 0.590 Lexical Entailed Yes No No No No No Must understand the semantics of “prompted”
PP 1970 The witness then spotted Rodriguez crouching behind a gate for protection. The witness rescued Rodriguez. FALSE Incorrect 3.15 8.03 Incorrect 0.482 0.590 More
No No No No No No
PP 1971 Rodriguez was spotted by a witness running away from the stairwell where the backpack was found seconds before it went off. A witness rescued Rodriguez. FALSE Incorrect 6.58 8.03 Correct 0.670 0.590 Lexical
No No No No No No
PP 1975 A matching fingerprint was discovered on the strap. A matching fingerprint was excavated from the strap. FALSE Incorrect 6.59 8.03 Incorrect 0.270 0.590 Lexical
No No No No No No
PP 1976 Bomb-sniffing dogs were brought to Rodriguez's Mulberry St. apartment. Bomb-sniffing dogs were estimated at Rodriguez's Mulberry St. apartment. FALSE Correct 8.96 8.03 Incorrect 0.097 0.590 Lexical
No No No No No No
PP 1985 Quebec woman and her mother accused of plotting to kill a four-year-old girl. Quebec woman murdered a four-year-old girl. FALSE Incorrect 3.08 8.03 Incorrect 0.210 0.590 Lexical
No No No No No No Must understand “accused of” does not imply truth
PP 1986 The girl was found in Drummondville. Drummondville contains the girl. FALSE Correct 12.09 8.03 Incorrect 0.557 0.590 ???
No No No No No No The wording is a bit awkward, and ambiguous (e.g. found alive or dead?)
PP 1993 Profits nearly doubled to nearly $1.8 billion. Profits nearly doubled from nearly $1.8 billion. FALSE Incorrect 0 8.03 Incorrect 0.110 0.590 Lexical
No No No No No No Requires knowledge of the argument structure of “double”
PP 2050 Five other soldiers have been ordered to face courts-martial. Five other soldiers have been refused to face courts-martial. FALSE Incorrect 2.98 8.03 Incorrect 0.125 0.590 Lexical
No No No No No No
PP 2051 Protesters, many from organized pro-government groups but including many ordinary citizens, carried anti-American banners and chanted slogans attacking U.N. Secretary-General Kofi Annan for his close alignment with U.S. policy. Protesters confiscated anti-American banners and chanted slogans attacking U.N. Secretary-General Kofi Annan for his close alignment with U.S. policy. FALSE Incorrect 4.73 8.03 Incorrect 0.061 0.590 Lexical
No No No No No No The sentences are exactly the same except for “carried” vs. “confiscated”, which could fool our system.
QA 595 Brown brushed off a threat made Monday by his counterpart, Foreign Trade Minister Wu Yi, that if China's effort to join the General Agreement on Tariffs and Trade (GATT) by the year's end is unsuccessful, Beijing will no longer be bound by previous trade and economic commitments. Wu Yi is the Foreign Trade Minister of China. TRUE Correct 5.8 8.03 Correct 0.346 0.590 More Plausible Yes No No No No No Must recognize that “Foreign Trade Minister Wu Yi” is not one big noun phrase, but rather “Foreign Trade Minister” is Wu Yi's title. Must also deduce somehow that he is Chinese.
QA 600 Thus began the journey that led Hannam, a 29-year-old free-lance journalist, on an international quest to solve one of the greatest mysteries of Asia: What happened to Lin Piao, the Chinese Communist Party leader accused of a 1971 plot to overthrow Chairman Mao. Lin Piao was the Chinese Communist Party leader. TRUE Correct 4.33 8.03 Correct 0.165 0.590 Syntactic Entailed Yes No No No No No
QA 617 In 1954, in a gesture of friendship to mark the 300th anniversary of Ukrainian union with Russia, Soviet Premier Nikita Khrushchev gave Crimea to Ukraine. Crimea became part of Ukraine in 1954. TRUE Correct 7.53 8.03 Correct 0.569 0.590 More Entailed No No No No No No Must know that when country X gives territory Z to country Y, Z becomes part of Y.
QA 622 Witnesses of genocide attacks on minority Tutsis in Rwanda are being singled out for execution by extremist Hutus in refugee camps in southwest Rwanda, a U.N. spokesman said in Kigali, the capital. The Hutu and Tutsi groups fought in Rwanda. TRUE Incorrect 17.01 8.03 Correct 0.587 0.590 More Plausible Yes No No No No No Hard. I don't know how it would deduce from this that the two groups fought.
QA 628 The human species may be more than 100,000 years old but the bulk of its numerical growth has come in just the last few decades: 2.6 billion of today's 5.6 billion people have been born since 1960. The world population is 5.6 billion. TRUE Correct 7.13 8.03 Correct 0.423 0.590 More Entailed No Yes No No No No Must determine that “today's 5.6 billion people” refers to the world population.
QA 636 How come a country (Vatican City), a so-called country, that is in essence 800 square acres of office space in the middle of Rome, that has a citizenry that excludes women and children, seems to attract the most attention in talking about public policy that deals with women and children? The Vatican City is in Rome. TRUE Correct 0.76 8.03 Correct 0.440 0.590 Lexical Plausible Yes No No No No No I call this implication because the phrase is in quotes.
QA 637 In 1969, more than 500 million people around the world sat in front of television sets, watching grainy images of two men in white bulky spacesuits planting a U.S. flag on the lunar landscape with its black horizon. The Apollo astronauts waved the American flag on the moon in 1969. TRUE Incorrect 10.9 8.03 Incorrect 0.736 0.590 More Plausible Yes No No No No No This sentence is very hard. It would need to somehow deduce that the men in spacesuits are Apollo astronauts, that those Americans weren't watching reruns from the '50s, and so on.
QA 642 For hundreds of family members holed up in a hotel not far from the Stockholm ferry terminal, where the boat Estonia was to have docked Wednesday morning after an overnight trip from the Estonian capital of Tallinn, the suspended rescue effort was just the latest in a day of heartbreaking reports. The Estonia set sail from Tallinn. TRUE Correct 5.89 8.03 Correct 0.276 0.590 More Entailed Yes No No No Yes No
QA 643 Practically every architect of international stature has vied for the bonanza of public and private projects -- including Renzo Piano, Peter Eisenman, Philip Johnson, Rafael Moneo, Helmut Jahn and Richard Rogers. Renzo Piano is an architect. TRUE Correct 4.33 8.03 Correct 0.366 0.590 Syntactic Entailed No No No No No No
QA 648 Yoko Ono, widow of murdered Beatles star John Lennon, has plastered the small German town of Langenhagen with backsides. Yoko Ono was John Lennon's wife. TRUE Correct 2.63 8.03 Correct 0.533 0.590 Lexical Entailed No No No No No No
QA 663 A remastered version of Paramount's "Breakfast at Tiffany's," starring Audrey Hepburn, is scheduled for release Nov. 2 at $40. Audrey Hepburn starred in "Breakfast at Tiffany's". TRUE Correct 0.96 8.03 Correct 0.177 0.590 Lexical Entailed No No No No No No “Starring” could probably be handled lexically.
QA 1453 The county Board of Supervisors accepted more than $700,000 in state grants Tuesday to extend a program that provides the controversial drug azidothymidine -- AZT -- to low-income residents with AIDS or AIDS-related illnesses. AZT is an AIDS treatment. TRUE Incorrect 10.95 8.03 Correct 0.366 0.590 More Plausible Yes No No No No No
QA 1454 In fact, Woolsey had had no first-hand experience with the world of spies until President Bill Clinton appointed him Director of Central Intelligence. James Woolsey is the director of the CIA. TRUE Incorrect 13.35 8.03 Correct 0.512 0.590 More Plausible Yes No Yes No No No Must somehow connect “Director of Central Intelligence” with “director of the CIA”, even though the acronyms are not an exact match. Must connect James Woosle with Woosley and “him” with Woosley.
QA 1456 On his oak mantelpiece are a drinking bowl from Lake Turkana and an East African gourd, both of which he bought in Nairobi where he went in the fall of 1993 to visit Kitum Cave, the suspected host site for the Marburg virus, which is closely related to Ebola. The Ebola virus has been found in Zaire. FALSE Correct 11.06 8.03 Correct 0.749 0.590 Syntactic
No No No No No No
QA 1474 To avoid further price-gouging, Norway's Ministry of Government Administration announced that it was freezing prices on lodging, food and beverages in the Lillehammer area during the Winter Olympics. Norway hosted the Winter Olympic Games. TRUE Incorrect 8.26 8.03 Incorrect 0.594 0.590 More Plausible Yes No No No Yes No Hard.
QA 1486 Now it's Romania-Argentina at the Rose Bowl today -- and with a victory, it's on to the quarterfinals at Stanford, and a victory there and Romania will be back in Pasadena for the semifinals and World Cup final. The World Cup final was held in Pasadena. TRUE Incorrect 9.11 8.03 Correct 0.564 0.590 More? Plausible No No No No No No Maybe this can be done lexically, because “in X for Y” implies that Y is located in X.
QA 1490 It's Only a Play Broadway is in New York. TRUE Correct 0.83 8.03 Correct 0.385 0.590 More Plausible No No No No No No I'm not quite convined that the hypothesis “follows” from the sentence, and if so, deducing it requires some complex reasoning.
QA 1493 We've never had a core (business) thing, Richard Branson's cola is called Virgin Cola. TRUE Correct 4.2 8.03 Correct 0.473 0.590 More Plausible Yes No No No Yes No Must recognize that Virgin Cola is a product, that it is Richard Branson's product, and that it is a kind of cola.
QA 1494 The demands on Italy's 53rd postwar government are apparent, as are the resentment and uncertainties Berlusconi has roused at the head of a new party with the catchy name Forza Italia (Go, Italy). Silvio Berlusconi is in Forza Italia. TRUE Incorrect 15.38 8.03 Correct 0.390 0.590 More Entailed No No No No Yes No Must know that Forza Italia is an organization, “at the head of” means “as the head of”, and if you are the head of an organization, you are in it.
QA 2044 Jacqueline B. Wender is Assistant to the President of Stanford University. Jacqueline B. Wender is the President of Stanford University. FALSE Incorrect 2.75 8.03 Incorrect 0.110 0.590 Syntactic
No No No No No No
QA 2046 Hoover's director reports directly to Stanford's president. Hoover is the president of Stanford University. FALSE Incorrect 7.08 8.03 Incorrect 0.197 0.590 Syntactic
No No No No No No
QA 2057 The Hotel Moulin de Paris in 2.5km away from Disneyland Paris, and is located in the charming city of Magny-Le-Hongre. EuroDisney is located in the charming city of Magny-Le-Hongre. FALSE Correct 8.49 8.03 Incorrect 0.239 0.590 Syntactic
Yes No No No No No
QA 2059 At eurodisney, once upon a time is now in this magical kingdom where childhood fantasies and make-believe come to life. EuroDisney is located in this magical kingdom. FALSE Incorrect 4.73 8.03 Incorrect 0.560 0.590 Syntactic
No No No No No No “Once upon a time is now” is tough to interpret, but probably isn't necessary for deciding this one.
QA 2060 On this particular trip, Airbus, in cooperation with its partners, organized a trip for 80 children between the ages of 9 and 16 years from nearby National Children's Homes to visit EuroDisney for a day. EuroDisney is located nearby National Children's Homes. FALSE Incorrect 6.39 8.03 Incorrect 0.360 0.590 More
No No No No No No Must determine that National Children's Homes is nearby Airbus, not EuroDisney.
QA 2061 Although Eurodisney is a great family day out, visit also the chateaux in the area and the various parks and pretty small villages. EuroDisney is located in the various parks. FALSE Incorrect 4.8 8.03 Incorrect 0.464 0.590 Syntactic
No No No No No No
QA 2064 The Osaka World Trade Center is the tallest building in Western Japan. The Osaka World Trade Center is the tallest building in Japan. FALSE Incorrect 0 8.03 Correct 1.000 0.590 More
No No No No No No Involves reasoning with superlatives.
QA 2065 The slender tower is the second tallest building in Japan. The slender tower is the tallest building in Japan. FALSE Incorrect 0 8.03 Correct 1.000 0.590 More
No No No No No No Requires knowledge of superlatives.
QA 2066 The tallest building in Tokyo and the second tallest building in Japan, the TMGO was conceived as a high-tech center from which Tokyo could be led into the promising twenty-first century. TMGO is the tallest building in Japan. FALSE Incorrect 3.55 8.03 Incorrect 0.360 0.590 More
No No No No No No Superlatives.
QA 2067 Andreessen, who helped define the Internet revolution as part of team that created the first Internet browser (Mosaic) and his co-founding Netscape, told a packed hall at the San Francisco Marriott hotel Thursday that he is "extremely committed" to his startup Loudcloud. The Internet browser Mosaic was created at the San Francisco Marriott hotel. FALSE Incorrect 5.87 8.03 Incorrect 0.282 0.590 Syntactic
Yes No No No No No
QA 2068 There are more than 4,000 street children in the capital city of Ulan Bator. The population of Ulan Bator is 4,000. FALSE Correct 12.11 8.03 Incorrect 0.500 0.590 More
No Yes No No No No
QA 2069 Grieving father Christopher Yavelow hopes to deliver one million letters to the queen of Holland to bring his children home. Christopher Yavelow is the queen of Holland. FALSE Incorrect 3.02 8.03 Correct 1.000 0.590 Syntactic
Yes No No No No No I say coreference because it requires deciding that the two terms are NOT coreferent.
QA 2072 In 1932 he joined the then illegal Communist party and held high government and party posts from 1942, becoming home secretary in 1948, when the Communist party took control in Hungary. Communist control in Hungary ended in 1948. FALSE Incorrect 7.32 8.03 Correct 1.000 0.590 Lexical
Yes Yes No No Yes No I say quantity because the years are important, and coreference because it requires recognizing that “Communist party” and “Communist” in “Communist control” refer to the same entity.
QA 2078 Twelve of Jupiter's moons are relatively small and seem to have been more likely captured than to have been formed in orbit around Jupiter. Jupiter has Twelve moons. FALSE Correct 8.45 8.03 Incorrect 0.485 0.590 More
No Yes No No No No This sentence could be true or false, depending on your interpretation, as in the classic riddle “Which months have 28 days?”
QA 2080 VCU School of the Arts In Qatar is located in Doha, the capital city of Qatar. Qatar is located in Doha. FALSE Incorrect 0.06 8.03 Incorrect 0.275 0.590 Syntactic
Yes No No No No No Must know that “VCU School of the Arts In Qatar” and “Qatar” do not refer to the same entity.
QA 2081 The main race track in Qatar is located in Shahaniya, on the Dukhan Road. Qatar is located in Shahaniya. FALSE Incorrect 3.98 8.03 Incorrect 0.229 0.590 Syntactic
No No No No No No
QA 2084 Microsoft Israel was founded in 1989 and became one of the first Microsoft branches outside the USA. Microsoft was established in 1989. FALSE Incorrect 1.9 8.03 Incorrect 0.397 0.590 More
Yes No No No No No Must know Microsoft Israel does not refer to the same entity as Microsoft.
QA 2086 Jigish Avalani, group manager of Windows DNA marketing at Microsoft, explains that Windows DNA refers to the Windows Distributed interNet Application architecture, launched by Microsoft in fall of 1997. Microsoft was established in fall of 1997. FALSE Incorrect 1.5 8.03 Incorrect 0.424 0.590 Syntactic
No No No No No No
QA 2089 Prince Charles was previously married to Princess Diana, who died in a car crash in Paris in August 1997. Prince Charles and Princess Diana got married in August 1997. FALSE Incorrect 7.6 8.03 Incorrect 0.172 0.590 Syntactic
No No No No Yes No “Got married” may be considered an idiom.
RC 172 The terrorist is suspected of being behind several deadly kidnappings and dozens of suicide attacks in Iraq. The terrorist may have caused suicide attacks and kidnappings. TRUE Correct 6.21 8.03 Correct 0.513 0.590 Lexical Plausible No No No No Yes No
RC 179 The SEC's new rule to beef up the independence of mutual fund boards represents an industry defeat. Industries were defeated by the SEC. FALSE Incorrect 3.98 8.03 Incorrect 0.196 0.590 Lexical
No No No No Yes No The two uses of “industry
RC 180 The Securities and Exchange Commission's new rule to beef up the independence of mutual fund boards represents an industry defeat. The SEC's new rule will give boards independence. TRUE Correct 4.86 8.03 Correct 0.388 0.590 Lexical Entailed Yes No Yes No Yes No
RC 182 AT and T Corp. says it will stop selling traditional local and long-distance residential service in seven states, blaming its move on a court decision that it says will result in higher prices for its use of regional networks. AT and T blames a court decision that will cause higher prices. TRUE Correct 7.49 8.03 Correct 0.304 0.590 Lexical Plausible Yes No No No No No
RC 191 Oil prices edged up today as Norwegian employers said they would respond to a strike by locking out all workers next week, threatening to shut off almost all oil and gas output. Norwegian employers caused oil prices to rise. TRUE Incorrect 10.01 8.03 Correct 0.524 0.590 More Plausible No No No No Yes No
RC 194 Wal-Mart has received a lot of negative publicity recently, including allegations that it used illegal workers and made employees work without pay during lunch breaks, as well as complaints that it generally underpays employees. Wal-Mart complains about negative publicity. FALSE Incorrect 2.29 8.03 Incorrect 0.287 0.590 Syntactic
No No No No No No Should not connect two uses of “complain”
RC 198 Even though the rug was ripped out from under Lollapalooza this week, Jane's Addiction frontman Perry Farrell is optimistic that the eclectic music festival he co-founded in 1991 will resurface and it might be sooner than you think. Perry Farrel hopes to found another music festival. FALSE Correct 9.7 8.03 Incorrect 0.360 0.590 More
Yes No No No Yes No “Resurface” might be considered an idiom.
RC 238 Following the assassination attempt in 1981, Reagan said he felt God had spared him for a purpose, and he intended to devote the rest of his life in dedication to his God and to that purpose. Regan was almost assassinated in 1981. TRUE Incorrect 12.78 8.03 Incorrect 0.757 0.590 More Plausible Yes No No Yes No No
RC 242 On opening day Salesforce.com's stock was up $6.20 to $17.20, a 56.4% increase over the initial offering price of $11. Opening day of the stock of Salesforce.com rose 56.4% over the initial offering price. TRUE Incorrect 8.15 8.03 Correct 0.240 0.590 More Entailed Yes Yes No No Yes No The grammatical structures of these sentences are a bit ambiguous, which probably makes them hard to parse.
RC 331 Chancellor Schroeder has presided over three years of almost zero growth in the economy and an unemployment rate that has remained stubbornly above four million people. More than four million people have remained stubbornly unemployed in the last three years. FALSE Correct 9.71 8.03 Incorrect 0.371 0.590 More
No Yes No No No No Tricky.
RC 345 Ahern, who was travelling to Tokyo for an EU-Japan summit yesterday, will consult with other EU leaders by telephone later this week in an effort to find an agreed candidate. A summit between Europe and Japan is taking place in the Japanise capital. TRUE Incorrect 20.26 8.03 Correct 0.555 0.590 More Plausible Yes No Yes No No No Need to know Tokyo is the capital of Japan, and that EU is the acronym for the European Union, which in diplomatic contexts is the same as Europe.
RC 346 Ahern, who was travelling to Tokyo for an EU-Japan summit yesterday, will consult with other EU leaders by telephone later this week in an effort to find an agreed candidate to presidency of the European commission. EU leaders found an agreement about a candidate to the post of European commission president. FALSE Correct 8.62 8.03 Incorrect 0.385 0.590 More
No No No No No No
RC 347 Protest votes as citizens of the 25 EU nations punished their governments. The European Union is made up of 25 nations. TRUE Incorrect 16.09 8.03 Incorrect 0.611 0.590 More Plausible Yes Yes Yes No No No I don't know how to treat “the 25 EU nations”.
RC 348 Protest votes as citizens of the 25 EU nations punished their governments for everything from high unemployment to involvement in Iraq. EU nations' citizens protest against punishment for unemployment. FALSE Incorrect 4.2 8.03 Incorrect 0.308 0.590 Lexical
No No No No No No
RC 349 Protest votes as citizens of the 25 EU nations punished their governments for everything from high unemployment to involvement in Iraq. Citizens of the 25 EU nations are punished by their governments for everything. FALSE Incorrect 5.58 8.03 Incorrect 0.164 0.590 Syntactic
No No No No No No
RC 908 Time Warner is the world's largest media and Internet company. Time Warner is the world's largest company. FALSE Correct 9.21 8.03 Incorrect 0.202 0.590 More
No No No No No No Superlatives.
RC 912 Vice President Dick Cheney on Tuesday hurled an obscenity on the Senate floor to punctuate an angry exchange with Vermont Sen. Patrick Leahy as all senators gathered for their annual photo. Cheney cursed at Sen. Patrick Leahy. TRUE Correct 7.61 8.03 Correct 0.371 0.590 More Plausible Yes No No No Yes No
RC 916 Medicare will begin paying later this year for certain oral cancer drugs and intravenous drugs that can be self-administered under a $500 million demonstration project unveiled yesterday. Medicare will start paying users $500 million. ( FALSE Correct 16.76 8.03 Incorrect 0.379 0.590 Syntactic
No Yes No No No No
RC 917 Medicare will select by lottery 10 percent of those who apply by Aug. 16 to begin receiving benefits starting Sept. 1. Medicare will choose 10 percent of applicants to receive benefits by Aug. 16. FALSE Correct 10.38 8.03 Incorrect 0.150 0.590 Lexical
No No No Yes No No I counted “applicants” as a nominalization, although strictly it may not be.
RC 918 Toshiba has produced a fuel cell with no moving parts. Toshiba has no moving parts. FALSE Correct 9.84 8.03 Incorrect 0.360 0.590 Syntactic
No No No No No No
RC 921 Saginaw police responded to a domestic violence complaint at the wrong house and ended up shooting the owners' dog. Saginaw police mistakenly went to the wrong house in response to a complaint. TRUE Correct 7.78 8.03 Correct 0.459 0.590 More Plausible No No No No No No
RC 923 The Arabic-language television network Al-Jazeera reports it has received a statement and a videotape from militants. Al-Jazeera is an Arabic-language television network. TRUE Correct 0.24 8.03 Correct 0.327 0.590 Syntactic Entailed No No No No No No
RC 925 A rocket attack killed two Israelis, including a 3-year-old boy. A three-year-old was killed in a rocket attack. TRUE Correct 2.61 8.03 Correct 0.167 0.590 Syntactic Entailed Yes Yes No No No No
RC 926 Startling new research into mobile phones claims they may reduce a man's sperm count by up to 30%. Male fertility may be affected by use of a mobile phones. TRUE Incorrect 9.03 8.03 Incorrect 0.618 0.590 More Plausible Yes Yes No No No No
RC 927 More than 150 dolphins, marine turtles and beaked whales have been washed up dead on beaches in Africa. Dead dolphins, turtles and whales have been found on African beaches. TRUE Incorrect 19.6 8.03 Correct 0.306 0.590 More Plausible No Yes No No No No Need to reason that we would only know the sea creatures are on the beaches if someone found them there.
RC 928 Suncreams designed for children could offer less protection than they claim on the bottle. Suncreams designed for children protect at the level they advertise. FALSE Incorrect 5.67 8.03 Incorrect 0.360 0.590 More
Yes Yes No Yes Yes Yes Although not strictly negation, “offer less” should be taken as evidence that not as much protection is offered.
RC 930 After giving nearly 5,000 people a second chance at life, doctors are celebrating the 25th anniversary of Britian's first heart transplant which was performed at Cambridgeshire's Papworth Hospital in 1979. The first heart transplant in Britian was performed in 1979. TRUE Correct 2.57 8.03 Correct 0.260 0.590 Syntactic Entailed No No No No No No
RC 973 Political leaders pledged that construction of the so-called Freedom Tower -- which will rise 1,776 feet into the air and be the world's tallest building -- will be finished on schedule by the end of 2008. The Freedom Tower will be the world's tallest building. TRUE Correct 0.25 8.03 Correct 0.196 0.590 Syntactic Entailed No No No No No No
RC 975 After watching fireworks yesterday evening in Cedar Rapids, Kerry went to his wife's suburban Pittsburgh farm, where he was expected to hold a barbecue for supporters this afternoon. Kerry watched fireworks. TRUE Correct 5.39 8.03 Correct 0.229 0.590 Syntactic Entailed No No No No No No
RC 984 Cool, humid weather Sunday helped slow the advance of a fire that caused the evacuation of hundreds of homes and businesses in Alaska's Interior. There was a fire in Alaska. TRUE Correct 6.73 8.03 Correct 0.478 0.590 More Entailed No No No No No No
RC 988 Merrill Lynch and Co. and Smith Barney, now a unit of Citigroup, in 1998 settled discrimination cases involving hundreds of female employees. Merrill Lynch and Co. and Smith Barney are now a unit of Citigroup. TRUE Correct 3.49 8.03 Correct 0.208 0.590 Syntactic Entailed Yes No No No No No
RC 991 Wal-Mart Stores has asked a US federal appeals court to review a judge's order approving class-action status for a sex-discrimination lawsuit. A lawsuit was filed against Wal-Mart. TRUE Incorrect 9.45 8.03 Correct 0.431 0.590 More Plausible No No No No No No
RC 992 Wal-Mart Stores has asked a US federal appeals court to review a judge's order approving class-action status for a sex-discrimination lawsuit. The judge approves of sex-discrimination. FALSE Incorrect 6.62 8.03 Incorrect 0.381 0.590 Syntactic
No No No No No No
RC 1001 The so-called "grandmother hypothesis", based on studies of African hunter-gatherer groups, suggests that infertile women are vital for successful child-rearing despite being unable to produce children themselves. The "grandmother hypothesis" suggests that infertile women are very important for raising children. TRUE Correct 6.56 8.03 Correct 0.254 0.590 Lexical Entailed No No No No No No
RC 1002 The so-called "grandmother hypothesis", based on studies of African hunter-gatherer groups, suggests that infertile women are vital for successful child-rearing despite being unable to produce children themselves. Infertile women are vital for successful child-rearing, according to the "grandmother hypothesis". TRUE Correct 6.65 8.03 Correct 0.278 0.590 Lexical Entailed No No No No No No
RC 1003 Infertile women are vital for successful child-rearing despite being unable to produce children themselves. Infertile women cannot produce children. TRUE Correct 6.47 8.03 Correct 0.434 0.590 Lexical Entailed Yes No No No No No
RC 1005 Jessica Litman, a law professor at Michigan's Wayne State University, has specialized in copyright law and Internet law for more than 20 years. Jessica Litman is a law professor. TRUE Correct 0.49 8.03 Correct 0.229 0.590 Syntactic Entailed No No No No No No
RC 2128 The kidnappers want the Kuwaiti firm that employs the men to stop doing business in Iraq and to pay compensation to the victims of U.S. strikes in Falluja. The kidnappers paid compensation to the victims of the U.S. FALSE Incorrect 3.01 8.03 Incorrect 0.225 0.590 Lexical
No No No No No No
RC 2131 U.S. light crude was trading erratically on the New York Mercantile Exchange, hitting $44.50 a barrel before easing back a bit to $44.10, still a gain of $1.27. The U.S. light crude is $1.27 a barrel. FALSE Incorrect 5.67 8.03 Incorrect 0.269 0.590 Lexical
No Yes No No Yes No
RC 2132 Ralph Fiennes, who has played memorable villains in such films as 'Red Dragon' and 'Schindler's List,' is to portray Voldemort, the wicked warlock, in the next Harry Potter movie. Ralph Fiennes will play Harry Potter in the next movie. FALSE Incorrect 3.38 8.03 Incorrect 0.322 0.590 Syntactic
No No No No No No
RC 2180 Speaking outside the hospital moments after getting her child back, Ciancio said she was confident all along that the baby would be found. Ciancio was hospitalized for speaking to the baby. FALSE Incorrect 5 8.03 Incorrect 0.459 0.590 Syntactic
Yes No No No No No The two uses of “industry” should not be paired.
RC 2181 Two bombs planted near an Islamic school in Pakistan killed eight people and injured 42 others yesterday in the latest outbreak of violence gripping the southern port of Karachi. 42 people killed in Karachi bomb attack. FALSE Correct 14.66 8.03 Incorrect 0.360 0.590 Syntactic
No No No No No No
RC 2182 Two bombs planted near an Islamic school in Pakistan killed eight people and injured 42 others yesterday in the latest outbreak of violence gripping the southern port of Karachi. Eight people planted two bombs near an Islamic school in Pakistan. FALSE Incorrect 7.24 8.03 Incorrect 0.294 0.590 Syntactic
No No No No No No
RC 2183 One of the dead was a child, passing by with his parents, said Iqrar Abbasi, a doctor at Civil Hospital Karachi. A doctor was killed by his parents. FALSE Incorrect 4.08 8.03 Incorrect 0.499 0.590 Syntactic
No No No No Yes No
RC 2184 Scotland Yard has been granted two more days to question nine of the 13 men arrested in co-ordinated anti-terrorism raids across the country last Tuesday. 13 men killed in co-ordinated anti-terrorism raids across the country last Tuesday. FALSE Incorrect 2.66 8.03 Incorrect 0.249 0.590 Lexical
No No No No No No
RC 2185 The nine men, aged between 19 and 32, are being held at Paddington Green high security police station in central London. between 19 and 32 men are being held at Paddington Green high security police station in central London. FALSE Incorrect 4.94 8.03 Incorrect 0.102 0.590 Syntactic
No Yes No No No No
RC 2186 One of two men "released" early on Saturday Mohammed Dawoud, 19, of Willesden, north-west London will appear before Horseferry Road magistrates' court today charged with possession of forged identity documents, police said. Willesden will appear before Horseferry Road magistrates' court today. FALSE Incorrect 3.04 8.03 Incorrect 0.213 0.590 Syntactic
Yes No No No No No
RC 2188 A second man, aged 24, technically freed yesterday was bailed after being questioned over alleged forged identity documents, but has been further detained on immigration matters, a spokesman for Scotland Yard said. A spokesman for Scotland Yard was freed yesterday. FALSE Incorrect 3.89 8.03 Incorrect 0.309 0.590 Syntactic
No No No No No No
RC 2189 Abu Eisa al-Hindi was said to have been involved in a plot to attack Heathrow airport, details of which were allegedly discovered on the computer of Abu Eisa al-Hindi, 25, an al-Qa'ida suspect recently arrested in Pakistan. Abu Eisa al-Hindi is the father of Abu Eisa al-Hindi. FALSE Incorrect 7.8 8.03 Incorrect 0.189 0.590 Syntactic
No No No No No No

Sheet 2: Stats
Rajat:























Task Total True % Correct of True # False % Correct of False % Correct % True # True-Entailed % Correct of True-Entailed # True-Plausible % Correct of True-Plausible # More-True % Correct of More-True # Lexical-True % Correct of Lexical-True Total Syntactic-True % Correct Syntactic-True







CD 24 0.58 24 0.88 0.73 0.5 11 0.55 13 0.62 7 0.43 2 0.5 4 1







IE 25 0.88 25 0.08 0.48 0.5 18 0.94 7 0.71 8 0.88 12 0.83 5 1







IR 10 0.4 10 0.2 0.3 0.5 7 0.57 3 0 5 0.2 4 0.75 1 0







MT 23 0.3 22 0.77 0.53 0.51 13 0.23 10 0.4 11 0.27 10 0.3 2 0.5







PP 9 0.67 12 0.25 0.43 0.43 8 0.75 1 1 0 0 8 0.63 1 1







QA 18 0.61 20 0.2 0.39 0.47 8 0.88 10 0.4 13 0.46 3 1 2 1







RC 25 0.32 24 0.67 0.49 0.51 13 0.92 11 0.36 11 0.27 6 1 7 1


















































































Task Total Coref-True % Correct of Coref-True Total Coref-False % Correct of Coref False Total Quantity T % Correct Quantity True Total Quantity False % Correct Quantity False Total Acronyms True % Correct Acronyms True Total Acronyms False % Correct Acronyms False Total Noms True % Correct Noms True Total Noms False % Correct Noms False Total Idioms True % Correct Idioms True Total Idioms False % Correct Idioms False # Neg-T % Correct Neg-T # Neg-F % Correct Neg-F
CD 12 0.5 13 0.85 5 0 5 1 0 0 1 1 2 0 0 0 5 0.4 5 1 2 0 0 0
IE 8 0.88 11 0.09 1 1 1 0 0 0 0 0 2 1 0 0 5 0.8 7 0 0 0 1 0
IR 7 0.43 6 0.17 1 0 1 0 0 0 0 0 0 0 0 0 2 0 3 0 0 0 4 0
MT 18 0.28 15 0.8 2 1 2 0.5 3 0.33 3 1 5 0.6 2 1 12 0.08 6 0.83 0 0 1 1
PP 1 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 3 0.67 1 0 0 0 0 0
QA 10 0.5 6 0.17 1 1 3 0.67 1 0 0 0 0 0 0 0 4 0.5 2 0 0 0 0 0
RC 11 0.55 4 0.25 5 0.2 5 0.2 3 0.33 0 0 1 0 2 0.5 5 0.6 5 0.2 0 0 1 0

























Aria:























Task Total True % Correct of True # False % Correct of False % Correct % True # True-Entailed % Correct of True-Entailed # True-Plausible % Correct of True-Plausible # More-True % Correct of More-True # Lexical-True % Correct of Lexical-True Total Syntactic-True % Correct Syntactic-True







CD 24 0.75 24 0.79 0.73
11 0.73 13 0.77 7 1 12 0.5 4 0.75







IE 25 0.92 25 0.84 0.48
18 0.72 7 1 8 0.75 12 1 5 1







IR 10 1 10 0.2 0.3
7 0.86 3 1 5 1 4 1 1 1







MT 23 0.87 22 0.22 0.53
13 0.77 10 1 11 0.91 10 0.8 2 1







PP 9 1 12 0.08 0.43
8 1 1 1 0 0 8 1 1 1







QA 18 0.89 20 0.28 0.39
8 1 10 0.8 13 0.85 3 1 2 1







RC 25 0.88 24 0 0.49
13 1 11 0.73 11 0.73 6 1 7 1


















































































Task Total Coref-True % Correct of Coref-True Total Coref-False % Correct of Coref False Total Quantity T % Correct Quantity True Total Quantity False % Correct Quantity False Total Acronyms True % Correct Acronyms True Total Acronyms False % Correct Acronyms False Total Noms True % Correct Noms True Total Noms False % Correct Noms False Total Idioms True % Correct Idioms True Total Idioms False % Correct Idioms False # Neg-T % Correct Neg-T # Neg-F % Correct Neg-F
CD 12 0.75 13 0.77 5 0.8 5 0.6 0
1 1 2 0.5 0
5 0.6 5 0.8 2 1 0
IE 8 0.88 11 0 1 1 1 1 0
0
2 1 0
5 0.8 7 0.14 0
1 0
IR 7 1 6 0.33 1 1 1 0 0
0
0
0
2 1 3 0 0
4 0
MT 18 0.83 15 0.2 2 1 2 0 3 1 3 0 5 0.6 2 0.5 12 0.75 6 0.33 0
1 0
PP 1 1 0
2 1 0
0
0
0
0
3 1 1 0 0
0
QA 10 0.8 6 0.33 1 1 3 0.33 1 1 0
0
0
4 0.75 2 0.5 0
0
RC 11 0.73 4 0 5 0.6 5 0 3 0.67 0
1 0 2 0 5 1 5 0 0
1 0

Sheet 3: Common Errors
Problem Code Text Hypothesis

Hypothesis repeats words in text, but doesn't discuss the same thing 723 The city continued to grow through much of the 20th century. The city continued to grow, but its services deteriorated.

Also cases where hypothesis repeats words in text but draws incorrect conclusion 2159 The city ferry fleet shuttles about 70,000 people a day between Staten Island and Manhattan, a 5.2-mile trip across New York Harbor that takes about 25 minutes. The ferry taking commuters from Manhattan to Staten Island crashed into a pier. 


2015 BTA said those injured in the fire at Borovets, 22 miles southeast of Sofia, were from Britain, the Soviet Union, West Germany, the Netherlands, Greece and Denmark. Sofia is located in Britain.


2004 Bosch was found innocent in Venezuela of the bombing charges for a third time Aug. 7. Bosch lives in Venezuela. 































Synonym Issues 818 The new study refutes earlier findings by researchers at the University of California, Los Angeles, who concluded that the odds of getting head and neck cancers rose in tandem with the frequency and duration of marijuana use. The latest findings contradict a California study that implicated regular pot smoking as having markedly higher risks for head and neck cancers.


819 Recreational marijuana smokers are no more likely to develop oral cancer than nonusers. Smoking marijuana does not increase the risk of developing oral cancer.
Multi word synonyms could be important

802 Reagan was seriously wounded by a bullet fired by John Hinckley Jr. in an attempted assassination on March 30, 1981. Reagan almost died on March 30, 1981, when would-be assassin John W. Hinckley Jr. shot him. 


735 Even more than other economic activities, Mexico's financial services are concentrated in the capital. Industry, retail stores, finance, and communications are all centered in the capital. 


820 Kessler's team conducted 60,643 face-to-face interviews with adults in 14 countries. Kessler's team interviewed more than 60,000 adults in 14 countries. 


806 Investigators described the gunman as white, about 30 years old, of medium build with shoulder-length, wavy blond hair and as having a "scruffy" or "grunge" appearance. The gunman is described as a white male in his 30s with a medium build, shoulder-length blonde hair and an unkempt appearance. 


1605 Deployment of workers to Iraq has been halted after the reported hostage-taking of a Filipino man by gunmen. Filipino man has been kidnapped by gunmen. 


2137 Swedish Foreign Minister murdered. Swedish prime minister murdered. 


262 Winning Team John C. Harsanyi, John F. Nash and Reinhard Selten were jointly awarded the Nobel Prize in Economics. The three professors were singled out for their unique contributions to game theory, derived from studies of games such as poker or chess, in which players have to think ahead and devise a strategy based on expected countermoves from opponents. Three scholars shared a Nobel Prize for their studies about the game theory.


1178 Sources in Tegucigalpa reported that many Latin American countries are gearing up to face Hurricane "Mitch". South American countries are preparing for "Mitch". 


1153 Sultan Al-Shawi, a.k.a the Attorney, said during a funeral held for the victims, "They were all children of Iraq killed during the savage bombing.". The Attorney, said at the funeral, "They were all Iraqis killed during the brutal shelling.".  


1190 The Commission, designated to shed light on crimes and human rights violations during the apartheid period, accuses the African National Congress of carrying out torture and executions without trials in concentration camps, and also of playing an active role in the political violence incidents that took place between 1990 and 1994. The Commission accused the ANC of violence, executions and human rights violations. 


1232 The Damascus public have been welcomed into the election candidate's private homes or to specially erected tents where voters have been enjoying food, drink and entertainment, particularly in the home of the wealthy candidate, Adnan Mullah, where dozens of supporters and friends have been invited to dinner every evening. Candidates are entertaining voters in their homes. 


1197 Contact with the press will be restricted to "perodic meetings", as promised by all three parties, in order to "focus their energies" on the most important issues. The delegations may only speak periodically to the press, in order to concentrate on the issues.  

























Antonym Issues (this is related to synonym issues) 2075 Queen Victoria died in 1901, at Osborne House on the Isle of Wight. Queen Victoria was born in 1901. 


1946 Oil prices eased back from record high levels after Russia's justice ministry said embattled oil giant Yukos can continue oil production and sales. Oil prices rise.


1950 Crude oil dips below $43 on news that Russia's justice ministry will not force Yukos to halt sales. Crude oil rises.


1949 Oil prices fall back as Yukos oil threat lifted. Oil prices rise.  



















Modifier from text applied to incorrect entity in hypothesis 2007 The program, financed with a grant from the UN Agency for International Development, is aimed at reducing infant mortality in Madagascar, an island nation in the Indian Ocean off the eastern coast of Africa. The UN Agency for International Development is an organisation based in Madagascar. 


1604 A brother of Colombia's education minister has been killed by Marxist rebels who kidnapped him three years ago. Colombia's education minister has been kidnapped by Marxist rebels.


1927 The organizers of the 15th International AIDS Conference, scheduled for next month in Bangkok, Thailand, on Saturday responded to press reports that a prominent hotel in Bangkok discriminated against HIV-positive visitors attending a different conference this month. HIV-positive visitors take part in the 15th International AIDS Conference.   


2001 Christoffer's family was identified only as an Air Force family living in Southern California. Air Force is an organisation based in Southern California.  


1939 The plan was released by Mr Dean on behalf of the Secretary of Health and Human Services, Tommy Thompson, still recovering from a recent accident, at a Secretarial Summit on Health Information Technology that was attended by many of the nation's leaders in electronic health records. Mr Dean is the Secretary of Health and Human Services. 


944 The U.S. handed power on June 30 to Iraq's interim government chosen by the United Nations and Paul Bremer, former governor of Iraq. The U.S. chose Paul Bremer as new governor of Iraq.  


1935 Evans Paul, the pro-Aristide mayor of Port-au-Prince, said he had been told that the money will go to American lobbyists, politicians and journalists to campaign against the return of Aristide, Haiti's first democratically elected president. Aristide is the mayor of Port-au-Prince.


112 NIH's FY05 budget request of $28.8 billion includes $2 billion for the National Institute of General Medical Sciences, a 3.4-percent increase, and $1.1 billion for the National Center for Research Resources, and a 7.2-percent decrease from FY04 levels. A request was reduced to $1.1 billion.    


137 Acid attacks on women are carried out across Asia, with reports of such incidents from Burma, Cambodia, India and Pakistan. Acid attacks are carried out on women.    













Issues with when a part refers to a whole 2083 The Irish Sales, Services and Marketing Operation of Microsoft was established in 1991. Microsoft was established in 1991.  


2085 The first Windows DNA lab outside Microsoft was established in suburban Philadelphia in June 1998. Microsoft was established in June 1998. 


2002 In 1973, the United States and Communist China agreed to establish liaison offices in Beijing and Washington. Communist China is an organisation based in Beijing and Washington.  


36 Scripps Memorial Hospital Encinitas emergency room doctors and nurses treat two to three injured surfers. Scripps Hospital assists surfing accident victims.  













Verb Issues (subject/object etc.) 1914 Like the United States, U.N. officials are also dismayed that Aristide killed a conference called by Prime Minister Robert Malval in Port-au-Prince in hopes of bringing all the feuding parties together. Aristide kills Prime Minister Robert Malval. 


1939 The plan was released by Mr Dean on behalf of the Secretary of Health and Human Services, Tommy Thompson, still recovering from a recent accident, at a Secretarial Summit on Health Information Technology that was attended by many of the nation's leaders in electronic health records. Mr Dean is the Secretary of Health and Human Services. 

























Negatives not signalled by “not” (e.g., cancelled, doubted, etc.) 1933 Like the United States, U.N. officials are also dismayed that Aristide killed a conference called by Prime Minister Robert Malval in Port-au-Prince in hopes of bringing all the feuding parties together. U.N. officials take part in a conference called by Prime Minister Robert Malval.  


286 Security preparations kept protestors at bay at the recent G8 Summit on Sea Island (USA). Demonstrations against G8 Summit took place at Sea Island bay. 


72 On the eve of a widely anticipated retaliation against Russian forces in Grozny, military commanders withdrew key units from the city, leaving it vulnerable to guerrilla infiltration. Russian forces vulnerable to guerrilla fighters.  


2140 Turkey is unlikely to become involved in, or allow U.S. forces to use Turkish territory in a Middle East war that does not threaten her territory directly. U.S. to use Turkish military bases.  

























Idiom Issues 1933 Like the United States, U.N. officials are also dismayed that Aristide killed a conference called by Prime Minister Robert Malval in Port-au-Prince in hopes of bringing all the feuding parties together. U.N. officials take part in a conference called by Prime Minister Robert Malval.  


1946 Oil prices eased back from record high levels after Russia's justice ministry said embattled oil giant Yukos can continue oil production and sales. Oil prices rise.


1950 Crude oil dips below $43 on news that Russia's justice ministry will not force Yukos to halt sales. Crude oil rises.


1949 Oil prices fall back as Yukos oil threat lifted. Oil prices rise.  


1178




286 Security preparations kept protestors at bay at the recent G8 Summit on Sea Island (USA). Demonstrations against G8 Summit took place at Sea Island bay. 













Egregiously Ungrammatical Sentences 2011 Los Angeles Mayor James Hahn, along with top members of the Automobile Club of Southern California, along with will announce the kickoff of a $9 billion plan to fund Los Angeles County's Top Ten Traffic Busters. James Hahn is a member of the Automobile Club of Southern California.   


2095 Scientists had thought all gamma-ray bursts had a standard energy. The scientists had thought that all explosions gamma the ray have a standard energy.  


1244 The source added that the investigation proved that the bases of the genocide crime "were completed with a series of illegal arrests followed in some cases with assassinations or cases of disappearances and were preceded, according to information attached to the file, by cases of torture.". Investigators discovered that a series of illicit arrests of were often followed by disappearances or murders and were preceded by torture.



















Negation Issues 2141 No Weapons of Mass Destruction Found in Iraq Yet. Weapons of Mass Destruction Found in Iraq. 


2142 No stockpiles of weapons of mass destruction have been found in Iraq since Saddam's regime was toppled in a US-led invasion last year. Weapons of mass destruction found in Iraq. 


1160 Baghdad had announced that it will stop cooperating with UNSCOM completely but indicated that it will not ask for their departure. Baghdad will end it's cooperation with UNSCOM, and wants them to leave.







Quantity Reasoning 256 Eilat, Israel's most southerly port and the country's only access to the Red Sea, has long been a winter destination for tourists, 80% of whom are Europeans attracted by its temperate climates and clear waters. In winter most tourists to Eilat come from Europe. 


820 Kessler's team conducted 60,643 face-to-face interviews with adults in 14 countries. Kessler's team interviewed more than 60,000 adults in 14 countries. 













Credibility of Source 24 Dr. Amy Joy Lanou, director of the Physicians Committee for Responsible Medicine in America, was quick to point out health dangers of the Atkins diet. Atkins diet is associated with increased risk conditions. 


1178 Sources in Tegucigalpa reported that many Latin American countries are gearing up to face Hurricane "Mitch". South American countries are preparing for "Mitch". 







Acronym Issues 1153 Sultan Al-Shawi, a.k.a the Attorney, said during a funeral held for the victims, "They were all children of Iraq killed during the savage bombing.". The Attorney, said at the funeral, "They were all Iraqis killed during the brutal shelling.".  


1190 The Commission, designated to shed light on crimes and human rights violations during the apartheid period, accuses the African National Congress of carrying out torture and executions without trials in concentration camps, and also of playing an active role in the political violence incidents that took place between 1990 and 1994. The Commission accused the ANC of violence, executions and human rights violations.