1、May 21, 2003,Building a Terminological Database from Heterogeneous Definitional Sources,Smaranda Muresan, Peter T. Davis Samuel D. Popper, Judith L. KlavansColumbia University,Why Terminology is Important?,Each agency and each department might have different ways to define the same conceptWorking wi
2、th multiple databases requires understanding the data across multiple agencies and domains,Whats an Employee?,An appointed officer or employee of USDA including special Government employees (collaborators, consultants and panel members). The term excludes independent contractors.,An individual who i
3、s engaged or compensated by a railroad or by a contractor to a railroad, who is authorized by a railroad to use its wireless communications in connection with railroad operations.,A person who works for wages or salary in the service of an employer.,The term “employee“ does not include a director, t
4、rustee, or officer.,US Department of Agriculture,Federal Railroad Administration,Mine Safety and Health Administration,US SEC,Desiderata for Terminological Resources,Capture the ongoing evolution of languageProvide consistency, ease of sharing and integration across agencies.,Architecture,Collection
5、,SemanticAnalysis,Use,Heterogeneous Definitional Corpus,GetGloss,Definder,ParseGloss,Database Building,Terminological Database,dynamic sources,relations among concepts and their attributes,fast access, flexibility, sharing,database query,Building the terminological DB,Collection,MotivationDefinition
6、s are rich in terminological knowledgeOn-line dictionaries are static and generally incompleteNeed to capture the evolution of language,Acquisition of Heterogeneous Definitional Corpus,GetGloss,Definder,Solution GetGloss identification and extraction of glossariesDefinder - extraction of definitions
7、 from online free text,Building the Terminological DB,Motivation Need to identify relationships among concepts e.g. synonyms, hypernyms, cross-referenceNeed to store this conceptual information for easy and fast access and integration,Semantic Analysis,ParseGloss,Database Building,Terminological Dat
8、abase,Definitional Corpus,Gasoline: See Motor Gasoline (Finished). Motor Gasoline (Finished): A complex mixture of relatively volatile hydrocarbons with or without small quantities of additives, blended to form a fuel suitable for use in spark-ignition engines. Motor gasoline, as defined in ASTM Spe
9、cification D 4814 or Federal Specification VV-G-1690C, is characterized as having a boiling range of 122 to 158 degrees Fahrenheit at the 10 percent recovery point to 365 to 374 degrees Fahrenheit at the 90 percent recovery point. “Motor Gasoline“ includes conventional gasoline; all types of oxygena
10、ted gasoline, including gasohol; and reformulated gasoline, but excludes aviation gasoline.,Term: Motor Gasoline (Finished)Source: (source (agency “EIA“) (resource “Gasoline Glossary“) (url ) Paren-modifier: FinishedFull Definition: A complex mixture. for use in . Core Definition: A complex mixture
11、. for use in spark-ignition enginesGenus Phrase: A complex mixture of relatively volatile hydrocarbonsHead Genus Word: hydrocarbonsProperties:UsedIn:spark-ignition enginesExcludes-Includes:includes conventional gasolineincludes gasoholexcludes aviation gasoline .,Solution Transform definitional text
12、 into conceptual data ParseGloss partial semantic analysis of definitions to identify relations between concepts Store data into a relational database,Database Use,MotivationEnable the user to access the richness of terminological knowledge Assure easy and fast access to data Enable data sharing and
13、 integration across agencies Enable dynamic update of data,Terminological Database,SQL query for inflammation1. Redness, swelling, heat and pain resulting from injury to tissue (parts of the body underneath the skin). Also known as swelling. 2. A characteristic reaction of tissues to disease or inju
14、ry; it is marked by four signs: swelling, redness, heat, and pain. 3. The reaction of tissue to injury . 4. A response to irritation , infection , or injury , resulting in pain , redness , and swelling . ,Solution Query module for the relational database (SQL),Putting It All Together,Collection,Sema
15、nticAnalysis,User,Heterogeneous Definitional Corpus,GetGloss,Definder,ParseGloss,Database Building,Terminological Database,dynamic sources,relations among concepts and their attributes,fast access, flexibility, sharing,database query,Building the terminological DB,GetGloss Automatic Glossary Extract
16、ion,DGRC project Given a URL find the glossary file Challenges:glossaries can constitute small parts of a web page, being embedded inside there is no standard HTML tag formatting for marking pairs a web page can contain pairs, where information is not a definition.,True Positive,False Positive,Algor
17、ithm,Two step algorithm Identification Component Find candidate glossaries Keyword + Rule-based algorithm (6 rules) “glossary”, “dictionary” in HTML tags Terms in alphabetical order Classification Component Filter out false positives Rule-based method (9 rules)e.g filter if term is a Named Entity (e
18、.g California) Statistical method using SVM,Evaluating the Identification Component,10,000+ pages from 5 different sites 1,000 page sample: no glossaries (n=13) 286,579 page sample from 268 domains P=53% Estimating recall is hard Precision and Recall both very sensitive to perturbations (p=0 vs. p=5
19、3%),Klavans et. al (dg.o 2002),Evaluating the Classification Components,GetGloss Categorizer assigns a score to each candidate based on a linear combination of weighted featuresCorpus: 2400 glossary candidatesTest: 300 randomly chosen, manually categorized glossary candidates,Classification Componen
20、t Performance,0 if Score -100 1 if -100 Score -50 2 if -50 Score 0,3 if 0 Score 50 4 if 50 Score 100 5 if 100 Score,Definder- Automatic Extraction of Definitions from Text,Definder- Automatic Extraction of Definitions from Text,Definder,Part of NSF funded digital library project Medical domain Extra
21、ct definitions from consumer oriented medical text Corpus Medical articles written by doctors for lay audience Different genre (articles, manual chapters),Algorithm,Shallow parsing Simple definitions (e.g NP-NP pairs for synonyms) Candidate complex definitionsFull parsing (Charniak00 parser) apposit
22、ions, relative clauses, complex definitional sentences,Definition Patterns and Examples,Simple definitions NP ( NP ) moving x-ray pictures ( angiograms ) hypertension ( high blood pressure ) NP - NP - tachycardia racing heartbeat - . NP of NP ( NP ) enlargement of the heart muscle ( hypertrophy ),De
23、finition Patterns and Examples,Complex definitions S CNP NP , CNP CNP CNP CNP VP VP S Angina, the pressing chest pain most people associate with heart problems, . S CNP NP - CNP CNP CNP CNP S atherosclerosis the progressive narrowing of the hearts own arteries by cholesterol plaque buildups, which s
24、tarves the heart itself for oxygen and nutrients.,Evaluation,Quantitative 4 human subjects , 10 articles 53 definitions gold standard DEFINDER p=86.27%, r = 84.60% Qualitative Usefulness and readability (non-specialists) Completeness and accuracy (medical specialists),Klavans and Muresan(JCDL01) Mur
25、esan and Klavans (LREC02),Coverage of On-line Dictionaries,Characteristics of Automatically Built Definitional Corpus,Large amount of data Heterogeneous Structure Language Semantics Multiple definitions of the same term,Ozone: (O3)A colorless gas with a pungent odor, having the molecular form of O3
26、, found in two layers of the atmosphere, the stratosphere (about 90% of the total atmospheric loading) and the troposphere (about 10%). Ozone is a form of oxygen found naturally in the stratosphere that provides a protective layer shielding the Earth from ultraviolet radiations harmful health effect
27、s on humans and the environment. In the troposphere, ozone is a chemical oxidant and major component of photochemical smog. Ozone can seriously affect the human respiratory system. See atmosphere, ultraviolet radiation. Ozone: A form of oxygen in which atoms combine in groups of three . ,Environment
28、 domain Column Ozone: ozone between the Earths surface and outer space. Ozone levels can be described in several ways. One of the most common measures is how much ozone is in a vertical column of air. The dobson unit is a measure of column ozone. Other measures include partial pressure, number densi
29、ty, and concentration of ozone, and can represent either column ozone or the amount of ozone at a particular altitude. Medical domain Arrhythmia: A disturbance in the beating pattern of the heart .,Addison Disease - a rare disease that results from a deficiency in adrenocortical hormones- an endocri
30、ne disorder that affects about 1 in 100,000 people- a degenerative disease that is characterized by low blood pressure and dark brown pigmentation of the skin,Ozone: (O3)A colorless gas with a pungent odor, having the molecular form of O3 , found in two layers of the atmosphere, the stratosphere (ab
31、out 90% of the total atmospheric loading) and the troposphere (about 10%). Ozone is a form of oxygen found naturally in the stratosphere that provides a protective layer shielding the Earth from ultraviolet radiations harmful health effects on humans and the environment. In the troposphere, ozone is
32、 a chemical oxidant and major component of photochemical smog. Ozone can seriously affect the human respiratory system. See atmosphere, ultraviolet radiation. Ozone: A form of oxygen in which atoms combine in groups of three . ,Putting It All Together,Collection,SemanticAnalysis,User,Heterogeneous D
33、efinitional Corpus,GetGloss,Definder,ParseGloss,Database Building,Terminological Database,dynamic sources,relations among concepts and their attributes,fast access, flexibility, sharing,database query,Building the terminological DB,ParseGloss Partial Semantic Analysis,Challenges heterogeneous collec
34、tion of definitions Focus on identifying the main semantic relations among concepts Partial Semantic Analysis based on shallow parsing Genus phrase and genus term Synonyms, cross-reference Other common relations between the defined term and the concepts inside the definition,Example,Term: Motor Gaso
35、line (Finished)Source: (source (agency “EIA“) (resource “Gasoline Glossary“) (url ) Paren-modifier: FinishedFull Definition: A complex mixture. for use in . Core Definition: A complex mixture . for use in spark-ignition enginesGenus Phrase: A complex mixture of relatively volatile hydrocarbonsHead G
36、enus Word: hydrocarbonsProperties:UsedIn:spark-ignition enginesExcludes-Includes:includes conventional gasolineincludes gasoholexcludes aviation gasoline ,Evaluation,Task - User based evaluation to build a gold standard for genus phrase to ask people to identify the most important properties inside
37、each definition Data 100 term-definition pairs 7 different glossaries 26 subjects,Results,Complex evaluationDifferent notions of agreement and overlap: Head only 64% precision Genus phrase 59% precision,Building the Database,Relational databaseXML data transferStatistics 8000 terms 12,500 definition
38、s 2000 different sources (web pages, articles, etc.),Distribution of terms with multiple definitions,Putting It All Together,Collection,SemanticAnalysis,User,Heterogeneous Definitional Corpus,GetGloss,Definder,ParseGloss,Database Building,Terminological Database,dynamic sources,relations among conce
39、pts and their attributes,fast access, flexibility, sharing,database query,Building the terminological DB,Query the terminological Database,- Term: AuditDefinition: An examination of the financial statements, accounting records, and other supporting evidence of an institutionSource: www.fdic.gov- Ter
40、m: Industrial radiographyDefinition: means an examination of the structure of materialsSource: www.nrc.gov- Term: Medical SurveillanceDefinition: is the systematic examination of medical monitoring data to determineSource: www.osha.gov,Conclusions,Proposed a framework for solving the heterogeneous t
41、erminology problem Automatically building a heterogeneous collection of definitions from dynamic sources Partial semantic analysis of the definitions to identify main semantic relations between concepts Building a database for fast, easy access, dynamic update of data, sharing across agencies,Future
42、 Work,Deep domain specific semantic analysis But, how to automatically classify the glossary entries and definitions? Based on the classification of their source (sites, articles, etc) When and how to merge different definitions for the same term? Integrate this acquired terminological knowledge into the DGRC system,
