1、Accessing Tacit Knowledge and Linking it to the Peer-Reviewed Literature,Michael Shepherd Web Information Filtering Lab Faculty of Computer Science Dalhousie University,Research Team,Students Qiufen Qiu (MD and MCS) Zhixin Chen (MHI and BSc) Computer Science Faculty Michael Shepherd Qigang Gao Syed
2、Sibte Raza Abidi Anaesthesia & Psychology G. Allen Finley,Overview,Introduction Research Program Results to Date Summary,Pediatric Pain Discussion List,Clinical discussion on pediatric pain Informal email-based discussion among professionalsInitiated in 1993 Over 700 subscribers world-wide More than
3、 10,000 messages,Date: Wed, 04 Jan 1995 16:54:48 -0500 (EST) From: poster Subject: opioids and meningitisX is a 13 month (9.8kg) old boy suffering from acute meningitis (pneumocoque) treated with IV cefotaxime; at day three, I have been called as pediatric pain consultant to assess X; I have discove
4、red an extreme painfull state: one could not handle or touch him without producing screaming. The child was unable to move spontaneously he looked paralysed by pain and hypertonia ; he also presented a neurological complication : ptosis at the right side.The pain treatment was IV acetaminophen. The
5、first day I have prescribed IV Nalbuphine (weak opioid u antagonist and agonist) 11mg/24h after a loading dose of 1.4 mg; Pain at rest has been succesfully relieved but not the mobilisation pain; the dose has been increased at 14 mg/day wihout relieving the pain associated with moving; he has moved
6、spontaneously limbs 2 days later; nalbuphine has been stopped 4 days later. Neurological examination and CT scan have been still normal (except ptosis) during this period. No opioids side effects have been observed.What do you think of this case ?Have you any experience with opioids and acute mening
7、itis ?Dr Poster, Pediatric pain unit, Poster Hospital,Date: Wed, 04 Jan 1995 17:27:25 -0500 (EST) From: first reply Subject: re: opioids and meningitisIs there any periosteal involvement? If so an NSAID (ibuprofen or naproxen) may be much more effective than even opioid.-,Date: Wed, 04 Jan 1995 19:0
8、6:32 -0400 From: second reply Subject: Re: opioids and meningitisPoster writes: X is a 13 month (9.8kg) old boy suffering from acute meningitis. extreme painfull state: one could not handle or touch him without producing screaming The first day I have prescribed IV Nalbuphine . succesfully relieved
9、but not the mobilisation pain;. has moved spontaneously limbs 2 days later; nalbuphine has been stopped 4 days later. Neurological examination and CT scan have been still normal.I have used IV morphine for similar severe meningitis pain, with success. I wouldnt hesitate to use a pure opioid agonist
10、(in conjunction with acetaminophen, NSAID, and/or tricyclics). However, it sounds like you have the situation under control.Second Reply, Associate Professor, Dept and University-,Date: Thu, 05 Jan 1995 18:58:32 -0800 (PST) From: Third Reply Subject: Re: opioids and meningitisI wonder if the problem
11、 is not due to severe arachnoiditis that is secondary to the inflammation. I would suggest a trial of steroids in this patient, perhaps in combination with a benzodiazepine to reduce the spasm. Narcotics may reduce the pain but I would not like to keep X on them for too long. Good luck Third Reply-,
12、Tacit and Explicit Knowledge,Tacit knowledge is what the knower knows and is derived from experience Explicit knowledge is represented by some artifact such as a document or journal article,Knowledge Transformation Processes,Knowledge Transformation Processes,Research Questions,ExternalizationHow ca
13、n we capture the tacit knowledge in such a discussion list and transform it into explicit knowledge?CombinationHow can we organize this explicit knowledge?InternalizationHow do we provide access to this explicit knowledge so that users can internalize this knowledge?Linking Tacit Knowledge to Best E
14、videnceHow do we map this transformed tacit knowledge to the appropriate best evidence literature?,Mapping Tacit Knowledge to Explicit Knowledge in Medical Literature,Mesh Terminology Map,Externalization,Combination,Linking,Internalization,Data Cleaning,Remove duplicate messages (subject & time stam
15、p) Remove responses that were generated automatically by “vacation” mail programsRemove other “junk” e-mailsRemoving unnecessary content of the messages themselves. This unnecessary content included non-textual material such as images that would not be used in the clustering process and included ori
16、ginal messages that were more than ten lines long as these would skew the clustering process.The initial stage of this cleaning was done manually until patterns were recognized and then programs were written to clean the data based on these patterns.,Externalization: Creating Threads,Messages were t
17、hreaded based on time stamps and subject headings. Those messages that had a blank subject field were processed based on the included original messages to which they had replied.,Thread Representation,Each thread is treated as though it were a contiguous document The original messages that are embed
18、ded in the reply messages are removed. Stop words are removed If not on the stop list, they are matched against a synonym dictionary manually created by a pediatric pain specialist. The remaining terms are stemmed The stemmed terms are assigned tf.idf weights,Data Set,An archived sample of 6939 mess
19、ages from 1993-1999After cleaning 4033 messagesAfter threading 1289 threadsEach thread is represented by a vector of 4111 term weights,term1 term2 term3 . . . term4111 thread1 w1,1 w1,2 . . . w1,4111 thread2 w2,1 w2,2 . . . w2,4111.thread1289 w1289,1 w1289,2 . . . w1289,4111,Thread-Term Matrix,Combi
20、nation: Organizing the Threads,Text clustering unsupervised learning process groups documents into clusters so that the documents within a cluster have high similarity with one another, but are very dissimilar to the documents in the other clusters Text classification or categorization supervised le
21、arning process Assigns documents to pre-defined classes or categories,k-means clustering with k=2,1,2,3,4,5,6,7,k-means clustering with k=2,1,2,3,4,5,6,7,k-means clustering with k=2,1,2,3,4,5,6,7,Evaluation of Clustering,Performed a study in which 100 randomly selected threads were presented to two
22、experts for clustering and to our clustering algorithmResults of clustering between the experts measured Results of clustering between the experts and the system measured,Clusters and labels created by expert 1 a psychologist,Clusters and labels created by expert 2 a medical doctor,Inter-Rater Relia
23、bility,The Redundancy(X, Y) is the proportion of uncertainty about X that is removed by knowing YIn this instance, X and Y represent the two sets of clusters generated by the experts. The measure is asymmetrical and the calculated redundancy measures are:R(Expert-1, Expert-2) = 0.51 R(Expert-2, Expe
24、rt-1) = 0.44,Evaluation of the Automatically Generated Clustering,Assume each manually created cluster is correctCompare the manually created cluster against an automatically created clusterRecall the proportion of those items in the manually created cluster that appear together in the same automati
25、cally generated clusterPrecision the proportion of those items in an automatically created cluster that appear together in the same manually created clusterFmeasure = 2PR / (P+R),Hierarchy k=2,C-1,1,C-2,1,C-3,1,C-3,2,C-3,3,C-3,4,C-4,1,C-4,2,C-2,2,F-measure for a classification,The overall F-measure
26、is used to reflect the quality of the whole hierarchy. The overall F-measure is the average weighted F-measure for all the clusters in a humanly generated clustering and is defined to be:Overall F-measure = ( |T| * F(T) / |T|,Evaluation of Clustering,Each experts set of clusters was compared to the
27、automatically generated hierarchical clustering. The hierarchy was generated ten times using different seed centroids for each run.The results of the paired-samples t tests (p=0.05) show that there was no significant difference between the two sets of manually generated clusters when used to evaluat
28、e the automatically generated clustering (k = 6).,Evaluation of k-means Clustering,We now have 3 different clusterings with inter-rater reliability of .50k-means generated a large number of term representatives for each cluster with no elegant way of mapping the terms into MeSH. Therefore, the k-mea
29、ns clustering algorithm was replaced with a SOM in the expectation that the clustering results would be better and that a smaller set of term representatives for each cluster might be identified.,SOM Self Organizing Maps,Invented by Teuvo Kohonen Provide a way of representing multidimensional data i
30、n much lower dimensional spaces - usually one or two dimensions. Create a network that stores information in such a way that any topological relationships within the training set are maintained,Example 2-D Lattice of Nodes,Mapping 3 Dimensional Colour Vectors Into 2 Dimensions,Notice that in additio
31、n to clustering the colours into distinct regions, regions of similar properties are usually found adjacent to each other.,SOM Neighbourhood Decreases,Mapping 3 Dimensional Colour Vectors Into 2 Dimensions,Notice that in addition to clustering the colours into distinct regions, regions of similar pr
32、operties are usually found adjacent to each other.,term1 term2 term3 . . . term4111 thread1 w1,1 w1,2 . . . w1,4111 thread2 w2,1 w2,2 . . . w2,4111.thread1289 w1289,1 w1289,2 . . . w1289,4111,Thread-Term Matrix,Principal Component Analysis for Feature Length Reduction,PCA Vectors,Eigen Values,SOM Ve
33、ctor Length 150,Growing Hierarchical SOM,SOM Results,Problems,Mesh Terminology Map,Externalization,Combination,Linking,Internalization,Mapping Tacit Knowledge to Explicit Knowledge in Medical Literature,Mesh and UMLS,Externalization,Combination,Linking,Internalization,Combination: Organizing the Thr
34、eads,Text clustering unsupervised learning process groups documents into clusters so that the documents within a cluster have high similarity with one another, but are very dissimilar to the documents in the other clusters Text classification or categorization supervised learning process Assigns doc
35、uments to pre-defined classes or categories,MetaMap Transfer (MMTx),Discovers UMLS Metathesaurus concepts in textText is parsed into components including sentences, paragraphs, phrases, lexical elements and tokens. Produces a shallow syntacitc analysis with part-of-speech tagging.Variants are genera
36、ted from the resulting phrases. Includes acronyms, abbreviations and synonyms.Candidate concepts from the UMLS Metathesaurus are retrieved and evaluated against the phrases. The best of the candidates are organized into a final mapping in such a way as to best cover the text.,Metathesaurus Candidate
37、s,The word “discharge“ returns Semantic Group: Anatomy Discharge, Body Substance (C0012621) - Body Substance Discharge, Body Substance, Sample (C0600083) - Body Substance Semantic Group: Procedures Patient Discharge (C0030685) - Health Care Activity from the UMLS Knowledge Server,Metathesaurus Candi
38、dates,“He is to be discharged home.“ Phrase: “discharged“ Meta Candidates (3) 966 C0030685:Discharge (Patient Discharge) Health Care Activity 966 C0600083:Discharge (Discharge, Body Substance, Sample) Body Substance 966 C0012621:Discharge, NOS (Discharge, Body Substance) Body Substance Phrase: “home
39、“ Meta Candidates (3) 1000 C0442517:Home Manufactured Object 928 C0237154:homeless (Homelessness) Finding 928 C0019863:homeless (Homeless persons) Population Group,Using the MMTx Results,The MMTx results were used in three different ways: Organize the PPML threads according to the UMLS Semantic Grou
40、ps -134 semantic types in 15 semantic groups Organize the PPML threads according to the MeSH Hierarchy 15 MeSH trees Select terms that can be used as queries to PubMed,Organization by UMLS Semantic Group,Organization by MeSH Tree,There are 15 MeSH treesIt was determined to keep only two trees: The C
41、 tree (Diseases) as the PPML largely deals with disorders and diseases The D tree (Chemicals and Drugs) as the PPML contains discussions on drugs hence it was deemed important to retain drug-related terminology.,Filtering to Generate PubMed Queries,Filtering approach operates at the semantic/concept
42、ual level as opposed to the term levelUMLS semantic types associated with each MeSH term are used as the basis for term filteringWorking at the semantic level we can Establish a medical context for the thread which can assist in subsequent search for corresponding literature; Characterize the entire
43、ty of medical terms into a small number of medical concepts Design filtering rules that apply to broad semantic types as opposed to focused individual terms,Filtering UMLS Concepts Associated with MeSH Terms Found in Subject Line,If mapping score = 1000 then retain the MeSH term.If semantic type = A
44、ge group (T100) then retain the MeSH term.If semantic group = CHEM | DISO | ANAT AND (mapping score 800) then retain the MeSH term.If semantic type = Diagnostic Function (T060) | Therapeutic or Preventive Procedure (T060) | Laboratory or Test Result (T034) AND (mapping score 800) then retain the MeS
45、H term.,Generating a PubMed Query,Query Terms: Feline osteogenesis imperfecta, Adolescent, Osteoporosis,x,x,Summary,We have various hierarchical organizations of the PPML threads that can be browsed by the userWe have linked the PPML to the best-evidence literature via PubMed,Knowledge Transformatio
46、n Processes,Future Research,Improve the filtersLink from medical literature to PPMLEvaluate the overall system with respect to the users: Is it useful? Is it helpful? Does it improve outcomes?,Thank You,Web Information Filtering Labhttp:/www.cs.dal.ca/wifl/,Closeness of Document Vectors,information science Doc0 0 0 Doc1 0 1 Doc2 1 0 Doc3 1 1,cos 0o = 1 cos 90o = 0,Doc2,Doc1,
