1、Beyond Basic Search,Gary Marchionini University of North Carolina at Chapel Hill marchils.unc.eduCS Colloquium Computer Science Department University of Illinois at Urbana-Champaign September 10, 2007,Outline,Information retrieval R&D has stimulated a multi-billion dollar industry The challenges of
2、information seeking and exploratory search that get searcher(s) more actively involved Some early examples: faceted search, dynamic queries and agile views Evaluation challenges,Content-Centered Retrieval as Matching Document Representations to Query Representations,A powerful paradigm that has driv
3、en IR R&D for half a century. Evaluation metric is effectiveness of the match. (e.g., recall and precision). A half duplex (at best) process that is strongly dependent on pre-processing,Document Space,Sample,Sample,Query Space,Surrogates,Surrogates,Terms,Vectors,Etc,Query Form B,Etc,Match Algorithm,
4、Query Form A,Lookup,Learn,Investigate,Fact retrieval Known item search Navigation Verification Question answering,Knowledge acquisition Comprehend/Interpret Compare Aggregate/Integrate Socialize,Accrete Analysis Exclude Synthesis Evaluation Discovery Plan/Forecast Transform,Explore,Information Seeki
5、ng Goals: Focus on Exploratory Search,Characteristics of the Exploratory Search Process,Multiple sessions Multiple queries Recall important Collaborative Substantial time spent in results Coordinate with other tools Relevance judging more difficult (subjective, domain dependent, nuanced) Objects of
6、interest tend toward abstract and complex,What are we trying to support and evaluate? Active humans with information needs, information skills, powerful IR resources (that include other humans), and situated in global and local connected communities, all of which evolve over time,Dynamic Queries,Dir
7、ect manipulation (e.g., slider move, hover) defines and executes query with immediate feedback (see http:/www.cs.umd.edu/hcil/spotfire/ for history) An alternative to text query forms,Faceted Search,Combine text search with category selection Many E-commerce sites Metadata critical (database backend
8、s loved) Examples: Flamenco http:/flamenco.berkeley.edu/ mSpace http:/www.mspace.fm/ Endeca http:/ also see http:/www.lib.ncsu.edu/endeca/ Relation Browser http:/idl.ils.unc.edu/rave/examples.html,AgileViews,A view is a partition of an information space There are many possible partitions for any spa
9、ce since many attributes may be used to slice and dice the space People should be able to effortlessly change views Focus can change based on granularity Focus can change based on attribute,AgileViews Framework,Relation Browser Example with all EIA pages RB demo here,RB Goals,Facilitate exploration
10、of the relationships between (among) different data facets Display alternative partitions of the database with mouse actions Support string search within partitions Serve as an alternative to existing search and navigation tools,Relation Browser Principles,Architectural Principle: Juxtapose facets T
11、wo or more with 5-15 categories per facet Topic is one important facet for most applications Interaction Principle: Dynamic exploration of relationships among facets and categories Database driven to promote flexible applications (requires systematic metadata),Key Challenges,Technical evolutions (Ja
12、va, metadata to client side) User expectations and preparations Getting metadata and mapping to RB scheme Given the cost and difficulty with hundreds of thousands of web pages, can we automate this process?,Selected User Studies,ACM/IEEE JCDL 2004 RB and Web UI for movie catalog in UNC Library 15 pa
13、rticipants, 3 kinds of task, 10 trials each UI (counterbalanced), time, accuracy, satisfaction measures Results: no SRD for simple lookups, RB SRD better for data exploration and analysis; and satisfaction JCDL 2007 Relation Browser, Vanilla Facet, BLS website 40 participants, Between and within sub
14、ject designs with known item and exploratory tasks Results: NSR differences familiarity influences expectationsinstalled base syndrome Automatically generated categorization comparable to carefully crafted website layouts,Why Mixed Results?,Human factors (domain knowledge, motivation, system experie
15、nce, physical and mental capabilities, etc.) Task factors (precise mappings of types onto content, e.g., number of facets, fact versus concept, etc.) Appropriate and/or subjective metrics (e.g., accuracy, errors, satisfaction, and time) Facet and category generation (automatic metadata): machine lea
16、rning, knowledge-based, manual, and combinations,Open Video Example www.open-video.org,Open access digital library of digital video for education and research 5000 video segments: MPEG1, MPEG-2, MPEG-4, QuickTime Multiple visual surrogates Agile Views Design Framework Facet partitions (collections,
17、genre, length, etc.) Multiple examples of views: Surrogates as previews (textual metadata, storyboard, except, fast forward, spoken descriptions/keywords) Dynamic control mechanisms Basic search (MySQL indexes),Alternative Previews for a Specific Video Segment OV demo here,Video Surrogate Studies,A
18、dozen studies over 6 years (CHI 07, JCDL 04, ASIST, AVI, MM 06, others) Story boards, slide shows, fast forwards, excerpts, spoken keywords, spoken descriptions, combinations Multiple tasks (gist, vist) Multiple measures (accuracy, time, satisfaction, gaze) Quant and qual: within, between, ethnograp
19、hic Results Words matter Visual adds value (conceptual and affective) People able to infer from few cues, tolerate high rates Coordination of multiple channels?,User Study Framework For Surrogation: The Evaluation Challenges,Evaluation Directions,Massive click through data with laboratory follow up
20、(e.g., White et al SIGIR 2007) Longitudinal user behavior (e.g., Kelly dissertation, 2003), new paradigms for behavioral streams via sensors or opt-in services Laboratory studies of highly specified features New metrics: use, understanding, adoption Social behaviors (e.g., marketplace metrics such a
21、s ROI and adoptionwhy do groups adopt?,Status and Preview,SIGIR, SIGCHI workshops (tools, theories, evaluation) CACM, IP&M special issues Search engine company R&D NSF HCC program Mashups and ideas/systems that integrate search into life activity (consider how successful and transparent autocomplete form fill has become, whereas explicit relevance feedback has not). e.g., UCAIR here at UIUC Grand challenges of Cyberinfrastructure and Personal information services,Thank You! Questions and Discussion marchils.unc.edu http:/ils.unc.edu/march,
