Supervised by Prof. LYU, Rung Tsong Michael.ppt

1、Supervised by Prof. LYU, Rung Tsong Michael,Department of Computer Science & Engineering The Chinese University of Hong Kong,Prepared by: Chan Pik Wah, PatNgai Cheuk Han, Table,LYU0102 XML for Interoperable Digital Video Library,Outline,Introduction to XVIP Review of Project Extraction Techniques Fa

2、ce Detection Speech Recognition Presentation XSL SMIL Transformation,Motivations,Rapid increase in the usage of multimedia information New approach: DIGITAL VIDEO LIBRARY,Project Outline,Motivations,Little attention paying on video information extraction and storage Scalability of the system in term

3、s of adding new extraction components Lack of a generic framework for presentation and visualization of video information,Project Outline,Overview of XVIP,Project Outline,Achievements in last semester,Project Outline,Achievements in this semester,Project Outline,Extraction Techniques,Extraction Tech

4、niques,Face Detection,Object-presence detections are also an important technique. Identify and index features to support image similarity matching. Face detection is a good example,Extraction Techniques,Face Detection,Name of people appearing in the video How they are interacting with the environmen

5、t More searchable,Extraction Techniques,Face Detection,Neural Network-Based Algorithm The basic algorithm used for face detection,Extraction Techniques,Face Detection,Face Recognition Facial Expression Analysis Enrich the XML Easier for user to search the content of video,Extraction Techniques,Speec

6、h Recognition,Speech recognition technology can make any spoken data useful for library indexing and retrieval,Extraction Techniques,Speech Recognition Engine,Extraction Techniques,Speech Recognition,ViaVoice Error rate 50%,Extraction Techniques,Presentation of the video,XML to SMIL Transformation S

7、MIL XSL Transformation process,SMIL,SMIL,SMIL stands for Synchronized Multimedia Integration Language is currently a W3C Recommendation. It is a markup language that can synchronize and integrate multimedia. It enables authors to specify when and what should be presented.,SMIL,Advantages,SMIL is tex

8、t-based Easy to develop with a text editor Generate customized presentations Due to text-based, can generate customized SMIL file for each visitor based on preferences recorded in the visitors browser SMIL effort is led by the W3C W3C tries to shape a specification that is beneficial to all parties

9、involved. Avoid using container formats. SMIL can stream many media formats, need not to merge clips into a single streaming file.,SMIL,Document Structure,Quite similar to the structure of HTML document. SMIL has the , , and elements. SMIL is case-sensitive, all the tags should be written in lower c

10、ase. SMIL is XML-based, so all the tags should be ended properly.,A SMIL document example: Here is the head Here is the body ,SMIL,Head,It normally includes the , , and element as the children of .,SMIL,Body,It allows user to design when and what to be displayed in the regions defined in the head of

11、 the document. The media object elements allow the inclusion of different media objects into a SMIL presentation. Media objects are included by reference (using a URI). Different types of media object(animation, audio, img, video, text or textstream) is reflected in the element name .,SMIL,Timing an

12、d Synchronization,Sequence Example:,Parallel Example: ,SMIL,Players and Browsers,RealPlayer RealNetworks RealPlayer is one of the best-equipped Web multimedia players around. The G2 version of RealPlayer started supporting SMIL. SMIL for RealPlayer is the most dominant use of SMIL nowadays. QuickTim

13、e GRiNS Internet Explorer Approaching SMIL from the browser side,SMIL,XSL,Stands for “Extensible Stylesheet Language” XSL is the language defined by the W3C to add formatting information to XML data. Stylesheets allow data to be formatted based on the structure of the data, so one stylesheet can be

14、used with many similar XML documents.,XSL,XSL,XSL consists of two different standards: Transformation language (XSLT) A common language for transforming one XML document into another Defines how to create a result tree from a source tree. Formatting objects (FO) specifications for how to display doc

15、ument content currently no XSL-FO processing applications in common use.,XSL,Working Principles,XSL,XSLT Processors,Xalan both Xalan C+ (using C+) and Xalan-J (using Java) available Saxon runs on Java MSXML Can be used in VB, VC+, scripts,XSL,Transformation Process,Input files: XML file generated af

16、ter doing video information extraction (e.g. speech recognition, scene change detection, VOCD ) Data files in XML format for the additional information (e.g. details of major cities, famous people),Transformation,Transformation Process,Output files for SMIL presentation: a .smil file for layout and

17、synchronization of the whole presenetation some .rt files for realtext (e.g. displaying transcript of the video, details of major cities and famous people),Transformation,Design 1,Write the whole transformer with our own code. Details: Use VC+ to read all the input files, get the information and com

18、bine them. Create the output the files for the SMIL presentation, simply using fprintf in VC+.,Transformation,Design 1,Disadvantage:The layout and design of the SMIL presentation needed to be hard-coded in the VC+ program, make it program-dependent. The layout and design of the presentation is incon

19、venient to be changed by user. The transformer is hard to extend as whenever a new module is added, a lot modification on the program is needed.,Transformation,Design 1 with modification,Provide an additional file or interface as a template for user to define the layout or design the SMIL presentati

20、on.Disadvantage: The flexibility provided is still limited. Not a standard way to define a template.,Transformation,Design 2,Use XSLT processor assisting the XML to SMIL transformation. User can define their own template with XSL. Advantage: Program-independent Extensible Standard templates,Transfor

21、mation,Transformation Process,Two Parts: Knowledge Enrichment - Combine the data files of additional information with the original XML file generated after video information extraction Create output files - Use the enriched result to create the output files for the SMIL presentation using XSL.,Trans

22、formation,Knowledge Enrichment,Do combination between the two XML files Since XSLT cannot read 2 input files and it is not target for comparing and combining files. We have to implement this part with our own code.,Transformation,Combined XML,This XML file contains the details of the additional info

23、rmation that will be used in the SMIL presentation.,香港 中國南部一個沿海城市 China紐約 隸屬美國紐約州的城市 America,Transformation,Create output files,For different realtext files,Transformation,Create output files,For the SMIL file It contains layout and content to be displayed in different regions An XSL file defining t

24、he presentation layout is required Different regions need different XML data file storing the relative information Different regions have different timing and synchronization order for each child object Challenge Need to combine all the above information together, with proper layout for the presenta

25、tion and correct displaying order for each child object in different media.,Transformation,Create output files,Limitation of XSL: It can only read one input data file and one XSL file, and generate one output. It cannot do combination between files. Solution: We use separate XSL files to generate th

26、e layout of the whole presentation and the displaying order for the required regions. Then, combining the generated result together with our own code.,Transformation,Create output files,Solution,Combine-city.xml,Mapdisplay .xsl (XSLT),Map-display. txt,(Combination),.Smil,Combine-peop.xml,Peopdisplay. Xsl (XSLT),Peopdisplay. txt,Transformation,THE END,