The Common Component Architecture and XCAT.ppt

1、The Common Component Architecture and XCAT,Indiana University Extreme! Lab,What is a Component Architecture?,A systematic way of encapsulating special functionality and behaviors of a piece of software into reusable units. More than an object model. It defines the rules for the way objects can be in

2、stantiated and composed. It defines the environment of services that a component can use. It defines the way in which we discover how to interact with them.,Isnt this what a subroutine library does?,Subroutine libraries or class libraries have well defined interfaces encapsulate functionality But ar

3、e often hard to reuse because they often have complex resource requirements make conflicting assumptions about their operating environments. Subroutine libraries that follow very strict sets of design and use rules get reused. Above all else, a component architecture is framework of standard rules o

4、f behavior for objects.,Standard Component Archtectures,Examples: Microsoft COM/DCOM, COM+ the foundation of all Microsoft office products Java Beans Java standard for building user interfaces Enterprise Java Beans a standard for client-server applications in Java CORBA 3.0 The new component spec fo

5、r CORBA.,Look at One Design in Detail,The U.S. Dept of Energy DOE2000 project The Common Component Architecture Lawrence Livermore National Lab Sandia Labs Argonne National Labs Los Alamos National Labs Universities: Utah, NCSA, Indiana A specification for component design for parallel and distribut

6、ed applications,Two parts to the CCA Architecture,Components An encapsulated “object” defined by its public interfaces. Can be Java, C, Python, C+ or C+ and Fortran. Frameworks The software systems that provides basic services to components Used to compose components to make apps. Many implementatio

7、ns: Parallel and Distributed.,Some Concepts in Detail,Ports: the public interfaces of a component defines the different ways we can interact with a component and the ways the component uses services and other components.,Image Processing Component,setImage(Image I),Image getImage(),adjustColor(),set

8、Filter(Filter),calls doFFT(),Provides Ports - interfaces functions provided by component,Uses Ports - interface of a service used by component,How do you “compose” two components?,There are three basic approaches: Events: A component can generate an “event” of a particular type. Other components are

9、 added as “listeners” for such events from that component. Services: A source component “uses” services “provided” by a target components. “use” = calls a function provided by the target. Dataflow: A typed data “output” stream is connected to a similarly typed “input” port of another. In XCAT all th

10、ree are supported. The services model is the basic mechanism. Dataflow is emulated in CCA by “pushing” data from a user to a provider, or by having a user “pull” data from a provider. Events are managed by a separate mechanism.,Acme FFT component,Building Applications by Composition,Connect uses Por

11、ts to Provides Ports.,Image Processing Component,getImage(),adjustColor(),Image tool graphical interface component,Image database component,setImage(),doFFT(),Ports and Interfaces (CCA),Component composition: Connect (possibly at runtime) a “provides” port of one component to a “uses” port of anothe

12、r. A provides port is simply an implementation in the component of the interface defined by the port type (interface defn.). A Uses port is a “proxy” that a component “uses” to make a request of another component.,Source Component,Target Component,A Provides port,A uses port,How do you describe the

13、interfaces a port has?,Standard Methods: Use an Interface Definition Language - a formal description of the interface objects. CORBA IDL is one. The LLNL Scientific IDL by Scott Kohn, Andrew Cleary, Steven Smith, and Brent Smolinski is better for CCA. A simple IDL compiler can be used to automatical

14、ly generate the Uses and Provides Port code for a specific IDL type. For the XCAT java version the interface of a component port are defined as Java interfaces.,Component Communication,Use a simple Remote Procedure Call MechanismXSOAP Implementation of Simple Object Access Protocol (SOAP)Performance

15、 Issuesin-process callsEvents/MessagesObjects encoded as XML documentsProteusMultiprotocol Messaging and RMI,X,Creation Service,Creates a running instance of another component Encapsulates authentication issues,Creation Service Component,Launch an instance of component X on resource Y,Returns: remot

16、e reference to new component instance,Globus resource Y,Connection Service,A component that can be used to connect a “uses” port of one component to the “provides” port of another Can export ports of another component,Y,Connection Service Component,X,Connect port A of component X to port B of compon

17、ent Y,A,B,Builder Service,combination of Creation and Connection service standardized as part of recently updated CCA specification we are now in process of implementing it in XCAT,Simple Example in Java,public class SimpleEchoImpl implements SimpleEcho public String echoHello(String s) throws Remot

18、eException return “SimpleEchoImpl says: Hello “ + s; ,Simple Example in Java,public class EchoPrinterComponent implements Component public void setServices(Services cc)PortInfo portType = new PortInfoImpl(“simpleEchoProvidesPort“, “http:/ = new SimpleEchoImpl();cc.addProvidesPort(simpleEchoImpl,port

19、Type); ,Simple Example in Java,public class EchoGeneratorComponent implements Component public void setServices(Services cc)PortInfo portType = new PortInfoImpl(“simpleEchoUsesPort“, “http:/ ,Simple Example in Java,SimpleEcho usesSimpleEcho = (SimpleEcho) cc.getPort(“simpleEchoUsesPort“);usesSimpleE

20、cho.echoHello(“Extreme Lab“);cc.releasePort(“simpleEchoUsesPort“);,Echo Generator Component,Echo Printer Component,A Provides port simpleEchoProvidesPort,A uses port simpleEchoUsesPort,Scripting XCAT Applications,import xcat generator = xcat.createComponent(GeneratorComponet) printer = xcat.createCo

21、mponent(Printer)xcat.setCreationMechanism(generator, gram) xcat.setCreationMechanism(printer, ssh)xcat.createInstance(generator) xcat.createInstance(printer)xcat.connectPorts(generator, simpleEchoUsesPort,printer, simpleEchoProvidesPort),Scripting XCAT Applications,generatorComponent = EnvObj() gene

22、ratorComponent.put(“exec-name“, “simpleGeneratorComponent“) generatorComponent.put(“exec-fqn“, “samples.simpleEchoGenerator.SimpleEchoGeneratorComponent“)printerComponent = EnvObj() printerComponent.put(“exec-name“, “simplePrinterComponent“) printerComponent.put(“exec-fqn“, “samples.simpleEchoPrinte

23、r.SimpleEchoPrinterComponent“)# create component wrappers generator = cca.createComponent(generatorComponent) printer = cca.createComponent(printerComponent)# assign a machine name printermc = “exodus.extreme.indiana.edu“ generatormc = “exodus.extreme.indiana.edu“ cca.setMachineName(generator,genera

24、tormc) cca.setMachineName(printer, printermc),# create live instances cca.createInstanceWithTimeOut(printer, 120000) cca.createInstanceWithTimeOut(generator, 120000)# connect their ports cca.connectPorts(generator, “simpleEchoUsesPort“, printer, “simpleEchoProvidesPort“)# start the components invoke

25、 go() on GoPortusesGoPortClassName = “samples.idl.goPort.UsesGoPort“ usesPortType = “http:/ providesPortName = “providesGoPort“ methodName = “go“ methodParams = zeros(0, Object) cca.invokeMethodOnComponent(generator, usesGoPortClassName, usesPortType, providesPortName, methodName, methodParams)print

26、 “Done“,Scientific Components and Applications,Recognized as one of the “Top Ten Science Achievements in 2002” by the DOE Office of Science (see http:/www.sc.doe.gov/sub/accomplishments/top_10.htm) Many demonstrated at SC2001 and SC2002 Combine application-specific components with more general-purpo

27、se components that can be reused across a range of applications More than 40 components, many reused in apps such as PDEs on unstructured and adaptive structured meshes Unconstrained minimization problems Leverage and extend parallel software developed at different institutions Including CUMULVS, CV

28、ODES, Global Arrays, GrACE, MPICH, PETSc, PVM, and TAO,Current Interface Development Activities,CCA Scientific Data Components Working Group Basic Scientific Data Objects Lead: D. Bernholdt (ORNL) Unstructured Meshes Lead: L.F. Diachin (SNL, formerly ANL) Collaboration with TSTT ISIC TSTT = Terascal

29、e Simulation Tools and Technologies, PIs: J. Glimm, D. Brown, L.F. Diachin, http:/www.tstt-scidac.org Structured Adaptive Mesh Refinement Lead: P. Colella (LBNL) Collaboration with APDEC ISIC APDEC = Algorithmic and Software Framework for Applied PDEs, PI: P. Colella, http:/davis.lbl.gov/APDEC,Other

30、 Groups Linear and nonlinear solvers, eigensolvers, and optimizers Coordinator: L.C. McInnes (ANL) Collaboration with TOPS ISIC TOPS = Terascale Optimal PDE Simulations, PI: D. Keyes, http:/tops-scidac.org MxN Parallel Data Redistribution Lead: J. Kohl (ORNL) Part of CCTTSS MxN Thrust Quantum Chemis

31、try Leads: C. Janssen (SNL) and T. Windus (PNNL) Part of CCTTSS Applications Integration Thrust,Motivation for Common Interfaces,Many-to-Many couplings require Many 2 interfaces Often a heroic effort to understand details of both codes Not a scalable solution Common Interfaces: Reduce the Many-to-Ma

32、ny problem to a Many-to-One problem Allow plug-and-play interchangeability & interoperability Require domain specific experts Typically difficult & time-consuming A success story: MPI Challenges Interface agreement Functionality limitations Maintaining performance,NWGrid,Overture,MDB/CUBIT,AOMD,Hypr

33、e,PETSc,SuperLU,mesh libraries,linear solver libraries,NWGrid,Overture,MDB/CUBIT,AOMD,SuperLU,PETSc,Hypre,D a t a,Others ,Others ,S o l v e r s,TSTT Data Interfaces,TOPS Solver Interfaces,Framework Stays “Out of the Way” of Component Parallelism,Single component multiple data (SCMD) model is compone

34、nt analog of widely used SPMD model Each process loaded with the same set of components wired the same way,Components: Blue, Green, Red Framework: Gray,MCMD/MPMD also supported,Different components in same process “talk to each” other via ports and the framework,Same component in different processes

35、 talk to each other through their favorite communications layer (i.e., MPI, PVM, GA),There are 3 CCA Frameworks,Theres 3 parallelism models in scientific computing,Ccaffeine,SPMD,Distributed,Threaded,Uintah,XCAT,Ccaffeine,Ccaffeine,SPMD GUI and Scripted Interface Interactive or Batch Serial or Paral

36、lel Components written in C+,Separates CCA Pattern from Implementation 3 Bindings Classic Components SIDL Components Chasm Components Demonstrated at SC2001 & SC2002 Used in CCA Tutorials,Characteristics,Achievements,SCIRun/BioPSE/Uintah,multithreaded & distributed C+ only Used in “real science” (go

37、vt, academic, commercial) 1K procs,Testbed for CCA Concepts Novel work in IDL-based MxN Parallelism and concurrency research for CCA done on Uintah SCIRun2 will integrate SCIRun, BioPSE and Uintah,Characteristics,Achievements,XCAT,Distributed/Grid model Web Services,Developed Proteus multi-protocol

38、communication package Novel MxN work at MPI-I/O level Java and C+ components,Characteristics,Achievements,Material Archive,Go,Data,GS,Cntl,Go,Go,GS,GS,Go,Data Provider Comp,Simulation Component,exported,exported,exported,Application Coordinator,sendParameters, start, kill,Application Specific compon

39、ent Application Factory Service,Directory/ Discovery Service,1,2,3,Authorization Service,Resource Broker Service,4,5,6,7 wsdl,Ensemble Application,8 Application control,A Science Portal View of “programming” the Grid,A Science Portal is a Web server that Uses a “prepackaged” set of scripts to Get th

40、e users proxy cert from a cert repository Configure a specific application to users needs Contact the appropriate application factories Look for event histories of application execution Allow the user to contact and control the app.,User Portals/ Science Portals,Launch, configure And control,App fac

41、tory,App Instance,Application Factory Service,A service that understands a how to instantiate a running instance of an app component. You provide it with appropriate requirements initial conditions, etc. via an XML file The service checks you credentials and authorization May consult resource broker

42、 launches the app or runs the appropriate grid script.,App factory,App Instance,Provide me with an instance of application X,Application Component Execution Environment Specification,WebsterComponentWebster ComponentIndiana University Extreme! computingDennis Gannonolympus.cs.indiana.edulinbox2.extr

43、eme.indiana.edurainier.extreme.indiana.edugramssh exec-dir/u/gannon/xcat_tutorial/scripts.,The information needed to generate input to the factory service,Steps in creating a app instance from the portal to the factory,User Configures Application from Web Browser Sets application parameters Launches

44、 job,User Portals/ Science Portals,Web Server contacts appropriate application factory service (FS) Supplies FS with task parameters FS contacts Resource Broker and secures job launches. Returns App WSDL to server to browser,Job begins execution Publishes its contact point (WSDL) to discovery servic

45、e Begins publishing status events to event channel.,Web server discovers application Allows user to interrogate it or retrieve event traces,An Application Instance,Typical Instance Publishes event stream to “well known” channel.Has a Control Interface to allow portal level interrogation or remote st

46、eeringMay be linked to other components/services,Links To Other Apps/ services,Control Interfacecheck statuscontrol messages,Application Instance,Event stream,Encapsulating Legacy Apps,Common Case Legacy App that reads files and writes files. Use a “scripted component” called an app manager. Compone

47、nt runs a python script loaded at startup or through a control command The App Script Stages files Launches and monitors application Writes Output files publishes event streams,application,App Mgr,input files,outputfiles,App Script,Event Stream,Control,XCAT And OGSI,A component based model to the Gr

48、id services Component Assembly: Composition in space Workflow: Composition in time Ability to use widely available Web services tooling to interact with components A richer messaging and notification system that extends the model proposed by OGSI An application factory service that extends the stand

49、ard OGSI factory service,XCAT-OGSI Big Picture,XCAT Component,GridService Port (for lifecycle andmetadata),Other XCAT Provides Ports,ComponentID (GSH) (uniquely identifies Component),ServiceData (includes list of Port references),Incorporating OGSI into XCAT,Converting an XCAT component to a Grid se

50、rvice Addition of an OGSI GridService Port Addition of Service Data Elements (SDEs) containing references to each of the other ports Uniquely identifying the component using a ComponentID (GSH) Using a WSDL representation for the GSR Adding a set of helper services that are OGSI compliant A HandleResolver service for mapping a GSH to a GSR Modifications to the XCAT Creation service to make appropriate calls to the Factory and GridService ports for lifetime management,