1、Internet and Intranet Protocols and Applications,Lecture 14Introduction to MQSeries And Asynchronous Messaging May 1, 2002Joseph Conron Computer Science Department New York University jconroncs.nyu.edu,What is MQSeries?,A middleware product that implements a messaging and queuing framework.Middlewar
2、e - an intermediate software component that bridges dissimilar computing environments. Unix, MVS, OS/400 Tandem, VMS, NT, etc.SNA, NetBios, TCP/IP Cobol, C, JAVA,Messaging and Queueing,Messaging - programs communicate by sending data in messages rather than by calling each other directly. Queuing -
3、messages are put on queues in storage, eliminating the need for programs to be logically connected. A messaging and queuing framework is inherently ASYNCHRONOUS!,Asynchronous vs. Synchronous Communications,Synchronous: App sends request, then blocks until request is processed. Requires service avail
4、able at EXACTLY same time as client needs service. Asynchronous: App sends request and checks at some future time if complete. Service need not be available when client sends request,Synchronous: good/bad,Easy to program Outcome is known immediately Error recovery easier (usually) Better real-time r
5、esponse (usually),Service must be up and ready requestor blocks, held resources are “tied up” Usually requires connection-oriented protocol,Good,Bad,Requests need not be targeted to specific server. Service need not be available when request is made No blocking, so resources could be freed Could use
6、 connectionless protocol,Response times are unpredictable error handling usually more complex Usually requires connection-oriented protocol Harder to design apps?,Good,Bad,Asynchronous: good/bad,Messaging vs. Procedure Calls,As programmers, many of us think procedurally, so using procedures is “natu
7、ral” extension of how we think. Messages are an abstract concept: harder for us to conceptualize relationship between actions and messages! But Wait! Something good happens when we use abstractions!,We are free to concentrate on the design of the application itself. We are no longer concerned with d
8、etails of the environment. Our application is suddenly portable and to some degree, extensible.,A Brief History of MQSeries,1992 Systems Strategies (SSI) develops ezBridge, a messaging and queuing product for VMS, Tandem, and Unix IBM announces Networking Blueprint defining three standard APIs for p
9、rogram to program communication: CPI-C, RPC, MQI1992-3State Street Bank (Boston) evaluates IBM messaging product (code name “Victory”) for IBM CICS/ESA and SSIs ezBridge on VMS and Tandem. “State Street Bank would like to announce the wedding of IBM and Systems Strategies!” 1993 IBM buys intellectua
10、l property rights for ezBridge from SSI,The Brides Wedding Preparation (What SSI had to do),Implement IBM Channel Protocol over TCP/IP and LU6.2 (more about channels later).Implement the MQI interface functions (MQCONN, MQOPEN, MQPUT, MQGET, MQCOMMIT, MQCLOSE, MQDISC).Implement MQI Error Semantics (
11、failure conditions should look the same on all systems),MQSeries/ezBridge Integration Objectives,Applications written in C using ezBridge on VMS and Tandem had to exchange messages with applications written in (COBOL?) Using MQI under CICS/ESA.System intercommunication using channel protocol over TC
12、P/IP and LU6.2 (that is, A VMS system had to “look” to IBM machine just like another IBM system!).,MQSeries Platform Rollout,Initially, IBMs version of MQSeries ran only on mainframe (CICS/ESA, IMS/ESA, and eventually VSE).SSI version (called MQSeries Version 1) initially released on: VMS, Tandem, A
13、S/400, and Unix (SCO, UnixWare).1994/1995 -IBM releases the first three “distributed” platforms: AIX, OS/2, and AS/400. The AIX version becomes the “reference port”.Over time, IBM replaced SSI versions with “ports” of its reference system.Today - MQSeries runs on over 35 platforms,How Messaging & Qu
14、euing Works,Programs communicate by putting messages on queues. Here, program A puts a message on Queue1, which is read by program B.,Note: A and B need not be on the same machine!,How Messaging & Queuing Works (2),Communication can be one-way or two-way. Here, A sends to B on Queue1, and B responds
15、 to A on Queue2,Messaging and Queuing Characteristics,Three key facts about Messaging and Queuing differentiate it from other communication styles: 1) Communicating programs can run at different times. 2) There are no constraints on application structure. 3) Programs are insulated from environmental
16、 differences. Lets examine each of these characteristics in more detail.,Applications Can Run at Different Times,Either program can be unavailable,Key Concept: message queue exists independently from programs that use them!,No Constraints on Application Structure,There can be a one to many relations
17、hip between applications,Or a many to one relationship between applications,Applications Shielded from Environmental Differences,Dont care what OS is used. Dont care what language theyre written in Dont care what the underlying communication protocol is used.,Applications Shielded from Environmental
18、 Differences,Applications,Programmatic API,Communications using Message Channels,Queue Manager,Queue Manager,Message Queue,MQSeries Objects: Queue Manager,Queue Manager Controls access to queues: administration (create, delete, etc) usage (Put, Get) serves as transaction (syncpoint) coordinator for
19、all queue operations. Accessed through the Message Queue Interface (MQI) Queue Managers have names (identities) that are UNIQUE in a network (like host names).,MQSeries Objects: Queues,MQSeries defines four types of queues. A queue instance is fully qualified by its queue manager and queue name.Loca
20、l Queue - an actual queue for which storage is allocated.Remote Queue - a definition of a queue on a different queue manager (acts somewhat like a pointer)Alias Queue - another name for a local or remote queue. Typically used to switch queue destinations without modifying program code.Model Queue -
21、a template whose properties are copied when creating a new dynamic local queue (“ create queue xxx “like” queue yyy).,MQSeries Queues: Properties,Maximum Message Size Maximum Queue Depth High/Low Factors Enable/Disable Put or Get Persistent/Not Persistent,MQSeries Queues: Events and Triggering,Local
22、 queues can generate events (messages) under certain conditions (like queue full). These “event” messages can be used to “trigger” the execution of a program. These events are called trigger messages. The queue on which they are put is called an Initiation Queue.,MQSeries Objects: Processes,Process
23、defines an application to an MQSeries queue manager. A process definition object is used for defining applications to be started by a trigger monitor.A trigger monitor is a program that listens on an initiation queue and executes commands named in Process definitions.Triggering is useful when you do
24、nt want to deploy long-running programs.,MQSeries Objects: Message Channels,provide a communication path between two queue managers on the same, or different, platforms. A message channel can transmit messages in one direction only. If two-way communication is required between two queue managers, tw
25、o message channels are required. Question: why make message channels one-way?,MQSeries Objects: Message Channels(2),Types of message channels: Sender - initiates connection to Receiver Server - Accepts request to start from requester, then becomes Sender Receiver - Passive; waits for initiation sequ
26、ence form Sender Requester - Active at start, then becomes Receiver Cluster-sender (used amongst Cluster Queue Managers) Cluster-receiver (ditto),MQSeries: Message Channel Concepts,The Sender side of the session is the “transaction coordinator”. Message channels implement a protocol that includes a
27、commitment protocol. Channels recover from failure by agreement: they must agree on the last committed unit of work would this be harder if channels were bi-directional? Message Chanels are implemented by programs called Message Channel Agents (MCA),How messages move across channels,(1) Application
28、puts message on transmission queue,(2) Sender MCA gets message and sends to partner MCA,(3) Receiver MCA puts message on target queue,(4) Message is available on local queue for applications,MQSeries: Assured Delivery,If queues are persistent, and message is persistent, then MCAs will eventually del
29、iver the message to the target queue, and target application will get it!MCAs have recoverable state - theyre STATEFUL - and a connection-oriented protocol.Message is not removed from xmit queue until partner MCA confirms placement on target queue,MQSeries Objects: Messages,Messages consist of: Head
30、er (MQMD) Used by Queue Manager and application to handling properties of the message User Data The application-to-application data (“payload”) transparent to MQSeries,MQSeries Messages: Properties,Destination Queue Reply Queue name Time to live (expiry) Format Correlation ID Persistence “Report” op
31、tions,MQSeries Messages: Properties (2),Messages can be individually designated persistent or non-persistent (persistent messages are logged to enable recovery) Correlation ID - select which message to get from queue Segmented Messages - allows ending of VERY LARGE messages ( 100 MB),MQSeries Messag
32、es: message types,Request - used by one program to ask another program for something (usually data). A request message needs a reply. Reply - used in response to a request message. A one-way message, as you would expect, doesnt need a reply, though it can carry data. A report message is used when so
33、mething unexpected occurs, or to give extra information like: message delivered to target queue message taken by application message not deliverable message exceeded time-to-live,MQSeries Messages Units of Work and Transactions,Messages are added and removed from queues in Units of Work The smallest
34、 Unit of Work is one message. Units of work are atomic. When an app reads a message from a queue, a message “appears” to have been removed, but in fact, it is still in storage until the app “commits” the unit of work.,MQSeries: Transaction Support,Unit of recovery - a piece of work that changes data
35、 from one point of consistency to another.Syncpoint - A point of consistency (also called a or commit point). It is a moment at which all the recoverable data that an application program accesses is consistent.,MQSeries: Transaction Support (2),Applications are responsible for delimiting the beginni
36、ng and end of a transaction. How can messaging be coordinated with a data base update?MQSeries is XA compliant and can operate with other XA compliant systems as either a transaction manager (coordinator) or resource manager (particpant). Some examples: Sybase, DB2, Oracle.,MQSeries: Logging and Rec
37、overy,All operations that affect the “state” of the Queue Manager and its objects are logged to a log file. What is “state”? Object definitions (queue manager, queues, processes, channels, etc) Queue content (messages) What about message channel state? Message channel states are logged separately by
38、 each channel,MQSeries: Logging and Recovery (2),Two forms of Logging Circular log records are written sequentially across several files, then “wrap” back to the first file. Linear - log records are written sequentially across files. New files are allocated as current files fill. No automatic reuse
39、of file space! Problem: Length (in time) of longest running transaction vs amount of writes to log determines size of log needed.,MQSeries: Logging and Recovery (3),Observations: Circular logging is easy to manage, but is fatal if log is damaged (hard to backup circular logs!). Linear logging is har
40、d to maintain but provides for archiving of previous logs (still a problem if “current” log is damaged).,MQI: The MQSeries Programming Interface,MQCONN Connect to queue managerMQDISC Disconnect from queue managerMQOPEN Open queueMQCLOSE Close queueMQPUT Put messageMQGET Get message,MQI: The MQSeries
41、 Programming Interface (2),MQPUT1 Put one message (implied open/put/close)MQBEGIN Begin unit of workMQCMIT CommitMQBACK Back outMQINQ Inquire about queue attributesMQSET Set queue attributes,MQSeries: Language Support,C C+ Cobol JAVA (native and JMS) PL/I System 390 Assembler TAL (Tandem) Visual Basic For More Information:http:/
