View
188
Download
49
Category
Preview:
DESCRIPTION
mq presentation
Citation preview
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 1
INTRODUCTION TO MIDDLEWARE Websphere MQSeries
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 2
MQSeries
Middleware? It’s a software or software components that help heterogeneous
clients to connect to a common server/servers.
MQSeries? It allows independent and potentially non-concurrent applications on a
distributed systems to communicate with each other
Messaging & Queuing? Message queuing enables distributed applications to reliably
exchange data regardless of hardware, operating system, or available connectivity.
Analogous to e- Mail system Links applications across disparate platforms
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 3
Messaging & QueuingINTRODUCTION
Messaging & Queuing
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 4
Communication Styles
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 5
Messaging & Queuing
Messaging - Applications communicate by sending data in messages rather than by calling each other directly.
Queuing - Messages are put on queues in storage, eliminating the need for programs to be logically connected.
Applications doesn’t need to bother about the inherent network communications, which is taken care by the messaging & queuing products.
Messaging & Queuing provides synchronous & asynchronous communications between the applications.
A messaging and queuing framework is inherently ASYNCHRONOUS!
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 6
Asynchronous vs. Synchronous Communications
Synchronous: Application sends request, then blocks until request is processed.
Requires service available at EXACTLY same time as client needs service. It supports symmetrical, circuit-switched, point-to-point connections
Asynchronous: Application sends request and checks at some future time if complete.
Service need not be available when client sends request. It supports symmetrical and asymmetrical, packet-switched point-to-
multipoint connections .
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 7
Synchronous Communication
Advantages Disadvantages
Easy to programOutcome is known immediatelyError recovery easier (usually)Better real-time response (usually)
Service must be up and readyRequestor blocks, held resources are
“tied up”Usually requires connection-oriented
protocol
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 8
Asynchronous Communication
Advantages Disadvantages
Service need not be available when request is made
No blocking, so resources could be freed
Could use connectionless protocol
Response times are unpredictable Error handling usually more
complex
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 9
What's in a message?
Application messages also have two parts: 1. The application data (Content of message)2. The message envelope (Message Identifiers)
Two major components that make up the messaging environment are Messaging API Message-queue management services
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 10
What is in a Queue?
A queue is an abstract data type where Data is added to the tail of a list and Data retrieved and/or removed from the head of the list.
General Characteristics Queues are described as having First In, First Out (FIFO) behavior,
where the earliest added entries are removed first. Some queues enforce strict FIFO ordering, but the message position
may vary as per their priorities.
Physical nature of Queue A volatile buffer area in the memory of a computer or Data set on a permanent storage device (such as disk)
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 11
Queuing
Queuing is the mechanism by which messages are held until an application is ready to process them.
Queuing allows applications to:• Communicate between programs (which may each be running in different
environments) without having to write the communication code.
• Select the order in which a program processes messages.
• Store messages unless being retrieved by receiving application.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 12
Main Features of Messaging & Queuing
There are no direct connections between programs.
Communication between programs can be time-independent.
No Constraints on Application Structure
Communication can be driven by events.
Applications can assign a priority to a message.
Applications Shielded from Environmental Differences
Security.
Data integrity.
Recovery support.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 13
Main Features of Messaging & Queuing
No direct connections between programs
The mechanism by which a message moves from queue to queue is hidden from the programs. Hence the programs are simpler.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 14
Main Features of Messaging & Queuing
Time-independent communication Programs requesting others to do work do not have to wait for the reply to a request.
Either program can be unavailable
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 15
Main Features of Messaging & Queuing
There can be a one to many relationship between applications
No Constraints on Application Structure Programs requesting others to do work do not have to wait for the reply to a request.
There can be many to one relationship between applications
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 16
Main Features of Messaging & Queuing
Event-driven processing Programs can be controlled according to the state of queues. E.g. Programs
can start as soon as messages arrives on the queue.
Message priority A program can assign a priority to a message when it puts the message on
a queue. This determines the position in the queue at which the new message is added.
Applications Shielded from Environmental Differences Don’t care what OS is used. Don’t care what language they’re written in Don’t care what the underlying communication protocol is used
Security Authorization checks are carried out on each resource like queues
Data integrity Data integrity is provided via units of work.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 17
MQ Series Messaging Platforms
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 18
MQ Series Messaging Platforms
MQSeries Messaging Platforms
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 19
Websphere MQ Series Basics
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 20
MQ Series Basics
MQSeries is a middleware product from IBM that runs on multiple platforms and enables applications to send messages to other applications.
MQSeries is renamed as WebSphere MQ now.
MQSeries takes care of the storage, logging and communications details required to guarantee delivery of the message to
the destination queue.
MQSeries can take care of translating the data when source and target application uses different character sets.
EBCDIC on MVS vs. ASCII on NT or Unix
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 21
MQ Series Extended Products
MQ forms the basis of a family of products that include: MQSeries Integrator – A message broker with data transformation and
mapping functions (MQSI – now called WebSphere Business Integration Message Broker)
MQSeries Workflow - Model-driven e-business process automation and tracking
MQSeries Everyplace – Allow mobile workers with laptops, phones and PDA’s to participate in MQ networks.
WebSphere InterChange Server
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 22
MQ Series Basics
MQSeries use a common application programming interface known as Message Queue Interface (MQI)
MQSeries also provides a high level message handling API called Application Messaging Interface (AMI) which is a simple application programming interface, providing support for point-to-point and publish/subscribe messaging.
MQSeries supports a wide range of programming languages
ActiveXAssembler (MVS/ESA)C, C++COBOLJavaJMSAMILotusScriptPL/1RPG (AS/400)SmallTalkTAL (Tandem NSK)Visual BasicPowerbuilder
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 23
MQSeries Basics - Installation Types
MQSeries Server It’s the Messaging & Queuing software. Provide infrastructure for storing ,
forwarding and maintaining data integrity ,etc for messages
MQSeries Client It’s the software to connect to MQSeries Servers. The communication
between the client and server is synchronous as the client does not have any MQSeries Objects other than set of libraries for applications to connect to MQSeries Servers.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 24
MQ Series – Messaging Patterns
One-to-one or Point-to-Point Each Message is addressed to a specific
Queue Queues retain all messages till they are
consumed or the messages expire
Characteristics Each message has only one consumer. The message consumer can, if required,
acknowledge the successful processing of a message
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 25
MQ Series – Messaging Patterns
One-to-many message flows Messages are put onto a queue as before
but can be read from that queue by multiple processes.
For the purposes of load balancing, or simply that the message information needs to be used in different ways
Characteristics More than one possible consumer for a
message Message consumers can, if required,
acknowledge the successful processing of a message
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 26
MQ Series – Messaging Patterns
Many-to-one message flows Messages from multiple processes are
addressed to a single specific queue. Messages sent to them are retained until the
messages are either consumed or until the messages expire.
Characteristics No timing dependencies between the senders
and consumer Message consumer can, if necessary,
acknowledge the successful processing of a message
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 27
MQ Series – Messaging Patterns
Publish/subscribe Model It is a style of messaging application in which the providers of information (publishers) are
decoupled from the consumers of that information (subscribers) using a broker. Message brokers make sure that messages arrive at the correct destinations, and are
transformed to the format required at each destination.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 28
MQ Series – Messaging Patterns
Request/reply Request/reply pattern involves two processes:
• One consists of sending a message and expecting a reply
• Other sends reply messages upon receiving a request message.
Flow of message is complete when the initiating side receives the reply message.
The request/reply pattern can be used with either one-to-one (point-to-point) or publish/subscribe patterns.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 29
MQ Series – Messaging Patterns
One-way datagram, or send-and-forget pattern Sending application sends messages without expecting any reply from the
receiving application. Fully decoupled systems and communication is asynchronous
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 30
MQSeries Basics - Messages
Messages are String of Bytes Used to transmit information between programs
Two parts of a Message
Application Data Application determines the content
Message Descriptor/Header Identifies the Message and holds the control information
The message descriptor identifies the message (message ID) and contains control information, also called attributes, such as message type, expiry time, correlation ID, priority, and the name of the queue for the reply.
A message can be up to 4 MB or 100 MB long
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 31
MQSeries Basics – MQ Message
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 32
MQ Series – Message Descriptors
Version: • The version of the message descriptor depends on the MQSeries version and platform you use.
Message ID:• This is a byte string that is used to distinguish one message from another • Generally, no two messages should have the same message identifier, although this is not disallowed by the queue manager. The message identifier is a permanent property of the message, and persists across restarts of the queue manager.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 33
MQ Series - Message Descriptors
Correlation ID: These fields allow the application that receives the reply to correlate the
reply with its original request.
Persistent/Non-persistent• Persistent messages always arrive at their destination, even when the system
fails.
• “Hardened” - saved on disk
• specific message persistent or all messages on a particular queue
Time and Date: when the MQPUT occurred
Lifetime: When this date is reached and an MQGET is issued, then the message will be discarded
Format: use to decide whether data conversion can be done or not.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 34
MQSeries Basics – MQ Message Types
Message Types – 4 Types1. Datagram
A message containing information for which no response is expected.
2. Request A message for which a reply is requested.
3. Reply A reply to a request message.
4. Report• A message that describes an event such as the occurrence of an error or a
confirmation on arrival or delivery.• Types
• Exception Report Message
• Expiry Report Message
• Confirmation On Arrival (COA)
• Confirmation On Delivery (COD)
• Positive Action Notification (PAN)
• Negative Action Notification (NAN)
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 35
MQSeries Basics – MQ Message Types
Message Types – Application View1. Persistent
Delivery of persistent messages is assured. Slower Recovery
2. Non-Persistent Delivery is not assured Faster No Recovery
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 36
Websphere MQ Series Objects
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 37
MQSeries Basics – MQSeries Objects
Queue managers Queues Namelists Process definitions Channels Listener AuthInfo
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 38
MQSeries Basics – Queue Manager
Heart of MQSeries is the message queue manager (MQM).
It’s a MQSeries run-time program
Runtime program - It enables developers to run a development application
program that's written for that application. The software company that owns
the application may require a licensing fee or may allow developers to freely
distribute runtime versions with the development application.
Queue Managers have names (identities) that are UNIQUE in a network (like host names).
Multiple Queue Managers can reside on the same machine MQSeries Objects (like Queues & Channels) belong to Queue
Manager.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 39
MQSeries Basics – Queue Manager Functions
Manages MQSeries Objects and Messages Provides Message Queuing Interface (MQI) for applications to
communicate. Applications invoke functions using API Calls
Controls access to queues:
Administration (create, delete, etc)
Usage (Put, Get)
Applications puts Messages to Queue on a Queue Manager
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 40
MQSeries Basics – Queue Manager Functions
Queue manager transfers messages to other queue managers via
channels using existing network facilities, such as TCP/IP, SNA or
SPX. MQM’s require Channels for communication
Serves as transaction coordinator (syncpoint) for all queue operations.
Segments and Assembles the messages if required. Can send one message to more than one destination with one API call
using a user-defined distribution list, thus reducing network traffic.
MQSeries clients do not have a queue manager in their machines. Client machines connect to a queue manager in a server.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 41
MQSeries Basics – Queue Manager control Commands
Control Commands : Control commands to perform operations on queue managers, command servers, and channels.
Create a Queue Manager crtmqm –q (QM1)
Start the Queue Manager strmqm (QM1) (or) strmqm –> to start default QM
Stop the Queue Manager endmqm –i (QM1) c, i and p options
Where (c= Controlled, i= immediate, p=pre-emptive).
Delete the Queue Manager dltmqm (QM1)
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 42
MQSeries Basics – Queue Manager control Commands
Queues and Channels are defined in MQSeries command mode
To enter MQSeries command mode runmqsc QM1 (or) runmqsc -> for default QM
Display Queue Manager attributes Display qmgr
Alter Queue Manager attributes Alter qmgr *
Exit MQSeries command mode end
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 43
MQSeries Basics - Queues
Message queues are used to store messages sent by programs Types of Queues
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 44
MQSeries Basics - Queues1. Local Queue
Queue Owned by Queue Manager to which program is connected Store Messages for programs that use the Same Queue Manager PUT & GET Message The syntax used to create a local queue is given by “DEFINE QLOCAL(XX)” (XX IS THE NAME OF THE LOCAL Q NAME)
2. Remote Queue Queue owned by a different Queue Manager Local Definition of Remote Queue Local Queue Manager forward the message to remote Queue Manager Program cannot read messages from Remote Queue The syntax and the properties needed to be created along with a remote Q is given
by “DEFINE QREMOTE(XY) RNAME(YZ) RQMNAME(ZX) XMITQ(Z1)” (YZ IS THE NAME OF THE Q WHERE THE MESSAGE HAS TO BE
TRANSFERRED,ZX IS THE QMANAGER WHERE THE MESSAGE HAS TO BE TRANSFERRED,Z1 IS THE TRANSMITQ WHICH ACTS LIKE A MEDIATOR TO SEND MESSAGE TO ANOTHER QMANAGER)
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 45
MQSeries Basics - Queues
3. Transmission Queue Local Queue with special purpose Remote Queue is associated with Transmission Queue Intermediate step when sending message to remote queues Used internally by Queue Managers. The syntax and the property to be created along with a transmitQ is given
by
“DEFINE QLOCAL(X1) USAGE(XMITQ)” (X1 IS THE NAME OF THE LOCAL Q BUT AS WE HAVE GIVEN
USAGE AS XMITQ IT WILL BE USED AS TRANSMITQ)
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 46
MQSeries Basics - Queues
4. Dynamic Queue Local Queue defined “on fly” when application needs it. Can be retained by Queue Manager or can be automatically deleted. Types:
• Temporary queues – Do not survive Queue Manager restart
• Permanent Queues – Survive Queue Manager restart
5. Model Queue Not a real queue Collection of attributes to create a dynamic queue
6. Alias Queue Are definitions not real queues Used to assign different names to same physical queue
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 47
MQSeries Basics - Queues
7. Initiation Queue Local Queue used by Queue Manager for triggering purpose Syntax to create is as same as how we create a localQ.
8. Reply-To Queue Name of queue to which reply should be sent. Specified in request message
9. Dead-Letter Queue Queue into which Queue Manager writes the message if the messages cant be written
to queue. Some samples:
• The destination queue is full.
• The destination queue does not exist.
• Message puts have been inhibited on the destination queue.
• The sender is not authorized to use the destination queue.
• The message is too large.
• The message contains a duplicate message sequence number.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 48
MQSeries Basics - Queues
10. Repository Queue Used in conjunction with Clustering Holds the cluster information
11. Event queues Local Queues to hold instrumentation events. Messages are generated by Queue Manager Sample instrumentation (events) are
• A queue becoming full.
• A channel being started
12. Command queues Local Queue named SYSTEM.ADMIN.COMMAND.QUEUE For sending MQ Commands Created automatically along with Queue Manager Creation
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 49
MQSeries Basics - Namelists
Namelists Contains list of other MQ Objects
• Eg, in Cluster, identify a list of clusters for which the queue manager holds the repository
Typically used to identify group of queues Used with queue manager clusters to maintain a list of clusters referred to
by more than one WebSphere MQ object.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 50
MQSeries Basics - Process definitions
Process definitions Associated with triggering Define an application that starts in response to a trigger event. Process definition attributes include
• application ID
• application type
• data specific to the application
To allow an application to be started without the need for operator intervention the attributes of the application must be known to the queue manager. These attributes are defined in a process definition object. The ProcessName attribute is fixed when the object is created; you can change the others using the WebSphere MQ commands or the WebSphere MQ for z/OS operations and control panels. You can inquire about the values of all the attributes using the MQINQ call.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 51
MQSeries Basics – AuthInfo
AuthInfo: An AUTHINFO object contains authentication information used in Secure
Sockets Layer (SSL) encrypted transport of information. An AUTHINFO object of AUTHTYPE CRLLDAP provides the definitions required to perform Certificate Revocation List (CRL) checking using LDAP servers.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 52
MQSeries Basics - Channels
Channels Provide communication path between Queue Managers Logical communication link Channel Types
1. Message channels• Connects two Queue Managers via Message Channel Agents (MCAs).• Unidirectional• Consists of Sender & Receiver• Transfers messages from XMIT Queue to communication link and from Communication link to
target Queue2. MQI channels
• Transfer MQI calls from a WebSphere MQ client to a queue manager, and responses from a queue manager to a WebSphere MQ client.
• MQI Channel is Bi-Directional
Supported Transport Types
SNA LU 6.2
TCP/IP,
NetBIOS, SPX
DEC Net
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 53
MQSeries Basics - Channels
Channels SENDER CHANNEL(SDR) RECIEVER CHANNEL(RCVR) SERVER CONNECTION CHANNEL(SVRCON)
SENDER CHANNELS COMMANDS: DEFINE CHANNEL(Channel.name) CHLTYPE(SDR) CONNAME(‘IP
Address(port)’) XMITQ(Transmission Queue name) TRPTYPE(TCP) Creates a Channel with Channel.name And ChannelType is Sender(SDR)
With Connection Name along the Designation port number of the Listener And Defining XMITQ with transmission queue name and Transmit Type is TCP (We can use TCP,SUA etc.).
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 54
RECIEVER CHANNELS COMMANDS: DEFINE CHANNEL(Channel.name) CHLTYPE(RCVR)
Creates a Channel with Channel.name And ChannelType is Reciever(RCVR)
NOTE: Sender Channel name and corresponding peer side Reciever Channel name should be same.
SERVER CONNECTION CHANNEL(SVRCON) : DEFINE CHANNEL(Channel.name) CHLTYPE(SVRCON)
Creates a Channel with Channel.name And ChannelType is Server Connection(SVRCON)
MQSeries Basics - Channels
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 55
Listener: A channel listener is a program to start receiving (responder) MCAs Responder MCAs are started in response to a startup request from the
caller MCA; the channel listener detects incoming network requests and starts the
associated channel.
DEFINE LISTENER(Listener.name) TRPTYPE(TCP) PORT(Port no) Creates a Listener with Listener.name And Transmit Type is TCP And Port
no of the Listener
MQSeries Basics - Listener
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 56
MQSeries Basics – Installation & Basic Steps
Install MQSeries
Create Queue Manager crtmqm -q -u SYSYTEM.DEAD.LETTER.QUEUE MYQMGR
Creates Default Queue Manager by Name MYQMGR Creates dead letter queue SYSYTEM.DEAD.LETTER.QUEUE
Strmqm Starts Queue Manager
runmqsc Define Local Queue
define qlocal(’QUEUE1’) replace descr (’test queue’)
Define Remote Queue DEFINE QREMOTE(Q1) RNAME(Q1) RQMNAME(QMB) XMITQ(QMB)
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 57
MQSeries Basics - API
1. MQCONN Connect to queue manager
2. MQDISC Disconnect from queue manager
3. MQOPEN Open queue
4. MQCLOSE Close queue
5. MQPUT Put message
6. MQGET Get message
7. MQPUT1 Put one message
8. MQBEGIN Begin unit of work
9. MQCMIT Commit
10. MQBACK Back out
11. MQINQ Inquire about queue attributes
12. MQSET Set queue attributes
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 58
MQSeries Basics - How MQ Series Works
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 59
MQSeries Basics - How MQ Series Works
The queuing system consists of the following parts: Queue Manager (MQM) Listener Trigger Monitor Channel Initiator Message Channel Agent (MCA) or mover
Application uses MQI to communicate with Queue Manager
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 60
MQSeries Basics - How MQ Series Works
Steps1. Application program issues MQPUT API call when it wants to put a
message on a queue.
2. Queue Manager checks if Queue is Local or Remote
3. If Local Queue QM adds relevant Message Headers and places message in the Queue
4. If Remote Queue QM adds relevant Message Headers and places the message in the XMIT Queue.
• Header contains information from the remote queue definition, such as destination queue manager name and destination queue name.
• Each remote queue must be associated with an xmit queue.
• Usually, all messages destined for one remote QueueManager use the same xmit queue
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 61
MQSeries Basics - How MQ Series Works
Steps…5. Transmission is done via channels.
i. Channels can be started automatically or manually
ii. To start a channel automatically• To start a channel automatically, the xmit queue must be associated with a channel
initiation queue, so that Queue Manager puts a init message to Channel initiation queue
• Channel initiator Queue is monitored by Channel Initiator• Channel Initiator starts the MCA for particular channel
iii. MCA moves the message over the network to the other machine, using the sender part of the message channel pair.
iv. Listener program monitors a specified port (default port is 1414)
v. On Message arrival at the port Listener start MCA at receiving end
vi. MCA moves the message into specified local queue using the receiver part of the message channel pair.
Note: Both channel definitions, sender and receiver, must have the same name.
6. Trigger Monitor processes the incoming message
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 62
MQSeries Basics – Communication
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 63
MQSeries Basics – Communication
Each machine has a queue manager installed Each queue manager manages several local queues Messages destined for a remote queue manager are put into a remote
queue. A remote queue is associated with a transmission (xmit) queue, which
is a local queue. Usually, there is one xmit queue for each remote queue manager.
A transmission queue is associated with a message channel. Message Channel transmits messages unidirectionally.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 64
MQSeries Basics – Communication Components
Transmission Queue can be accessed in 2 ways:1. Remote Queue definitions
2. Direct Addressing in application• Queue Name
• Queue Manager Name
• No location transparency
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 65
MQSeries Basics – Two Way Communication Components
A remote queue definition that mirrors the local queue in the receiver machine and links to a transmission queue (Q1 in system A and Q2 in system B).
A transmission queue that holds all messages destined for the remote system until the channel transmits them (QMB in system A and QMA in system B).
A sender channel that gets messages from the xmit queue and transmits them to the other system using the existing network (QMA.QMB in system A and QMB.QM.A in system B).
A receiver channel that receives messages and puts them into a local queue (QMB.QMA in system A and QMA.QMB in system B); receiver channels can be started automatically by the queue manager when Channel Auto Definition (CHAD) is enabled.
A local queue from which the program gets its messages (Q2 in system A and Q1 in system B).
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 66
MQSeries Basics – Channel Concepts
Channels are unidirectional Two-way communication requires two channels Each Channel need to be defined twice
Once in the system that sends the message (sender channel) and once in the system that receives the message (receiver channel).
Each channel pair (sender and receiver) must have the same name.
MQSeries Cluster support has special channel types
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 67
MQSeries Basics – Communication Sample
System A (QMA) System B (QMB)
DEFINE QREMOTE(Q1) +RNAME(Q1) RQMNAME(QMB) +XMITQ(QMB) DEFINE QLOCAL(Q1)
DEFINE QLOCAL(QMB) +USAGE(xmitq)
DEFINE CHANNEL(QMA.QMB) +CHLTYPE(sdr) +XMITQ(QMB) +TRPTYPE(tcp) +CONNAME(9.24.104.123)
DEFINE CHANNEL(QMA.QMB) +CHLTYPE(rcvr) +TRPTYPE(tcp)
DEFINE QLOCAL(Q2)
DEFINE QREMOTE(Q2) +RNAME(Q2) RQMNAME(QMA) +XMITQ(QMA)
DEFINE QLOCAL(QMA) +
USAGE(xmitq)
DEFINE CHANNEL(QMB.QMA) +CHLTYPE(rcvr) +TRPTYPE(tcp)
DEFINE CHANNEL(QMB.QMA) +CHLTYPE(sdr) +XMITQ(QMA) +TRPTYPE(tcp) CONNAME(ABC1)
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 68
MQSeries Basics – Communication Sample
Start Communication Manually The commands to start listener and channel for queue manager QMA
are:strmqm QMAstart runmqlsr -t tcp -m QMA -p 1414runmqscstart channel (QMA.QMB)End
With the first command you start queue manager QMA, if not already started.
The next command starts the listener. Itlistens on behalf of QMA on port 1414. As transmission protocol TCP/IP is used.
The third command starts runmqsc in interactive mode. The channel QMA.QMB is started under control of runmqsc.
For the other queue manager you issue equivalent commands.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 69
MQSeries Basics – Communication Sample
Start Communication Automatically Use the channel initiator to start channels.
start runmqlsr -t tcp -m QMA -p 1414 start runmqchi
• With the first command you start the listener
• With the second the channel initiator program.
For the other queue manager you issue equivalent commands.
Test the sample
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 70
MQ Series – Channel Initiator
Channel Initiator: A channel initiator acts as a trigger monitor for sender channels, because a
transmission queue may be defined as a triggered queue. When a message arrives on a transmission queue that satisfies the
triggering criteria for that queue, a message is sent to the initiation queue, triggering the channel initiator to start the appropriate sender channel.
You can also start server channels in this way if you specified the connection name of the partner in the channel definition.
This means that channels can be started automatically, based upon messages arriving on the appropriate transmission queue.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 71
MQSeries Basics – Exits
An exit is a piece of code that is executed by the queue manager upon certain events. Those events include the placing of a message on a queue, the starting of a communications channel, the exchange of security information with a remote queue manager. Many exits are provided as add-on “SupportPacs”.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 72
MQSeries Basics – Undelivered Messages
What happens to the undelivered Messages: Message-retry
• If the MCA is unable to put a message to the target queue for a reason that could be transitory (for example, because the queue is full), the MCA has the option to wait and retry the operation later.
• Specify a message-retry time and interval for MQPUT errors when you define your channel.
Return-to-sender• If message-retry was unsuccessful, or a different type of error was encountered,
the MCA can send the message back to the originator.
• Specify the following options in Message Descriptors• The MQRO_EXCEPTION_WITH_FULL_DATA report option
• The MQRO_DISCARD_MSG report option
• The name of the reply-to queue and reply-to queue manager
Dead-letter queue• If a message cannot be delivered or returned, it is put on to the dead-letter queue
(DLQ).
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 73
MQSeries Basics – Communication between Client and Server
Define a Client/Server Connection Clients and servers are connected with MQI channels. MQI channel consists of a sender/receiver pair, called Client Connection
(CLNTCONN) and Server Connection (SVCONN) channel Client connection channel is defined as an environment variable set MQSERVER=CHAN1/TCP/9.24.104.206(1414) where:
• MQSERVER is the name of the environment variable.
• CHAN1 is the name of the channel to be used for communication between client and server. This channel is defined in the server. MQSeries will automatically create it should it not exist.
• TCP denotes that TCP/IP is to be used to connect to the machine with the address following the parameter.
• 1414 is the default port number for MQSeries.
Definition of the server is as follows:
DEFINE CHANNEL(’CHAN1’) CHLTYPE(SVRCONN) REPLACE TRPTYPE(TCP) MCAUSER(’ ’)
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 74
MQSeries Basics – Triggering
Why Triggering? Some WebSphere MQ applications that serve queues run continuously, so
they are always available to retrieve messages that arrive on the queues. This may not be desirable when the number of messages arriving on the
queues is unpredictable. In this case, applications could be consuming system resources even when
there are no messages to retrieve.
What is MQ Triggering? WebSphere MQ provides a facility that enables an application to be started
automatically when there are messages available to retrieve. This facility is known as triggering.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 75
MQSeries Basics – Triggering
Triggering allows Instantiation as required Conservation of system resources Automation of flow
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 76
MQSeries Basics – Triggering Components
Application queue Local queue, which, when it has triggering set on and when the conditions
are met, requires that trigger messages are written
Process Definition Holds details of the application that will get messages from the application
queue.
Transmission queue Required only when it is required to trigger a channel
Trigger Events Event that causes a trigger message to be generated by the queue
manager. Supported Triggering conditions are
• When the first message is put into the queue
• Every time a message is put into the queue
• When the queue contains a specified number of messages
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 77
MQSeries Basics – Triggering Components
Trigger message Message created by Queue Manager when it recognize an triggering event. Trigger messages have a fixed format Trigger messages contains information about
• Application to be started
• Application Queue
• Process Definition
Initiation queue Local queue on which the queue manager puts trigger messages. Queue manager can own more than one initiation queue
Trigger monitor Continuously-running program that serves one or more initiation queues When a trigger message arrives on an initiation queue, the trigger monitor retrieves
the message. The trigger monitor uses the information in the trigger message. It issues a command to start the application that is to retrieve the messages arriving
on the application queue, passing it information contained in the trigger message header, which includes the name of the application queue.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 78
MQSeries Basics - Flow of application and trigger messages
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 79
MQSeries Basics - Flow of application and trigger messages
1. Application A, puts a message on the application queue for which triggering is enabled.
2. The queue manager checks to see if the conditions are met under which it has to generate a trigger event. They are, and a trigger event is generated. Information held within the associated process definition object is used when creating the trigger message.
3. The queue manager creates a trigger message and puts it on the initiation queue associated with this application queue, but only if an application (trigger monitor) has the initiation queue open for input.
4. The trigger monitor retrieves the trigger message from the initiation queue.
5. The trigger monitor issues a command to start application B
6. Application B opens the application queue and retrieves the message.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 80
MQSeries Basics – Triggering Definitions
Create an initiation queue for your application queue. For example:DEFINE QLOCAL (initiation.queue) REPLACE +
LIKE (SYSTEM.DEFAULT.LOCAL.QUEUE) +
DESCR ('initiation queue description' )
The target queue must have “triggering” specified DEFINE QLOCAL (application.queue) REPLACE +
LIKE (SYSTEM.DEFAULT.LOCAL.QUEUE) +
DESCR ('appl queue description') +
INITQ ('initiation.queue') +
PROCESS ('process.name') +
TRIGGER +
TRIGTYPE (FIRST)
Process definition associated with the target queue can be this:DEFINE PROCESS(process.name) REPLACE +
DESCR(’Process to start server program’) +
APPLTYPE(WINDOWSNT) +
APPPLICID(‘c:\test\myprog.exe’)
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 81
MQSeries Basics – Trigger Types
EVERY
Every time a message is put in the target queue a trigger message is also put in the initiation queue. Use this when your program exits after processing one message or transaction, as shown above on the left.
FIRST
A trigger message is put in the initiation queue only when the target queue has been empty. Use this when the program exits only then when there are no more messages in the queue, as shown on the right.
n messages
A trigger message is put in the initiation queue when there are n messages in the target queue. For example, you can start a batch program when the queue holds 1000 messages.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 82
MQ Series Clusters
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 83
MQ Cluster
Clusters are groups of machines connected together. Even multiple queue managers on a single processor can build or be
part of a cluster.
Why Clustering Process very high message rates.
Process very large numbers of messages.
Single system to control and administer.
System should always be available.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 84
Cluster components
Cluster queue manager - Queue Manager that is a member of a cluster. Queue Managers can be part of multiple clusters (Overlapping clusters).
Cluster queue - Queue that is hosted by a cluster queue manager and made available to other queue managers in the cluster
Repository - A repository is a collection of information about the queue managers that are members of a cluster
Repository contains information such as queue manager names, their locations, their channels, which queues they host, etc.
Information is stored in form of messages in the a Queue called SYSTEM.CLUSTER.REPOSITORY.QUEUE
Typically, two queue managers in a cluster hold a full repository. The remaining queue managers all hold a partial repository.
Full repository - Queue manager that hosts a complete set of information about every queue manager in the cluster is referred to as having a full repository for the cluster.
Partial repository - Queue manager's partial repository contains information about only those queue managers with which the queue manager needs to exchange messages.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 85
Cluster components
Cluster-receiver channel - similar to a receiver channel used in distributed queuing, but in addition to carrying messages this channel can also carry information about the cluster.
Cluster-sender channel - similar to a sender channel used in distributed queuing, but in addition to carrying messages this channel can also carry information about the cluster.
Cluster transmission queue - transmits all messages from the queue manager to any other queue manager that is in the same cluster.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 86
MQ Clusters
Reasons for reduced MQSeries administration: Communication channels are created automatically. Queues are administered cluster-wide. No need to define explicit remote queues. Only one cluster transmission queue is required to send messages to
all queue managers in the cluster, and this queue is created automatically.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 87
Distributed Queues & Clustering Comparison
Four Queue Managers, each with two local queues
Definitions for distributed queuing
Description Number per QM
Total number
A sender-channel definition for a channel on which to send messages to every other queue manager
3 12
A receiver-channel definition for a channel on which to receive messages from every other queue manager
3 12
A transmission-queue definition for a transmission queue to every other queue manager
3 12
A local-queue definition for each local queue 2 8
A remote-queue definition for each remote queue to which this queue manager wants to put messages
6 24
Optionally, on z/OS, a process definition specifying trigger data if channels are to be triggered
3 12
Definitions for clustering
DescriptionNumber per QM
Total number
A cluster-sender channel definition for a channel on which to send messages to a repository queue manager
1 4
A cluster-receiver channel definition for a channel on which to receive messages from other queue managers in the cluster
1 4
A local-queue definition for each local queue
2 8
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 88
Creating MQ Clusters
Default Objects associated with Clusters SYSTEM.CLUSTER.REPOSITORY.QUEUE
• This local queue is used to store all the repository information.
SYSTEM.CLUSTER.COMMAND.QUEUE• This local queue is used to carry messages to the repository.
SYSTEM.CLUSTER.TRANSMIT.QUEUE• This is the transmission queue for all messages to all queues and queue
managers that are within clusters.
SYSTEM.DEF.CLUSSDR• This is used to supply default values for any attributes that you do not specify
when you create a cluster sender channel on a queue manager in the cluster.
SYSTEM.DEF.CLUSRCVR• This is used to supply default values for any attributes that you do not specify
when you create a cluster-receiver channel on a queue manager in the cluster.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 89
Creating MQ Clusters
Create a cluster with four queue managers using scripts
Create following Queue managers
QM Name TCP/IP Port
QM_1 1415
QM_2 1416
QM_3 1417
QM_4 1418
Create and Configure QM_1 & QM_3 first since they are full repositories.
mqcluster_demo.zip
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 90
MQSeries Basics – Samples & Exercises
Sample MQ Put
Sample MQ Get
MQ Browse
MQPut.java
MQGet.java
mqbrowse.java
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 91
Websphere MQ Series – Transaction Coordination
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 92
MQSeries Basics - Transaction Coordination
MQSeries is already an XA-compliant resource manager. Can participate in a two-phase commit coordinated by an XA transaction
manager Queue Manager is XA resource manager and an XA resource coordinator.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 93
MQSeries Basics - Transaction Coordination - Basics
Commit and Backout: When a program puts a message on a queue within a unit of work, that
message is made visible to other programs only when the program commits the unit of work. To commit a unit of work, all updates must be successful to preserve data integrity. If the program detects an error and decides that the put operation should not be made permanent, it can back out the unit of work. When a program performs a back out, WebSphere MQ restores the queue by removing the messages that were put on the queue by that unit of work. The way in which the program performs the commit and back out operations depends on the environment in which the program is running.
Syncpoint coordination, syncpoint, unit of work Syncpoint coordination is the process by which units of work are either
committed or backed out with data integrity.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 94
MQSeries Basics - Transaction Coordination
Unit of Work Global unit of work – Two Phase Commit
• Includes both database and queue updates
• Can update both queues and databases and need to be started with MQBEGIN.
• After the MQBEGIN call has been issued, local resources can be updated using MQGET, MQPUT or MQPUT1 calls made under syncpoint.
• The unit of work can be committed by an MQCMIT or an MQDISC call.
• Unit of work will be rolled back by an MQBACK call, or if the application terminates without first disconnecting.
Local unit of work - Single Phase Commit• Consists of queue updates only.
• Is started by an MQGET, MQPUT or MQPUT1 call which specifies the corresponding syncpoint option.
• Units of work can be committed explicitly by an MQCMIT call, or implicitly by an MQDISC call.
• Units of work can be rolled back explicitly by an MQBACK call, or implicitly if the application terminates without first disconnecting.
Global Unit of WorkMQCONN()….MQBEGIN()…..MQGET(MQGMO_SYNCPOINT)...EXEC SQL UPDATE...MQCMIT()MQDISC()
Local Unit of WorkMQCONN()...MQGET(MQGMO_SYNCPOINT)...MQCMIT()...MQPUT(MQPMO_SYNCPOINT)...MQDISC
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 95
MQSeries Basics - Transaction Coordination
Mixing Units of Work• Application can consist of both local and global units of work.• Units of work that make only queue manager updates can be started using an MQGET,
MQPUT or MQPUT1 call specifying the appropriate syncpoint option.
• Units of work that also need to update global resources owned by a database manager need to be started using an MQBEGIN call.
Valid Invalid Invalid
MQCONN ()
MQPUT (MQPMO_SYNCPOINT)
MQCMIT ()
......
MQBEGIN ()
MQGET (MQGMO_SYNCPOINT)
EXEC SQL UPDATE
MQCMIT ()
MQCONN ()
MQPUT (MQPMO_SYNCPOINT)
...
MQBEGIN ()
MQCONN ()
...
MQBEGIN ()
...
MQBEGIN ()
It is an error to try to start another unit of work while there is another already in progress.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 96
MQSeries Basics - Transaction Coordination
Local Unit of Work Sample
mqsync.java
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 97
MQSeries Basics – Message Segmenting and Grouping
Definitions Physical Message
The basic unit that appears in a queue.
Logical Message A single unit of application information.
Message Segment Part of a logical message.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 98
Websphere Message Segmenting and Grouping
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 99
MQSeries Basics – Message Segmenting and Grouping
Segmentation MQSeries (Queue Manager) or Application programmer can segment the
message Receiving application has the option to either receive the entire message in
one piece or each segment separately. Several physical messages build one logical message MQSeries (Queue Manager) ensures that the order of segments is
maintained. Used in situations where the Message length is more than that is allowed. Simplest way is to let MQSeries do the segmentation for you.
• The sending application has to inform the queue manager that it can segment the message if necessary, and the receiving application has to tell the queue manager to reassemble the message if it has been segmented.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 100
MQSeries Basics – Message Segmenting and Grouping
To allow the queue manager to perform segmenting on its own when the message exceeds MAXMSGL, specify in the message descriptor:
• md.Version = MQMD_VERSION_2;
• md.MsgFlags = MQMF_SEGMENTATION_ALLOWED;
The receiving application can get each message segment individually or have the queue manager reassemble the message and return it to the program after all segments have been received.
The queue manager makes sure that the segments are in the correct order.• gmo.OPTIONS += MQGMO_COMPLETE_MSG;
Note: If not specified segments are returned individually.
Exercise: Arbitrary Segmentation Take a message which is too big (either for the queue manager or for the specific
queue) And automatically segment it
• MQSeries breaks the message into smaller messages (called segments) and sends them wherever the “too large” message would have been sent.
• The local queue manager at the receiving end puts the pieces back together and is able to present the “too large” message to the GETting application as though it had never been segmented.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 101
MQSeries Basics – Message Segmenting and Grouping
Grouping Group several small logical
messages together and build one larger physical message
It is possible for a message that is segmented to be part of a group of messages.
Receiving application disassembles the message received.
Facilitates reduced network traffic. Messages in a group will all have
the same Group ID which can be controlled by the application or generated by the queue manager.
The logical sequence of messages in a group is maintained using the message sequence number field in the message header.
© 2004 Cognizant Technology Solutions. Proprietary and confidential. Page 102
THANK YOU
Recommended