Mom those things v1

中间件基础服务团队 鼬神Von Gosling@web world

MOM those things

Message-Oriented Middleware

Agenda

•Part 1 MOM intro. MOM spec.•Part 2 MOM core tech. MOM advanced features

. Part 3 MOM geomancy References

Part 1

MOM intro.I. whyII. whatIII. when

MOM spec.IV. JMSV. AMQPVI. STOMPVII. XMPPVIII. MQTT

MOM intro. - why

1970s single protocol connecting mainframe

MOM intro. - why

1980s many protocol connecting mainframe

MOM intro. - why

• not easy since data format , hardware etc.• so adapter coms out, but still face many problems, maintainable?

• so many technologies comes out , such as

RPC(COM/COM+,CORBA,EJB) – sync. call Message Queue – Async. event notification

Also ,many Enterprise messaging killer coming, such as WebSphere MQ, SonicMQ , TIBCO Rendezvous, SUN JCAPS…

MOM intro. - what

Message-oriented middleware (MOM) is

software or hardware infrastructure supporting

sending and receiving messages between

distributed systems. MOM allows application

modules to be distributed over heterogeneous

platforms and reduces the complexity of

developing applications that span multiple

operating systems and network protocols…

-- From Wikipedia

MOM intro. - what

Messaging mediator

MOM intro. - when

Heterogeneous application integration

Decoupled between applications(No RPC)

Event-driven backbone(CEP)

MOM spec. - JMS

•JMS 1.1 released in in 2002 (JMS 1.0.2 spec. since 1998)•JMS 2.0 launched in 2011 as JSR 343•JMS 2.0 released in 2013 (with Java EE 7)

MOM spec - JMS

JMS client (MessageProducer, MessageConsumer , JMSConsumer , JMSProducer)Non-JMS client (Instead of the JMS API , using JMS provider’s native client API)JMS producerJMS consumerJMS providerJMS messageJMS domainsAdministered objects(Connection factory,Destination)

MOM spec - JMS

MessageConsumer glance

MOM spec. - JMS

Non-JMS client - HornetQ

MOM spec - JMS

JMS message

MOM spec. - JMS

MOM spec. - JMS

How to use it ? Selector(SQL 92 subset ,exactly any retrieval tech. you can image)

MOM spec - JMS

Payload

MOM spec. - JMS

JMS domains PK.

MOM spec. - JMS

JMS 1.1 Core API

MOM spec. - JMS

MOM spec. - JMS

Snippet with JMS 1.1 spec.

MOM spec. - JMS

Snippet with JMS 2.0 spec.

MOM spec - JMS

Concurrency ConnectionFactory,Connection,Destination

Session,MessageProducer,MessageConsumer

MOM spec. - AMQP

2012.10 1.0 release

MOM spec. - AMQP

MOM spec. - AMQP

•Types•Transport•Messaging•Transactions•Security

MOM spec. - AMQP

Class Diagram of Communication Endpoints

MOM spec. - AMQP

Version Negotiation examples

MOM spec - AMQP

Protocol frames

MOM spec - AMQP

AMQP Frame Layout

MOM spec - AMQP

AMQP connection state Diagram

MOM spec - AMQP

AMQP message

MOM spec - AMQP

How to parser ？

MOM spec - AMQP

How to use ？

MOM spec. - STOMP

STOMP 1.2 Released on 2012 10STOMP 1.1STOMP 1.0

In a way, It’s seems combing memcached protocol with REST style operation(such as send,subscribe,unsubscribe,begin,commit,abort,ack,nack,disconnect).

Part 2

MOM core tech.I. TransportII. PersistentIII. TransactionIV. Integration

MOM advanced featuresV. HAVI. PerformanceVII. Administering and monitoringVIII. Plugins

MOM core tech. - Transport

AMQ Transport configuration snippets

MOM core tech. - Transport

AMQ log

MOM core tech. - Transport

Hornetq Transport configuration snippets

MOM core tech. - Persistent

RDB storeMemory store File store

Sequential disk access can in some cases be faster than random memory access! – from ACM Queue article

MOM core tech. - Persistent

MOM core tech. - Persistent

MOM core tech. - Persistent

File store structure

MOM core tech. - Persistent

AMQ store structure

MOM core tech. - Transaction

MOM core tech. - Transaction

MOM core tech. - Transaction

2PC , 3PC or Paxos Why not choice?

Complex, Round Trip…

MOM Core Tech. - Transaction

Client DB

Broker DB

①send

half m

essa

ge

②Op

③Com

mit/R

ollb

ack ①store half message

③ ⑤，commit: update message statusrollback： delete message

④perio

dic ch

eck

unco

mm

itted m

essa

ge

⑤Commit/Rollback

Local Tx

Local Tx

MOM core tech. - Integration

Jetty,TomcatHadoopStormESBServiceMix

MOM advanced features- HA

Static DiscoveryDynamic Discovery – ZK/UDP/JGROUP

Master-MasterMaster-Slave

MOM advanced features- HA

Shared store

MOM advanced features- HA

Replication

MOM advanced features- HA

Which one?

MOM advanced features- Performance

Broker performanceClient performance

TCP kernelProtocol

MOM advanced features- Performance

Broker performance

Static scales better than dynamic Asynchronous network connection

establishment Concurrent store and dispatch Store optimization

MOM advanced features- Performance

Client performance setDisableMessageID setDisableMessageTimeStamp Avoid ObjectMessage Avoid AUTO_ACKNOWLEDGE Avoid durable messages Send messages non blocking ProducerWindowSize(sendFailIfNoSp

aceAfterTimeout) Prefetch limit

MOM advanced features- Performance

TCP kernel

MOM advanced features- Performance

Protocol

OpenWire Text-based

No silver bullet...

MOM advanced features- Performance

MOM advanced features- Performance

Performance differs greatly depending on many different factors the network topology transport protocols used quality of service hardware, network, JVM and operating system number of producers, number of consumers distribution of messages across destinations

along with message size

So, No one is absolutely winner!Benchmark testing(SPECjms2007) is a Galileo thing !

MOM advanced features- Administering and monitoring

MetricsMDCJMX-HTTP

MOM advanced features- Plugins

RouterVisualizationStatistics

MOM advanced features- Plugins

Router

MOM advanced features- Plugins

Statistics

MOM advanced features- Plugins

AMQ plugins architecture

MOM advanced features- Plugins

AMQ - How plugins work

Part3

MOM geomancyI. ZMQII. Local features continueIII. Cloud Messaging

References

MOM geomancy - ZMQ

C++ Networking library

ZMQ Pattern:

Request-reply, which connects a set of clients to a set of services. This is a remote procedure call and task distribution pattern.Publish-subscribe, which connects a set of publishers to a set of subscribers. This is a data distribution pattern.Pipeline, which connects nodes in a fan-out/fan-in pattern that can have multiple steps and loops. This is a parallel task distribution and collection pattern.

MOM geomancy - ZMQ

MOM geomancy - ZMQ

MOM geomancy - ZMQ

MOM geomancy - ZMQ

ØMQ’s Built-in Proxy Function code snippet

MOM geomancy - local features continue

Close to latest stable spec. ,such as JMS 2.0,AMQP 1.0,STOMP 1.2 etc.;Promote communication kernel ,such as AIO, Actor pattern etc.;Full stack PUSH model , support web, android, ios etc.;More Cloud features… MQ

(Broker)Broker WebSocket Connection

Service

WebSocket Engine

JMSWebSocket

HTTPServer

Java* Application

JMS Client Runtime

WebSocket Connection Handler

Java API for WebSocket Tyrus Implementation

MOM geomancy - local features continue

Kestrel kernel

MOM geomancy - local features continue

But, if you biased towards jms,How to give your views about jms ?•Join and contribute to users@jms-spec.java.net •Submit ideas to JMS spec issue tracker (or comment on existing ideas)•Contact nigel.deakin@oracle.com•See jms-spec.java.net for details

MOM geomancy - Cloud Messaging

MOM geomancy - Cloud Messaging

MOM geomancy - References

MOM geomancy - References

•http://en.wikipedia.org/wiki/Message-oriented_middleware

•http://augustl.com/blog/2013/zeromq_instead_of_http/

•http://kafka.apache.org/documentation.html#design

•http://www.oschina.net/translate/client-side-messaging-essentials

•http://www.iron.io/mq

•http://java.dzone.com/articles/guaranteed-messaging-topics

•http://download.progress.com/5331/open/adobe/prc/psc/perf_tuning_a

ctivemq/index.htm

•http://www.cs.cornell.edu/courses/cs614/2007fa/Slides/FLP_and_Paxos.

pdf

MOM geomancy - References

•Enterprise Integration with WSO2 ESB

•http://lingo.codehaus.org/

•http://activespace.codehaus.org/

•https://engineering.groupon.com/2013/page/2/

•http

://blog.x-aeon.com/2013/04/10/a-quick-message-queue-benchmark-acti

vemq-rabbitmq-hornetq-qpid-apollo

/

•http://www.slashroot.in/linux-network-tcp-performance-tuning-sysctl

•http://activemq.apache.org/performance.html

•http://

activemq.apache.org/apollo/documentation/openwire-manual.html

•http://snarfed.org/transactions_across_datacenters_io.html

MOM geomancy - References

•JMS Performance Comparison

•amqp-core-complete-v1.0-os.pdf

•HornetQ User Manual.pdf

•ActiveMQ In Action(Manning-2011).pdf

•Building a High Availability and Disaster Recovery Solution using

AlwaysOn Availability Groups.docx

•RocketMQ_design.pdf

•ZeroMQ: Messaging for Many Applications.pdf

•ZeroMQ.pdf

Q & A

你问我答

共创，共建

Questions?

Thanks!