Embed Size (px)
DESCRIPTION
Citation preview
中间件基础服务团队 鼬神Von Gosling@web world
MOM those things
Message-Oriented Middleware
Agenda
•Part 1 MOM intro. MOM spec.•Part 2 MOM core tech. MOM advanced features
. Part 3 MOM geomancy References
Part 1
MOM intro.I. whyII. whatIII. when
MOM spec.IV. JMSV. AMQPVI. STOMPVII. XMPPVIII. MQTT
MOM intro. - why
1970s single protocol connecting mainframe
MOM intro. - why
1980s many protocol connecting mainframe
MOM intro. - why
• not easy since data format , hardware etc.• so adapter coms out, but still face many problems, maintainable?
• so many technologies comes out , such as
RPC(COM/COM+,CORBA,EJB) – sync. call Message Queue – Async. event notification
Also ,many Enterprise messaging killer coming, such as WebSphere MQ, SonicMQ , TIBCO Rendezvous, SUN JCAPS…
MOM intro. - what
Message-oriented middleware (MOM) is
software or hardware infrastructure supporting
sending and receiving messages between
distributed systems. MOM allows application
modules to be distributed over heterogeneous
platforms and reduces the complexity of
developing applications that span multiple
operating systems and network protocols…
-- From Wikipedia
MOM intro. - what
Messaging mediator
MOM intro. - when
Heterogeneous application integration
Decoupled between applications(No RPC)
Event-driven backbone(CEP)
MOM spec. - JMS
•JMS 1.1 released in in 2002 (JMS 1.0.2 spec. since 1998)•JMS 2.0 launched in 2011 as JSR 343•JMS 2.0 released in 2013 (with Java EE 7)
MOM spec - JMS
JMS client (MessageProducer, MessageConsumer , JMSConsumer , JMSProducer)Non-JMS client (Instead of the JMS API , using JMS provider’s native client API)JMS producerJMS consumerJMS providerJMS messageJMS domainsAdministered objects(Connection factory,Destination)
MOM spec - JMS
MessageConsumer glance
MOM spec. - JMS
Non-JMS client - HornetQ
MOM spec - JMS
JMS message
MOM spec. - JMS
MOM spec. - JMS
How to use it ? Selector(SQL 92 subset ,exactly any retrieval tech. you can image)
MOM spec - JMS
Payload
MOM spec. - JMS
JMS domains PK.
MOM spec. - JMS
JMS 1.1 Core API
MOM spec. - JMS
MOM spec. - JMS
Snippet with JMS 1.1 spec.
MOM spec. - JMS
Snippet with JMS 2.0 spec.
MOM spec - JMS
Concurrency ConnectionFactory,Connection,Destination
Session,MessageProducer,MessageConsumer
MOM spec. - AMQP
2012.10 1.0 release
MOM spec. - AMQP
MOM spec. - AMQP
•Types•Transport•Messaging•Transactions•Security
MOM spec. - AMQP
Class Diagram of Communication Endpoints
MOM spec. - AMQP
Version Negotiation examples
MOM spec - AMQP
Protocol frames
MOM spec - AMQP
AMQP Frame Layout
MOM spec - AMQP
AMQP connection state Diagram
MOM spec - AMQP
AMQP message
MOM spec - AMQP
How to parser ？
MOM spec - AMQP
How to use ？
MOM spec. - STOMP
STOMP 1.2 Released on 2012 10STOMP 1.1STOMP 1.0
In a way, It’s seems combing memcached protocol with REST style operation(such as send,subscribe,unsubscribe,begin,commit,abort,ack,nack,disconnect).
Part 2
MOM core tech.I. TransportII. PersistentIII. TransactionIV. Integration
MOM advanced featuresV. HAVI. PerformanceVII. Administering and monitoringVIII. Plugins
MOM core tech. - Transport
AMQ Transport configuration snippets
MOM core tech. - Transport
AMQ log
MOM core tech. - Transport
Hornetq Transport configuration snippets
MOM core tech. - Persistent
RDB storeMemory store File store
Sequential disk access can in some cases be faster than random memory access! – from ACM Queue article
MOM core tech. - Persistent
MOM core tech. - Persistent
MOM core tech. - Persistent
File store structure
MOM core tech. - Persistent
AMQ store structure
MOM core tech. - Transaction
MOM core tech. - Transaction
MOM core tech. - Transaction
2PC , 3PC or Paxos Why not choice?
Complex, Round Trip…
MOM Core Tech. - Transaction
Client DB
Broker DB
①send
half m
essa
ge
②Op
③Com
mit/R
ollb
ack ①store half message
③ ⑤，commit: update message statusrollback： delete message
④perio
dic ch
eck
unco
mm
itted m
essa
ge
⑤Commit/Rollback
Local Tx
Local Tx
MOM core tech. - Integration
Jetty,TomcatHadoopStormESBServiceMix
MOM advanced features- HA
Static DiscoveryDynamic Discovery – ZK/UDP/JGROUP
Master-MasterMaster-Slave
MOM advanced features- HA
Shared store
MOM advanced features- HA
Replication
MOM advanced features- HA
Which one?
MOM advanced features- Performance
Broker performanceClient performance
TCP kernelProtocol
MOM advanced features- Performance
Broker performance
Static scales better than dynamic Asynchronous network connection
establishment Concurrent store and dispatch Store optimization
MOM advanced features- Performance
Client performance setDisableMessageID setDisableMessageTimeStamp Avoid ObjectMessage Avoid AUTO_ACKNOWLEDGE Avoid durable messages Send messages non blocking ProducerWindowSize(sendFailIfNoSp
aceAfterTimeout) Prefetch limit
MOM advanced features- Performance
TCP kernel
MOM advanced features- Performance
Protocol
OpenWire Text-based
No silver bullet...
MOM advanced features- Performance
MOM advanced features- Performance
Performance differs greatly depending on many different factors the network topology transport protocols used quality of service hardware, network, JVM and operating system number of producers, number of consumers distribution of messages across destinations
along with message size
So, No one is absolutely winner!Benchmark testing(SPECjms2007) is a Galileo thing !
MOM advanced features- Administering and monitoring
MetricsMDCJMX-HTTP
MOM advanced features- Plugins
RouterVisualizationStatistics
MOM advanced features- Plugins
Router
MOM advanced features- Plugins
Statistics
MOM advanced features- Plugins
AMQ plugins architecture
MOM advanced features- Plugins
AMQ - How plugins work
Part3
MOM geomancyI. ZMQII. Local features continueIII. Cloud Messaging
References
MOM geomancy - ZMQ
C++ Networking library
ZMQ Pattern:
Request-reply, which connects a set of clients to a set of services. This is a remote procedure call and task distribution pattern.Publish-subscribe, which connects a set of publishers to a set of subscribers. This is a data distribution pattern.Pipeline, which connects nodes in a fan-out/fan-in pattern that can have multiple steps and loops. This is a parallel task distribution and collection pattern.
MOM geomancy - ZMQ
MOM geomancy - ZMQ
MOM geomancy - ZMQ
MOM geomancy - ZMQ
ØMQ’s Built-in Proxy Function code snippet
MOM geomancy - local features continue
Close to latest stable spec. ,such as JMS 2.0,AMQP 1.0,STOMP 1.2 etc.;Promote communication kernel ,such as AIO, Actor pattern etc.;Full stack PUSH model , support web, android, ios etc.;More Cloud features… MQ
(Broker)Broker WebSocket Connection
Service
WebSocket Engine
JMSWebSocket
HTTPServer
Java* Application
JMS Client Runtime
WebSocket Connection Handler
Java API for WebSocket Tyrus Implementation
MOM geomancy - local features continue
Kestrel kernel
MOM geomancy - local features continue
But, if you biased towards jms,How to give your views about jms ?•Join and contribute to [email protected] •Submit ideas to JMS spec issue tracker (or comment on existing ideas)•Contact [email protected]•See jms-spec.java.net for details
MOM geomancy - Cloud Messaging
MOM geomancy - Cloud Messaging
MOM geomancy - References
MOM geomancy - References
•http://en.wikipedia.org/wiki/Message-oriented_middleware
•http://augustl.com/blog/2013/zeromq_instead_of_http/
•http://kafka.apache.org/documentation.html#design
•http://www.oschina.net/translate/client-side-messaging-essentials
•http://www.iron.io/mq
•http://java.dzone.com/articles/guaranteed-messaging-topics
•http://download.progress.com/5331/open/adobe/prc/psc/perf_tuning_a
ctivemq/index.htm
•http://www.cs.cornell.edu/courses/cs614/2007fa/Slides/FLP_and_Paxos.
MOM geomancy - References
•Enterprise Integration with WSO2 ESB
•http://lingo.codehaus.org/
•http://activespace.codehaus.org/
•https://engineering.groupon.com/2013/page/2/
•http
://blog.x-aeon.com/2013/04/10/a-quick-message-queue-benchmark-acti
vemq-rabbitmq-hornetq-qpid-apollo
/
•http://www.slashroot.in/linux-network-tcp-performance-tuning-sysctl
•http://activemq.apache.org/performance.html
•http://
activemq.apache.org/apollo/documentation/openwire-manual.html
•http://snarfed.org/transactions_across_datacenters_io.html
MOM geomancy - References
•JMS Performance Comparison
•amqp-core-complete-v1.0-os.pdf
•HornetQ User Manual.pdf
•ActiveMQ In Action(Manning-2011).pdf
•Building a High Availability and Disaster Recovery Solution using
AlwaysOn Availability Groups.docx
•RocketMQ_design.pdf
•ZeroMQ: Messaging for Many Applications.pdf
•ZeroMQ.pdf
Q & A
你问我答
共创，共建
Questions?
Thanks!
