1CONFIDENTIAL

MICROSERVICES

ARCHITECTURE OVERVIEW

DZMITRY SKAREDAU, SOLUTION ARCHITECT

MAY 17, 2016

2CONFIDENTIAL 2

Follow my Microservices Series

• Microservices architecture overview

• https://epa.ms/TechTalkMicroservices (Oct 08/Oct 16)

• Spring Cloud + Netflix OSS overview

• https://epa.ms/TechTalkSpringCloud (Mar 30)

• Microservices and Domain Driven Design

• Cloud Native applications: closer look

• CI/CD for Microservices using Docker and Kubernetes

• Spring Cloud workshop

ABOUT

Dzmitry SkaredauSolution Architect

@dskaredov

• ~15 years in software development

• Java Competency Center Expert

https://epa.ms/SkillsMatrix

https://epa.ms/RnDSaaS

https://epa.ms/JavaTechTalks (KB)

https://epa.ms/MinskTechTalks (Yammer)

https://epa.ms/Microservices (Yammer)

https://epa.ms/JavaRadar

3CONFIDENTIAL 3

• Why do we need it

• Architecture patterns

AGENDA

• Microservice

• API Gateway

• Service Discovery

• Stateless/Shared-Nothing

• Configuration/Service Consumption

• Fault Tolerance

• Request Collapsing

• API Versioning

• Decomposition Recipes

• Q/A

4CONFIDENTIAL

WHY DO WE NEED IT

5CONFIDENTIAL 5

REALITYBECAUSE OF

6CONFIDENTIAL 6

Delivering world-class applications requires an agile, multidimensional

approach to architecture. It’s increasingly obvious that the old, linear,

three-tier architecture model is obsolete.

BECAUSE OF

“„

Gartner Application Architecture, Development & Integration Summit 2014

7CONFIDENTIAL 7

Traditional application architectures obsolete, and digital business

demands more agility than ever.

BECAUSE OF

“ „Anne Thomas, VP Distinguished Analyst

13 years at Gartner

36 years industry experience

8CONFIDENTIAL 8

Users expect millisecond response times and close to 100% uptime.

And by «user» I mean both humans and machines. Traditional

architectures, tools and products such simply won’t cut it anymore.

We can’t make the horse faster anymore, we need cars for where we

are going.

BECAUSE OF

“„

Jonas Bonér

Founder & CTO of Lightbend (former Typesafe)

9CONFIDENTIAL 9

MICROSERVICES VS MONOLITH

Simple code base

Modularity with exact borders

Change circles decoupled

Efficient scaling

Newcomers adopting faster

Per service(s) team responsibility

No technology lock

MONOLITH MICROSERVICES

Complex code base

Hard to maintain modularity

Change circles tightly coupled

Inefficient scaling

Scaring for newcomers

Hard to scale development team

Tied to chose technology

10CONFIDENTIAL 10

MICROSERVICES VALUES

Microservices Values

Continues Delivery

principles

Bounded Context

Team and Release Cycle independence

System resilience

Technology agnostic

Independent scaling

11CONFIDENTIAL 11

MICROSERVICES VALUES VS COMPLEXITY

Team autonomy

Time to market

Scaling

Componentization

Technology variation

Cross teams communication

Continues Deployment

Fault tolerance

Versioning

Maintenance

VALUES COMPLEXITY

12CONFIDENTIAL

ARCHITECTURE PATTERNS

13CONFIDENTIAL 13

ARCHITECTURE PATTERNS

• Microservice

• API Gateway

• Service Discovery

• Stateless/Shared-Nothing

• Configuration Management

• Fault Tolerance

• Request Collapsing

14CONFIDENTIAL

MICROSERVICE

15CONFIDENTIAL 15

NOT ABOUT SIZE, BUT RESPONSIBILITY

Mario Fusco

RedHatter, Drools developer, Java Champion, Milano Jug coordinator, speaker, co-author of Java 8 in Action. A functional mind hosted in an object oriented brain

16CONFIDENTIAL 16

BOUNDED CONTEXT

Bounded Context is a

central pattern in

Domain-Driven

Design. It is the

focus of DDD's

strategic design

section which is all

about dealing with

large models and

teams.

17CONFIDENTIAL 17

SIZE OF MICROSERVICE

2 pizza size team

Ideal Size 7 +/-2 persons

18CONFIDENTIAL 18

DECENTRALIZED DATA MANAGEMENT

Microservices prefer letting each service

manage its own database, either different

instances of the same database technology,

or entirely different database systems - an

approach called Polyglot Persistence.

You can use polyglot persistence in a

monolith, but it appears more frequently

with microservices.

19CONFIDENTIAL 19

SHARED DATABASE

20CONFIDENTIAL 20

SHARED DATABASE

21CONFIDENTIAL 21

WHAT ABOUT DISTRIBUTED TRANSACTIONS?

In general, application developers simply do not implement

large scalable applications assuming distributed transactions.

Transactions are fine within the individual microservice

“ „Pat Helland, Software Architect

SQL Architecture Team @ Microsoft

Bing @ Microsoft

Cloud based multi-tenanted file system and database technology @ salesforce.com

22CONFIDENTIAL 22

I NEED A “DISTRIBUTED TRANSACTIONS” STILL

Implement the Saga pattern and break the service interaction (the

business process) into multiple smaller business actions and

counteractions. Coordinate the conversation and manage it based on

messages and timeouts.

How can you reach distributed consensus between services without

transactions?

“„

Arnon Rotem-Gal-Oz

Author of the SOA Patterns book

SAGA PATTERN

23CONFIDENTIAL 23

DESIGN FOR FAILURE

Distributed systems are

much complex than

monolith.

When we have more

systems there is more

chances to fail.

If more places when you

can fails then more often

you can deal with failures.

24CONFIDENTIAL 24

GRACEFUL DEGRADATION

An escalator can never break: it

can only become stairs. You should

never see an Escalator Temporarily

Out Of Order sign

25CONFIDENTIAL 25

KEY CONSIDERATION

Before you go into production with a microservices system, you need to ensure

that you have key prerequisites in place

• DevOps Culture

• Rapid Provisioning

• Basic Monitoring / Debugging capabilities

• Rapid Application Deployment

27CONFIDENTIAL 27

MICROSERVICE VS SOA

Martin Fowler

Chief Scientist at ThoughtWorks

Subset of SOA

Zhamak Dehghani

Principal Consultant at ThoughtWorks

Style of SOA

28CONFIDENTIAL

API GATEWAY

29CONFIDENTIAL 29

API GATEWAY

How many

microservices

could be involved

here?

30CONFIDENTIAL 30

API GATEWAY

~9

How many

microservices

could be involved

here?

31CONFIDENTIAL 31

CHATTY NATURE

32CONFIDENTIAL 32

API GATEWAY: REDUCE CHATTINESS

34CONFIDENTIAL 34

API GATEWAY: UNDER THE HOOD

35CONFIDENTIAL

SERVICE DISCOVERY

36CONFIDENTIAL 36

SERVICE DISCOVERY PROBLEM

37CONFIDENTIAL 37

SERVICE DISCOVERY PROBLEM

38CONFIDENTIAL

STATELESS/SHARED-NOTHING

39CONFIDENTIAL 39

STICKY SESSIONS

40CONFIDENTIAL 40

STICKY SESSIONS

41CONFIDENTIAL 41

STATELESS/SHARED-NOTHING

• Store state at the client

• Store state at database

• Distributed session

• Stateless services

42CONFIDENTIAL

CONFIGURATION MANAGEMENT

43CONFIDENTIAL 43

SIMPLE CONFIGURATION

http://12factor.net

http://12factor.net/config

The twelve-factor app stores config in environment variables (often shortened to env vars or env). Env

vars are easy to change between deploys without changing any code; unlike config files, there is little

chance of them being checked into the code repo accidentally; and unlike custom config files, or other

config mechanisms such as Java System Properties, they are a language- and OS-agnostic standard.

ENVIRONMENT VARIABLES

44CONFIDENTIAL 44

ADVANCED CONFIGURATION

• Changing logging levels of a running application in order to debug a production issue

• Change the number of threads receiving messages from a message broker

• Report all configuration changes made to a production system to support regulatory audits

• Toggle features on/off in a running application

• Protect secrets (such as passwords) embedded in configuration

COMPLEX CASES

45CONFIDENTIAL 45

ADVANCED CONFIGURATION

• Versioning

• Auditability

• Encryption

• Refresh without restart

NEEDED CAPABILITIES

46CONFIDENTIAL 46

EXTERNALIZE CONFIGURATION

47CONFIDENTIAL 47

EXTERNALIZE CONFIGURATION

48CONFIDENTIAL 48

SUBSCRIBE

49CONFIDENTIAL

FAULT TOLERANCE

50CONFIDENTIAL 50

DISTRIBUTED SYSTEMS

51CONFIDENTIAL 51

POTENTIAL FAILURE

52CONFIDENTIAL 52

POTENTIAL DOWNTIME

Availability % Downtime per year Downtime per month Downtime per week Downtime per day

90% ("one nine") 36.5 days 72 hours 16.8 hours 2.4 hours

95% 18.25 days 36 hours 8.4 hours 1.2 hours

97% 10.96 days 21.6 hours 5.04 hours 43.2 minutes

98% 7.30 days 14.4 hours 3.36 hours 28.8 minutes

99% ("two nines") 3.65 days 7.20 hours 1.68 hours 14.4 minutes

99.5% 1.83 days 3.60 hours 50.4 minutes 7.2 minutes

99.8% 17.52 hours 86.23 minutes 20.16 minutes 2.88 minutes

99.9% ("three nines") 8.76 hours 43.8 minutes 10.1 minutes 1.44 minutes

99.95% 4.38 hours 21.56 minutes 5.04 minutes 43.2 seconds

99.99% ("four nines") 52.56 minutes 4.38 minutes 1.01 minutes 8.66 seconds

99.995% 26.28 minutes 2.16 minutes 30.24 seconds 4.32 seconds

99.999% ("five nines") 5.26 minutes 25.9 seconds 6.05 seconds 864.3 milliseconds

99.9999% ("six nines") 31.5 seconds 2.59 seconds 604.8 milliseconds 86.4 milliseconds

99.99999% ("seven nines") 3.15 seconds 262.97 milliseconds 60.48 milliseconds 8.64 milliseconds

99.999999% ("eight nines") 315.569 milliseconds 26.297 milliseconds 6.048 milliseconds 0.864 milliseconds

99.9999999% ("nine nines") 31.5569 milliseconds 2.6297 milliseconds 0.6048 milliseconds 0.0864 milliseconds

Without taking steps to

ensure fault tolerance,

30 dependencies each

with 99.99% uptime

would result in 2+ hours

downtime/month

(99.99%30 ≈ 99.7%

uptime = 2+ hours in a

month)

http://techblog.netflix.com/2012/02/fault

-tolerance-in-high-volume.html

0.3% means that the one

million request will have

3000 failed

53CONFIDENTIAL 53

CIRCUIT BREAKER

The basic idea behind the circuit breaker

is very simple. You wrap a protected

function call in a circuit breaker object,

which monitors for failures. Once the

failures reach a certain threshold, the

circuit breaker trips, and all further calls

to the circuit breaker return with an

error, without the protected call being

made at all. Usually you'll also want some

kind of monitor alert if the circuit

breaker trips.

54CONFIDENTIAL 54

CIRCUIT BREAKER: UNDER THE HOOD

55CONFIDENTIAL 55

FAULT TOLERANCE: CIRCUIT BREAKER

57CONFIDENTIAL 57

FALLBACK DEGRADATION

Fallback logic scene involving network access, such as cache access.

58CONFIDENTIAL

REQUEST COLLAPSING

59CONFIDENTIAL 59

REQUEST COLLAPSING

In addition to the isolation

benefits and concurrent

execution of dependency

calls we have also need to

leverage the separate threads

to enable request collapsing

(automatic batching) to

increase overall efficiency

and reduce user request

latencies.

Collapse multiple requests into a single execution

based on a time window and optionally a max batch

size.

This allows an object model to have multiple calls to

the command that execute/queue many times in a

short period (milliseconds) and have them all get

batched into a single backend call.

Typically the time window is something like 10ms

give or take.

60CONFIDENTIAL 60

REQUEST COLLAPSING: UNDER THE HOOD

In addition to the isolation

benefits and concurrent

execution of dependency

calls we have also leveraged

the separate threads to

enable request collapsing

(automatic batching) to

increase overall efficiency

and reduce user request

latencies.

Collapse multiple requests into a single execution

based on a time window and optionally a max batch

size.

This allows an object model to have multiple calls to

the command that execute/queue many times in a

short period (milliseconds) and have them all get

batched into a single backend call.

Typically the time window is something like 10ms

give or take.

61CONFIDENTIAL

API VERSIONING

62CONFIDENTIAL 62

API VERSIONING

• Adding authentication

• Adding authorization rules

• Removing a service

• API contract changes

REASONS SOLUTIONS

• URL Versioning

• Media Type Versioning

• Custom header

• Hostname

• Data parameter

63CONFIDENTIAL 63

API VERSIONING

One method for indicating versioning is via the URI, typically via a path prefix:

Twitter: http://api.twitter.com/1.1/

Last.fm: http://ws.audioscrobbler.com/2.0/

Etsy: http://openapi.etsy.com/v2

Some APIs will provide the version via a query string parameter:

Amazon Simple Queue Service: ?VERSION=2011-10-01

URL

64CONFIDENTIAL 64

API VERSIONING

Media type versioning provides the ability to use the same URI for multiple versions of an API, by specifying the version as part of the Accept media type.

The Accept header can provide versioning in two different ways:

• As part of the media type name itself: application/vnd.status.v2+json. In this case, the segment v2 indicates the

request is for version 2. You can provide the version string however you desire.• As a parameter to the media type: application/vnd.status+json; version=2. This option provides more

verbosity, but allows you to specify the same base media type for each version.

Many REST advocates prefer media type versioning as it solves the "one resource, one URI" problem cleanly, and allows

adding versioning support after-the-fact. The primary argument against it is the fact that the version is not visible when

looking at the URI.

MEDIA TYPE

65CONFIDENTIAL 65

API VERSIONING

The above two versioning types are the most common; however, other types exist:

• Custom header. As an example,

• X-API-Version: 2

• GData-Version: 2.0

• X-MS-Version: 2011-08-18

• etc.

• Hostname. Facebook, when migrating from the first API version, switched from the host http://api.facebook.com to

http://graph.facebook.com.

• Data parameter. This could be a query string parameter for GET requests, as noted above, but a content body parameter for

other request methods.

OTHER METHODOLOGIES

67CONFIDENTIAL 67

API VERSIONING

It seems that there are a number of people recommending using Content-Negotiation (the HTTP

“Accept:” header) for API versioning.

However, none of the big public REST APIs I have looked at seem to be using this approach. They almost

exclusively put the API version number in the URI.

68CONFIDENTIAL 68

API VERSIONING

Twitter URI

Atlassian URI

Google Search URI

Github API URI/Media Type in v3

Intention is to remove versioning in favour of

hypermedia – current

application/vnd.github.v3

Azure Custom Header x-ms-version: 2011-08-18

Facebook URI/optional versioning graph.facebook.com/v1.0/me

Bing Maps URI

Netflix URI parameter

http://api.netflix.com/catalog/titles/series/

70023522?v=1.5

69CONFIDENTIAL 69

API VERSIONING

Google data API (youtube/spreadsheets/others)

URI parameter or custom

header “GData-Version: X.0” or “v=X.0”

Flickr No versioning?

Digg URI http://services.digg.com/2.0/comment.bury

Delicious URI https://api.del.icio.us/v1/posts/update

Last FM URI http://ws.audioscrobbler.com/2.0/

LinkedIn URI

http://api.linkedin.com/v1/people/~/connec

tions

Foursquare URI

https://api.foursquare.com/v2/venues/40a55

d80f964a52020f31ee3?oauth_token=XXX&v=YY

YYMMDD

70CONFIDENTIAL 70

API VERSIONING

paypal parameter &VERSION=XX.0

Twitpic URI http://api.twitpic.com/2/upload.format

Etsy URI http://openapi.etsy.com/v2

Tropo URI https://api.tropo.com/1.0/sessions

Tumblr URI api.tumblr.com/v2/user/

openstreetmap URI and response body http://server/api/0.6/changeset/create

Ebay URI (I think)

http://open.api.ebay.com/shopping?version=

713

71CONFIDENTIAL 71

API VERSIONING

Wikipedia no versioning I think?

Bitly URI https://api-ssl.bitly.com/v3/shorten

Disqus URI

https://disqus.com/api/3.0/posts/remove.js

on

Yammer URI /api/v1

Drop Box URI

https://api.dropbox.com/1/oauth/request_to

ken

Amazon Simple Queue Service (Soap) URI Parameter and WSDL URI &Version=2011-10-01

72CONFIDENTIAL 72

AVOID BREAKING CHANGES

Darrel Miller

API Evangelist at Microsoft. Web APIs, Hypermedia APIs, REST based LOB systems are my area of expertise

73CONFIDENTIAL 73

SELF-DESCRIBE PROTOCOLS

Roy T. Fielding

Senior Principal Scientist at Adobe, co-founded Apache, authored the REST architectural style and Web standards for URI, HTTP/1.x, and URI Templates

74CONFIDENTIAL 74

HAL / HATEOAS

75CONFIDENTIAL 75

BEFORE HATEOAS

HTTP/1.1 200 OK

Content-Type: application/json

Content-Length:

{

"account": {

"account_number": "9999",

"balance": {

"currency": "usd",

"balance": "1100.00"

}

}

}

76CONFIDENTIAL 76

AFTER HATEOAS

HTTP/1.1 200 OK

Content-Type: application/json

Content-Length:

{

"account": {

"account_number": "9999",

"balance": {

"currency": "usd",

"balance": "1100.00"

},

"links": [

{

"rel": "deposit",

"href": “/v1/account/9999/deposit"

},

{

"rel": "withdraw",

"href": "/v2/account/9999/withdraw"

},

{

"rel": "transfer",

"href": "/v1/account/9999/transfer"

},

{

"rel": "close",

"href": "/v1/account/9999/close"

}

]

}

}

77CONFIDENTIAL

DECOMPOSITION RECIPES

78CONFIDENTIAL 78

#1: IF IT WORKS DON'T TOUCH IT

79CONFIDENTIAL 79

#2: START NEW FEATURES AS MICROSERVICES

...the team decided that the best approach to deal with the

architecture changes would not be to split the Mothership immediately,

but rather to not add anything new to it. All of our new features were

built as microservices...

“„

Phil Calcado, Director of Engineering, Core, SoundCloud

80CONFIDENTIAL 80

#3: CREATE ANTI-CORRUPTION LAYER

81CONFIDENTIAL 81

#4: STRANGLING THE MONOLITH

... gradually create a new system around the edges of

the old, letting it grow slowly over several years until

the old system is strangled.

“ „Martin Fowler

82CONFIDENTIAL 82

#5: DECOMPOSITION CANDIDATES

Start from bounded contexts identified within the monolith

OR

Identify the areas of the monolith’s code that will need to change in order to

deliver the changed requirements, and then extract the appropriate bounded

contexts before making the desired changes.

83CONFIDENTIAL 83

WHEN TO STOP?

The monolith has been

completely strangled

Cost of additional service

extraction exceeds the

return on the necessary

efforts

86CONFIDENTIAL

QUESTIONS?