QCon 2014San Francisco

QCon Conferenceとは

QCon San Francisco 2014は今回で8回目を迎える、チームリード、アーキテクト、プロジェクトマネージャーのために開催されるエンジニアによるエンジニアのためのカンファレンス。100名以上の講演者が登壇し、15のトラックが行われ、オープンスペースなどさまざまな人脈作りの機会を提供する。

なぜ参加したか• Google, Twitter, Facebook, Netflixなど様々なWEB企業が参加し、

WEB技術における最前線をキャッチアップするため。

•海外における技術へのスタンス、カルチャーを理解し組織のあり方を学ぶため

•日本でも多く開催されているカンファレンス(QCon Tokyoなど)との情報格差を確認するため

Tracks Overview

Managing Microservices @ Scale無数のきめ細かなWebサービスを持つ企業(たとえばNetflix、LinkedIn)はその複雑性をどのように管理しているのか

The Future of Mobileモバイルとパフォーマンス改善の将来

Java at the Cutting EdgeJavaエコシステムの最先端

Continuous Delivery継続的デリバリーの哲学、カルチャー、課題、ベストプラクティス

Engineering Culture エンジニアリングカルチャー作りの成功と失敗

Applied Machine Learning and Data Scienceあなたの巨大なビッグデータを理解する

Unleashing the Power of Streaming Dataビジネスおよびエンジニアリング問題を解決するため、企業はどのように新しいデータストリーム処理パラダイムを活用しているか

Next gen HTML5 and JSWebは変化している、次に来るのは何か？

Containers, Dockers, Puppets大規模デプロイメント現場におけるデプロイメント自動化

Measuring Engineering and Product Success進捗を計測、トラックするための戦略

Real World Functional関数プログラミングのコンセプトを現場に活かす

Modern Computer Science In the Real World現在の問題に立ち向かうために、最新のコンピュータサイエンスが役に立てること

Reactive Service Architectureリアクティブ、レスポンシブ、フォールトトレラントなど

The Hottest Tech in the World of Big Dataビッグデータ技術領域における最新情報

Architectures You've Always Wondered about最新、最大のインターネットアーキテクチャ

雰囲気

9:00 - 18:00まで各セッションが行われる

朝食・昼食付き

ブレイクタイム

夜に親睦会あり〼

Keynote

Bruce SchneierSecurity Guru

Bruce Schneier is a cryptographer, computer security and privacy specialist, and writer. Called a "security guru" by The Economist, he is the author of 12 books as well as the famous ‘Schneier on Security’ blog. He keynoted at QCon NY weeks after the Snowden leak about prism in 2013 and was one of the top rated keynotes. @schneierblog

Leslie LamportTuring Award Winner of Paxos and LaTeX

2013 Turing Award Winner for "fundamental contributions to the theory and practice of distributed and concurrent systems,” much of the infrastructure implemented in places like Amazon.com is based on his research.

Melody MeckfesselGoogle Engineering Director

Melody Meckfessel leads a team of 200+ engineers responsible for Developer Infrastructure for Google and its Cloud platform. Her team builds highly scalable systems, from IDEs and automated test systems to deployment through Google’s massive production environment. She is a hands-on technical leader with 18 years of experience building and maintaining large-scale distributed systems.

•畑史彦 Functional programming & Reactive System

•伊藤和宏 Starting Apache Spark training

•渡邊雄作 MicroService Cutting Edge

MicroService Cutting Edge

渡邉雄作

Speaker: Paul OsmanDirector of Platform Engineering at 500px

Paul Osman is the Director of Platform Engineering at 500px, the world’s premier photography community. He and his team concentrate on building a scalable and maintainable infrastructure that powers a growing community of photography enthusiasts. Prior to 500px, Paul worked on the Platform team at SoundCloud. His primary interests are distributed systems, APIs and scalable teams. @paulosman Find Paul Osman at

Organizing your company to embrace Microservices

Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organisations

:Melvin Conway

システムを設計する組織は、その構造をそっくりまねた構造の設計を生み出してしまう

1. コードベース バージョン管理されている1つのコードベースと複数のデプロイ 2. 依存関係 依存関係を明示的に宣言し分離する 3. 設定 設定を環境変数に格納する

http://twelve-factor-ja.herokuapp.com/(日本語訳)

Herokuの中の人 Adam Wiggins氏によって書かれた、モダンでスケーラブルかつ継続的デプロイ可能なWebアプリケーションを構築するための方法論

Lesson 1DevOps

Lesson 2Service Design

Lesson 3Teams

Scalable Microservices at Netflix. Challenges and Tools of the Trade

Speaker: Sudhir TonseResponsible for Netflix Cloud Platform

Sudhir Tonse manages the Cloud Platform Infrastructure team at Netflix and is responsible for many of the services and components that form the Netflix Cloud Platform as a Service. Many of these components have been open sourced under the NetflixOSS umbrella. Open source contribution includes Archaius: a dynamic configuration/properties management library, Ribbon: an Inter Process Communications framework that includes Cloud friendly Software load balancers, Karyon: the nucleus of a PaaS service etc. Prior to Netflix, Sudhir was an Architect at Netscape/AOL delivering large-scale consumer and enterprise applications in the area of Personalization, Infrastructure and Advertising Solutions. Sudhir is a weekend golfer and tries to make the most of the wonderful California weather and public courses. @stonse Find Sudhir Tonse at

https://twitter.com/stonse http://www.linkedin.com/in/sudhirtonse

Monolithic App

デメリット •Large Codebase •Many Components, no clear ownership •Long deployment cycles

メリット •Single codebase

Easy to develop/debug/deploy Good IDE support

•Easy to scale horizontally •A Central Ops team can efficiently handle

User Accounts

Shopping Cart

Product Catalog

Customer Service

Monolithic Apps – Failure & Availability

MicroServices

You Think??

•Many smaller (fine grained), clearly scoped services Single Responsibility Principle Domain Driven Development Bounded Context Independently Managed

•Clear ownership for each service Typically need/adopt the “DevOps” model

MicroServiceの特徴

Faster and simpler deployments and rollbacks Independent Speed of Delivery (by different teams)

Right framework/tool/language for each domain Recommendation component using Python?, Catalog Service in Java ..

Greater Resiliency Fault Isolation

Better Availability If architected right …

なぜMicroServiceなのか？

MicroServices – separate single purpose services

Load Balancer

Account Service Catalog Service

Recommendation Service

Customer Service Service

Catalog DB

API Gateway

Customer DB

Microservices Architecture

Can lead to chaos if not designed right …

•Service Discovery

•Operational Overhead (100s of services; DevOps model absolutely required)

•Distributed Systems are inherently Complex •N/W Latency, Fault Tolerance, Serialization overhead ..

•Service Interface Versioning, Mismatches?

•Testing (Need the entire ecosystem to test)

•Fan out of Requests -> Increases n/w traffic

数々の挑戦をしてきた

Dependency Visualization

Homogeneity in A Polyglot Ecosystem

• Eureka – for Service Registry/Discovery • Karyon – for Server (Reactive or threaded/servlet container based) • Ribbon – for IPC Client

And Fault Tolerant Smart LoadBalancer • Hystrix – for Fault Tolerance and Resiliency • Archaius – for distributed/dynamic Properties • Servo – unified Feature rich Metrics/Insight • EVCache – for distributed cache • Curator/Exhibitor – for zookeeper based operations

MicroServiceにおいて大事なこと

•DevOps•Design•Team•Culture

QCon SF 2014 まとめ

キーワード •MicroService •Reactive Architecture •Functional Programming •Streaming Data