NSWI152 - Vývoj cloudových aplikací

Filip Zavoral, Jaroslav Keznikl, Stanislav Ulrychhttp://ulita.ms.mff.cuni.cz/pub/predn/NSWI152/

Průběh semestru, platformy Průběh semestru

0/2 Z, průběh semestru nepravidelný některé týdny cvičení - 'Guided Tour' v SW2 některé týdny praxe - samostatná práce

Platformy Windows Azure

Filip Zavoral 3 týdny cvičení - instalace, deployment, multi-tier aplikace

Google App Engine Pavel Jančík 3 týdny cvičení

OpenStack Lukáš Korous 2 týdny cvičení

zbytek semestru aplikačka (v libovolné platformě) poslední seminář: předvedení, prezentace, zhodnocení

Znalosti NSWI150 Virtualizace a cloud computing

speciálně část týkající se cloudů http://data.ksi.ms.mff.cuni.cz/svn/NSWI150pub/index.html

C#, ASP.Net, Python běžné věci na základní úrovni, copy-and-paste rozšiřující: NPRG057 Pokročilé programování pro .NET II

Účty Microsoft Account (Windows Live ID)

http://windows.microsoft.com/en-US/windows-8/microsoft-account#1TC=t1 Google Account

Virtuální stroje VMware vSphere Client / Web Client Windows 8.1, VS 2013, .Net 4.5

Prerekvizity a požadavky

Windows Azure

Filip Zavoral

Windows Azure Cloud Components

Execution ModelsVirtual Machines Web Sites Cloud Services

Mobile services Hi-Perf Computing Load Balancing

Cloud Storage SQL Database Key-Value Tables Blobs

Data Processing Map/Reduce Hadoop Zoo Reporting

Networking Virtual Network Connect Traffic Manager

Messaging Service Bus Queues Topics / Relays

Caching Caching Content Delivery

(Multi-)Media Media Services Streaming

Other ServicesGIS / Maps Searching / Indexing Marketplace

Gaming Language / Translate Collaboration

SDK C++ .Net Java PHP Python Node.js ...

Microsoft Azure

Plán Studijní materiály, odkazy, harmonogram

http://ulita.ms.mff.cuni.cz/pub/predn/NSWI152/ (Ulita Výuka NSWI152)

Příprava prostředí Virtual Machines, RDP Instalace SDK, WATK Aktivace Windows Azure, Azure Management Portal

Customer Management System Příprava cloudové infrastruktury Windows Azure Training Kit Vytvoření, deployment a spuštění aplikace

GuestBook Vícevrstvá komplexnější reálná aplikace Web role, worker role, queues, blob, tables

Vlastní aplikačka Individuálně nebo komplexnější aplikace ve dvojici Libovolný jazyk: .Net, Java, Php, Node.js, Python ...

Příprava prostředí a aktivace Virtual Machines

VMware vSphere Client bobr.ms.mff.cuni.cz login: cloud_xx Grupíček pwd: v Grupíčku

vSphere Web client https://bobr.ms.mff.cuni.cz/vsphere-client

[VCenter] VM: BOBR / VIRTLAB / Cloud / Students / cloud_xx login: administrator pwd: v Grupíčku nastavena IP 10.2.1.xx ověřit! subnet 255.255.0.0, GW 10.2.0.1, DNS 195.113.19.71/77

RDP enable RDP: Start / Computer / Properties / Remote Settings / Allow Power Options - never sleep acheron.ms.mff.cuni.cz:102xx aktualizace! - povolit auto, provést

Aktivace Windows Azure http://ulita/pub/predn/NSWI152/azure/WindowsAzurePlatformAcademicPass.pptx http://www.windowsazurepass.com/azureu

Window Azure Management Portal https://manage.windowsazure.com/

Have a codeRedeem

Microsoft Azure Academic Licence Cloud Services

2 Small Instances for Cloud Services or Virtual Machines 10 Shared Web Sites, 10 Shared Mobile Services

Data Management Two 1 GB Web Edition SQL Databases 35 GB of Storage 50,000,000 Storage Transactions

Data Transfers 16 GB Bandwidth (8 GB In, 8 GB Out)

Messaging 750 Service Bus Relay Hours 250,000 Service Bus Messages

Identity 100,000 Access Control Transactions

Web Platform Installer - Azure SDK, WATK Windows Azure SDK 2.5 pro VS 2013

VS 2013 / Tools / Updates / Product / MS Azure SDK 2.5 Windows Azure Training Kit

http://www.microsoft.com/en-us/download/details.aspx?id=8396 c:\watk\HOL - hands-on labs

Checklist Před 2. cvičením mít zprovozněno:

virtual machine, RDP přes acheron W8.1 + aktualizace VS 2013 + Windows Azure SDK 2.5 Windows Azure Training Kit (WATK)

Microsoft Account (Windows Live ID) Aktivace Windows Azure - redeem code login to Azure Management Portal

v případě jakýchkoliv problémů se včas (do pátku) přihlaste platí i pro ty, kteří nebyli na prvním cvičení!

Studijní materiály Building Real-World Cloud Apps with Windows Azure leden 2014

http://www.windowsazure.com/en-us/develop/net/building-real-world-cloud-apps/ Azure documentation, články, linky, ...

http://ulita.ms.mff.cuni.cz/pub/predn/NSWI152/azure.html

Customer Management System

Customer Management System Windows Azure Training Kit

Building and Publishing ASP.NET Applications with Windows Azure Web Sites watk\HOL\ASPNETAzureWebSites Customer Management System

jednoduchý adresníček

Objectives Create a Web Site from the Windows Azure Management Portal Use MS VS 2013 to build a new ASP.NET MVC 5 application Deploy the application using Web Deploy from Visual Studio

Exercises E1: Creating a Windows Azure Web Site E2: Creating an MVC 5 Application with Entity Framework E3: Publishing an MVC 5 Application using Web Deploy (opt. E4: publish an ASP.NET MVC 5 application using Git or ftp)

Create Web Site E1: Creating a Windows Azure Web Site (with Visual Studio)

Connect to Azure [1-3] View / Server Explorer / Windows Azure / [RMB] Connect to WA / … login

Create Site – se studentskými licencemi nefunkční! [4] SE / Windows Azure / Web Sites / Add New Site [5-6] Site name (unique! URL), any location, Database: Create new server DB admin login & pwd, Create – unable to create…

Create Site Using Azure Management Portal Azure Management Portal – dole ✚ / Compute / Website / Custom Create – URL, region, SQL DB, connection string DB name, new, login, pwd Create check: Web Sites, SQL Databases

Back to WATK [7] Portal: explore Web Sites / mywebsite / Scale (do not change anything) [8] VS/SE: Refresh, Web Sites! (SQL Databases still empty) mywebsite / Open in Browser

Zapište si!

Create Application E2: Creating an MVC 5 Application with Entity Framework

Create New Project [1-2] Installed Templates, Visual C# Web, .NetFramework 4.5, ASP.Net Web Application [3] MVC, check: Authentication: Individual User Accounts

Model-View-Controller Solution explorer: Models, Views, Controllers, ..., ...

Create Customer Model [4] Solution Explorer/Models: Add class Customer [5-6] Models/Customer.cs: add properties, build

Create Person Controller & View [7-8] Solution Explorer/Controllers, Add Controller, MVC5 using EntFwk, Add [9] vyplnit přesně dle WATKuse async controller, Customer model

[10] Addactions, data context & views are generated

[11] Controllers/CustomersController.cs: see generated methodsSee async, await – asynchroneous execution

Publish Application E3: Publishing an MVC 5 Application using Web Deploy

Establish Connection [1-2] SE / MVCSample.Web / [RMB] Publish [3-5] Publish Web: Profile ⊳ Import Import from Azure / mywebsite / OKDialog is filled-in with the imported connection data

[6] Connection ⊳ Validate Connection ☑ / Next Connect to Database

[7] Settings ⊳ CustomerContext / mydatabase / Next Publish

[8] Preview ⊳ PublishBuild started … [chvíli trvá] ... Site was published successfully

[9] Check the browser, check url Use the Application

[10] url: /Customer Play the game – new, edit, deleteCheck the source code

Git & ftp Access

Samostatně

E4: Publishing an MVC 5 Application using Git Dle WATK

E5: Ftp Access Portal / WebSites / mywebsite / Dashboard / Quick Glance Set Deployment Credentials ftp/ftps host name, ftp user (incl. prefix!) connect, browse

GuestBook

GuestBook Windows Azure Training Kit

HOL\IntroToCloudServicesVS2012 Objectives

Create applications in Windows Azure using web roles and worker roles Use Storage services including blobs, queues and key/value tables Publish an application to Windows Azure Cloud Services

Setup IntroToCloudServicesVS2012\Source\setup.cmd - Run as administrator !

nestahovat VC2012 !

web role ≠ web site

GuestBook Outline E1: Azure Application with Blobs and Tables

Task 1 - Creating the VS Project Task 2 - Creating a Data Model for Table Storage Task 3 - Creating a Web Role to Display the Guest Book and Process User Input Task 4 - Queuing Work Items for Background Processing

E2: Background Processing with Worker Roles and Queues Task 1 - Creating a Worker Role to Process Images in the Background

E3: Publishing a Windows Azure Application Task 1 - Creating a Storage Account and a Cloud Service Component Task 2 - Publishing the Application to the Azure Portal Task 3 - Configuring the Application to Increase the Number of Instances Task 4 - Testing the Application in the Staging Environment Task 5 - Promoting the Application to Production

E1T1 - Create Application E1: Building Windows Azure Application with Blobs and Tables

Task 1 - Creating the Visual Studio Project Create Cloud Service Project

[1-2] Run VS 2013 as administratorWindows Azure Compute Emulator

[3-4] Installed Templates, Visual C# Cloud, .Net 4.5, W Azure Cloud Service, vyplnitName: GuestBook, Solution name: Begin

Create Web Service Web Role [5] New W Azure Cloud Service, Visual C#, ASP.Net Web Role ⍄ GuestBook_WebRole MVC [6] Solution Explorer - explore

Projects in Solution GuestBook - service definition, local / production configuration of roles, .cscfg GuestBook_WebRole - ASP.Net Web Application GuestBook_WorkerRole, GuestBook_Data, ... - later

Task 2 - Creating a Data Model for Entities in Table Storage Task 3 - Creating a Web Role to Display the Guest Book and Process User Input Task 4 - Queuing Work Items for Background Processing

E1T2 - Data Model Task 2 - Creating a Data Model for Entities in Table Storage

Key/Value Tables Key/Value Table - scalable data storage Table storage API:WCF Data Services (ADO.Net Data Services Framework) - REST APIWCF Data Services Client Library (.Net Client Library) - direct access from C# objects

MyEntity : TableServiceEntity - schema class MyContext : TableServiceContext - table access MyContext.IQueryable<MyEntity> - table definition and mapping

Create Data Project [1-2] New Project, Installed, Visual C#, Windows, .Net 4.5, ClassLibrary, GuestBook_DataSolution: Add to Solution !

[3] delete Class1.cs [4-5] Solution explorer, GuestBook_Data, References, Add ReferenceFramework ⊳ System.Data.ClientServices, Extensions ⊳ Microsoft.WindowsAzure.Storage, Microsoft.WindowsAzure.Configuration,

Create Schema Class [6] SE: GuestBook_Data, Add, New Item Visual C# Items, Class, ⊳ GuestBookEntry.cs [7] inherit GuestBookEntry class from TableServiceEntityTSE defines PartitionKey, RowKey, TimeStamp

E1T2 - Data Model Task 2 - Creating a Data Model ... continued

Define Schema PartitionKey day - partitioning, RowKey reverse time - sort order [8] default constructor - initializes keys [9-10] other properties - data row , saveMessage, GuestName, PhotoUrl, ThumbnailUrl

Create Context Class [11] GuestBook_Data, Add, New Item Class, ⊳ GuestBookDataContext.cs [12-14] GuestBookDataContext, constructor, GuestBookEntry property [15-16] Add Class GuestBookDataSource.cs - object bound to data controls [17-18] add storageAccount, context, constructor [19-23] finalize bindings, queries and initializations, save

results = from g in this.context.GuestBookEntry where g.PartitionKey == DateTime.UtcNow.ToString("MMddyyyy") select g;

TableQuery<GuestBookEntry> query = new TableQuery<GuestBookEntry>() .Where( TableQuery.GenerateFilterCondition( "PartitionKey", QueryComparisons.Equal, DateTime.UtcNow.ToString("MMddyyyy")));return table.ExecuteQuery(query);

2013: BuilderGetGuestBookEntriesUpdateImageThumbnail

2012: LINQ

E1T3 - Web Role Task 3 - Creating a Web Role to Display the Guest Book and Process User Input

Create Application Page [1] SE: GB_WebRole, Add reference to Solution / GuestBook_Data [2] delete automatically generated default page GBWR/Default.aspx [3] Add existing item watk\....\Source\Ex1...\Assets\*.*page already prepared by watk - code, images, styles - explore

Process Guest Entry [4-6] GB_WebRole: Default.aspx.cs - add using, declarationsnote: using GuestBook_Data, CloudBlobClient

[7] add event handler to SignButton_Click - upload to blobInitializeStorage (impl. later)gen unique blob name, create blob, uploadnew table entrysave data using GBDataSource

GuestBookEntry entry = new GuestBookEntry() { GuestName = this.NameTextBox.Text, Message = this.MessageTextBox.Text, PhotoUrl = blob.Uri.ToString(), ThumbnailUrl = blob.Uri.ToString() };GuestBookDataSource ds = new GuestBookDataSource();ds.AddGuestBookEntry(entry);

E1T3 - Web Role Task 3 - Creating a Web Role ... continued

Load Page - Timer [8] Timer1_Tick - periodical refresh [9] Page_Load - enable timer

Initialize Storage [10] InitializeStorageexecute only once, read account settings, create and configure blob container

Create Storage [11] SE: GB, Roles, GBWR [RTMB] Properties Settings Add Setting ⊳ "DataConnectionString" / ConnectionString [...] [12-13] Connect using WAzure Storage Emulator, saveemulator v ramci VS SDK, pro deployment se dosadi spravny connection string - nezapomenout!

storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");blobStorage = storageAccount.CreateCloudBlobClient();container = blobStorage.GetContainerReference("guestbookpics");container.CreateIfNotExist();

E1T4 - Queue & Run Task 4 - Queuing Work Items for Background Processing

Declare and Initialize Queue [1-2] Default.aspx.cs, class _Default - declare queueStorage [3] InitializeStorage - add initialization code after blobqueue "guestthumbs"

Enqueue Message [4] SignButton_Click - locate queue, form message, enqueue - add code

Verification Run Application

Windows Azure Compute Emulator (Development Fabric), Storage Browsercheck: Run VS 2013 as administrator

Update to .Net 4.5 - Web.config - do sekce <appSettings>

[1-3] F5 build & run - browser ➠ /Default.aspxsystem tray - Show Compute Emulator, firewall - allow access

Play the game Storage Explorer

[4-9] VS: View/ServerExplorer/Azure/Storage/Development - table, blob, queue

<add key="ValidationSettings:UnobtrusiveValidationMode" value="None" />

na webu

E2 - Worker Role E2: Background Processing with Worker Roles and Queues

Task 1 - Creating a Worker Role to Process Images in the Background Create Worker Role

[1-2] stop debugging, continue with the solution [3-4] GuestBook, Roles, New Worker Role - GuestBook_WorkerRole [5-6] add references - GuestBook_Data, System.Drawing

Programming Worker Role - Create Thumbnails [7-8] WorkerRole.cs: add using, WorkerRole: add queue and blob properties [9] OnStart - worker initializationread config, init blob & queue, catch

[10] Run - replace default body - main worker logicfetch & parse msg, read blob, create thumbnail, write blob, update thumbnailusing - unmanaged IDisposable objects processing

[11] ProcessImage - create thumbnail, save to output stream Storage Service

[12] Create WAzure Storage Emulator ( E1T3.12-13) Verification

[1-6] Run - wait a moment, create an entry, wait a moment [4] Server Explorer - explore data

old images

E3T1 - Create Storage and Component E3: Publishing a Windows Azure Application

Task 1 - Creating a Storage Account and a Cloud Service Component Create Virtual Network

[1-4] Portal / ✚ / Networks / Virtual Network / Custom CreateNetwork Name, Affinity Group Name

[5-7] - no other options! Create Cloud Service

[8-9] ✚ / Compute / Cloud Service / Quick Createunique URL

Create Storage [10-11] ✚ / Data Services / Storage / Quick Createthe same URL as [9] (if possible)

[12-14] Manage Keys - Primary Access Key - copy to clipboard [15] do not close the browser

E3T2 - Publish Task 2 - Publishing the Application to the Azure Portal

Deployment Options Visual Studio / PowerShell WAzure cmdlets / WAzure Portal

Service Configuration [1-3] continue with the solution (or Ex3/.../GuestBook.sln) [4-7] SE: GuestBook / Roles / GuestBook_WebRole / Properties Settings DataConnectionString⊳ [...] Manually entered [8] Microsoft.WindowsAzure.Plugins.... totéž GuestBook_WebRole - totéž (oba klíče)

Generate Package [9] VS: GuestBook [RMB] Package: Cloud/Debugenable NuGet download...\app.publish\*.cspkg, .cscfg

Upload Package [11-19] Cloud Service, Dashboard, Staging, Upload [16] !! check Deploy even if one or more roles contain a single instance - wait Play the game, check Portal, ..., ...

Task 3 - Configuring the Application to Increase the Number of Instances Task 4 - Testing the Application in the Staging Environment Task 5 - Promoting the Application to Production

nejdřív vyzkoušet v staging, potom

upgrade to production

při překliknutí přes Your subscriptionse správně vyplní

otevře sefolder s package

Useful tools & practices Useful tools

Storage explorer http://azurestorageexplorer.codeplex.com/ V4 - stable

Useful Labs Mobile

AndroidAndMobileServices HTMLAppAndMobileServices MobileServicesCustomAPI

Other services IntroToWindowsAzureMediaServices ServiceBusTopics

DebuggingCloudServices

Practices Azure Blogs

http://blogs.msdn.com/b/windowsazure/ Using Trace

http://blogs.msdn.com/b/windowsazure/archive/2012/10/25/using-trace-in-windows-azure-cloud-applications-1.aspx

.cscfg: Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString

Cloud Development Patterns Automate everything

Use scripts to maximize efficiency and minimize errors in repetitive processes Source control

Set up branching structure in source control Continuous integration and delivery

Automate build and deployment with each source control check-in Web development best practices

Keep web tier stateless; Avoid session state; Use a CDN; Use asynchronous programming model Data storage options

Types of data stores; How to choose the right data store; Unstructured blob storage Data partitioning strategies

Partition data vertically, horizontally, or both to facilitate scaling a relational database Design to survive failures

Types of failures; Failure Scope; Understanding SLAs Monitoring and telemetry

Why you should both buy a telemetry app and write your own code to instrument your app Transient fault handling

Use smart retry/back-off logic to mitigate the effect of transient failures Distributed caching

Improve scalability and reduce database transaction costs by using distributed caching Queue-centric work pattern

Enable high availability and improve scalability by loosely coupling web and worker tiers Single sign-on - Active Directory

Building Real-World Cloud

Apps with Windows Azure

Vlastní aplikace Vlastní aplikace

individuálně nebo větší aplikace ve dvojici libovolná platforma cokoliv, co využije větší počet modulů a služeb

Web-Worker Roles, Key/Value Tables, BLOBs, Mobile, Media, Service Bus, Hadoop, .... vloni - pro inspiraci

Mobilní kurýr s balíčkem, Pet gallery, Azgool Plus, Twitter, Výsledky florbalu, Jídelníčky menz pro mobily, Konverze a filtr velkých XML, Renderovani map, Bugzilla, Cestovní agentura, Rapidshare, Kontroly revizorů, Beer locator, Jízdní řády

Dokončení a předvedení poslední cvičení 20.5. slajdy - prezentace

co to dělá jak je to udělané moduly a služby, vazby mezi nimi zhodnocení - co bylo zajímavé, kladně i záporně srovnání s ostatnímiplatformami

ukázka reálného běhu

Google App Engine

Pavel Jančík

OpenStack

Lukáš Korous

The End.