26
Scalable OSGeo Stacks From Tutorials to Production David Zwarg [email protected]

OSGeo in Production

Embed Size (px)

Citation preview

Page 1: OSGeo in Production

Scalable OSGeo StacksFrom Tutorials to Production

David Zwarg [email protected]

Page 2: OSGeo in Production

Tutorials

• A good place to start!

• A bad place to end!

Page 3: OSGeo in Production

Production Environments

• Your production environment is a beautiful snowflake.

!

• Mileage may vary

Page 4: OSGeo in Production

Definition of Terms

• “Production”

• A data-driven, dynamic, scalable, web application

• “Tutorial”

• Anything else

Page 8: OSGeo in Production

“web application”An application that provides rich interactivity in a user’s web browser.

Page 9: OSGeo in Production

Good Tutorials Warn You

• Bad tutorials don’t

Page 10: OSGeo in Production

Tutorial #1: django

• https://docs.djangoproject.com/en/1.7/intro/tutorial01/

• Do not ignore these helpful notes:

Page 11: OSGeo in Production

Tutorial #2: geodjango

• “Use the source”

• Source installs can be just as dependable as package management solutions

Page 12: OSGeo in Production

Side Note: Source Install

• Ensure that all script installations are scriptable

• Script & version package management installations

• Aim for a repeatable installation

Page 13: OSGeo in Production

Tutorial #3: celery

• More helpful log messages: “UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!”

Page 14: OSGeo in Production

Side Note: Workers

• Long-running processes should be performed “out of band”

• Use separate processes to do big tasks

Page 18: OSGeo in Production

Scaling GEOS & GDAL

• Single-threaded designs

• Apache: use the preform multiprocessing module (mpm_prefork)

• gunicorn: use synchronous workers with many worker_processes

Page 19: OSGeo in Production

Scaling Tiles

• Geoserver includes Geowebcache

• Cache WMS image tiles

• Automatic cache seeding

• Scriptable via REST

Page 20: OSGeo in Production

Scaling Tiles (cont.)

• Tilestache

• Plugs into Mapnik

• Use with MBTiles, Amazon S3, Memcache, etc

Page 21: OSGeo in Production

Scaling PostGIS

• Heroku provides the option of HA failover and PostGIS.

• Or, roll your own snowflake (WAL, Slony, Bucardo)

Page 22: OSGeo in Production

Frontend ScalingTips for Web Applications

Page 23: OSGeo in Production

OpenLayers

• Loading Strategies

• BBOX

• Vector “Resolution”

• Don’t drown in data

Page 24: OSGeo in Production

Use a CDN

• “Content Delivery Network”

• https://cdnjs.com

• https://ajax.googleapis.com

• http://cloudflare.net

• more!

Page 25: OSGeo in Production

General Tips

• Scale servers on demand

• Don’t overwhelm clients

• Focus on long-running processes & slow requests

• Use a cache

Page 26: OSGeo in Production

Scalable OSGeo StacksFrom Tutorials to Production

David Zwarg [email protected]