Doing More with Postgres - Yesterday's Vision Becomes Today's Reality

Embed Size (px)


PostgreSQL has surged forward in capability and market acceptance in recent years like no time before as the community responded to market forces and enhanced and extended the database in critical areas. Today's PostgreSQL has achieved new levels of usability, scalability and capacity for new workloads. Marc Linster, Senior Vice President of Products and Services at EnterpriseDB, delivered this presentation at PG Open 2014. He covered the powers of PostgreSQL today compared to the vision taking shape just a few short years ago. He addressed how performance and scalability has advanced to support enterprise resource planning solutions for global brands through EnterpriseDB's work with Infor, the world's fourth-largest ERP vendor. Finally, Linster discussed how capacity to support new NoSQL workloads has expanded and explored the new toolkit, PG XDK.

Text of Doing More with Postgres - Yesterday's Vision Becomes Today's Reality

  • 1. Postgres Evolutions Marc Linster 2013 EDB All rights reserved 8.1. 1
  • 2. Agenda Who is EDB? Market forces Its happening in the cloud Its happening Postgres powers ERP Its happening Not Only SQL Its happening platform coexistence 2013 EDB All rights reserved 8.1. 2
  • 3. PostgreSQL Who is EDB? Postgres Plus Advanced Server Postgres Plus 2013 EDB All rights reserved 8.1. 3 Cloud 180+ employees Global presence Focused on serving needs of enterprise, government and ISV markets Platinum Sponsor of PostgreSQL
  • 4. Market forces influence direction (PG Europe 2013) Easy to use / deploy New Workloads/Platforms (Big Data, NoSQL & Cloud) 2013 EDB All rights reserved 8.1. 4 High-end Enterprise Requirements PostgreSQL is growing from the middle-out
  • 5. Marketplace Demand Next Five Years Diagnosing Problems Configuring for success Still easier installs Tighter integration with frameworks Integration with other data stores Very simple in the cloud 2013 EDB All rights reserved 8.1. 5 Liasons with other communities FDW for common no-SQL DBs Continue to evolve new datatypes: JSON, XML, HStore Vertical Scale (parallel query) Horizontal Scale Performance Diagnostics Incremental Backup Integration with other data stores Zero down time upgrades Easy to use / deploy High-end Enterprise Requirements New Workloads & Platforms (Big Data/No/SQL/Cloud)
  • 6. Postgres: Cloud Its Happening Source: Gartner 2013 Forecast: Public Cloud Services, Worldwide, 2011-2017, 3Q13 Update 45 40 35 30 25 20 15 10 5 2013 EDB All rights reserved 8.1. 6 2011 2012 2013 2014 2015 2016 2017 160 140 120 100 80 60 40 20 Cloud PaaS/Database Management Systems ($M) 17 28 42 59 82 114 149 0 Cloud PaaS/DBMS ($M) Cloud PaaS has the highest growth rate 35%+ (according to Gartner Group) EDBs Postgres Plus Cloud Database at AWS confirms this Easy installation Elastic and scalable Automatic failover PITR VPC . 0 PPCD New Cluster# Growth/June Sept13
  • 7. Postgres powers ERP Infor LN (formerly Baan ERP) supported on Postgres Plus Advanced Server Joint announcement by Infor, Red Hat & EDB - April 2014 Part of the Infor Open Source Initiative We can deliver a better experience with open source because it is open enough to allow us to configure a seamless experience with our applications with fewer licensing and distribution restrictions. We can easily move images between on premise and cloud as customer needs dictate. - Charles Philips, CEO, Infor 2014 EnterpriseDB Corporation. All rights reserved. 7
  • 8. Infor LN/Postgres Stack LN / Infor Ming.le LN UI LN VM + DB Driver EDB OCL libpq Postgres Plus Red Hat Enterprise Linux Red Hat Enterprise Virtualization 2013 EDB All rights reserved 8.1. 8 Presentation Tier Red Hat JBoss Apache Tomcat Application Tier LN VM + LN DB Driver Database Tier Postgres Plus Advanced Server Operating System Tier Red Hat Enterprise Linux Red Hat Enterprise Virtualization
  • 9. Performance (250 Concurrent Users) Normalized Transaction Times 90th percentile (smaller is better) 2013 EDB All rights reserved 8.1. 9 Postgres 87.4 transactions/user, 15 minutes Proprietary RDBMS 88 transactions/user, 120% 100% 80% 60% 40% 20% EnterpriseDB Proprietary RDBMS Limit 15 minutes 0% Its happening! Postgres powers ERP!
  • 10. Its happening Not only SQL Postgres + Documents (JSON) + KVP (HSTORE) = Best possible NoSQL solution 2013 EDB All rights reserved 8.1. 10
  • 11. Standard Objections SQL databases cant handle NoSQL data types Postgres can (JSON and KVP) SQL databases are too slow and cant ingest the data fast enough Postgres can keep up We ran the tests Benchmarks are public Great feedback SQL databases cant support Web 2.0 development approaches Postgres can Support unstructured and structured data in the same environment Move data from unstructured to structured (and back) Use Web 2.0 languages inside the database (e.g.: PL/V8) or in the application (node.js) Integrate NoSQL data (FDW) 2013 EDB All rights reserved 8.1. 11
  • 12. JSON Performance Evaluation Goal Help our customers understand when to chose Postgres and when to chose a specialty solution Help us understand where the NoSQL limits of Postgres are Setup Compare Postgres 9.4 to Mongo 2.6 Single instance setup on AWS M3.2XLARGE (32GB) Test Focus Data ingestion (bulk and individual) Data retrieval 2013 EDB All rights reserved 8.1. 12
  • 13. Performance Evaluation Generate 50 Million JSON Documents Load into MongoDB 2.6 2013 EDB All rights reserved 8.1. 13 (IMPORT) Load into Postgres 9.4 (COPY) 50 Million individual INSERT commands 50 Million individual INSERT commands Multiple SELECT statements Multiple SELECT statements T1 T2 T3
  • 14. NoSQL Performance Evaluation Mongo DB 2.4/Postgres 9.4 Relative Performance Comparison (50 Million Documents) 276% 295% 500% 450% 400% 350% 300% 250% 200% 150% 100% 50% 2013 EDB All rights reserved 8.1. 14 465% 208% 0% Data Load Insert Select Size Postgres MongoDB Postgres MongoDB Data Load (s) 4,732 13,046 Insert (s) 29,236 86,253 Select (s) 594 2,763 Size (GB) 69 145 Correction to earlier versions: MongoDB console does not allow for INSERT of documents > 4K. This lead to truncation of the MongoDB size by approx. 25% of all records in the benchmark. Thank you Alvaro Tortosa!
  • 15. Performance Evaluations Next Steps Initial tests confirm that Postgres can handle many NoSQL workloads The test scripts are publically available EDB encourages community participation to better define where Postgres should be used and where specialty solutions are appropriate Join us to discuss the findings at 2013 EDB All rights reserved 8.1. 15
  • 16. PG XDK Showcase Postgres & Web 2.0 Development Postgres Extended Document Type Developer Kit Provides end-to-end Web 2.0 example all sources included Deployed as free AMI First Version Postgres 9.4 (beta) w. HSTORE and JSONB Python, Django, Bootstrap, psycopg2 and nginx Next Version: PL/V8 & Node.js Final Version: Ruby on Rails 2013 EDB All rights reserved 8.1. 16 AWS AMI PG XDK v0.2 - ami-1616b57e
  • 17. Installing PG XDK Select PG XDK v0.2 - ami-1616b57e on the AWS Console Use 1#LaunchInstanceWizard:ami=ami-1616b57e Works with t2.micro (AWS Free Tier) Remember to enable HHTP access in the AWS console 2013 EDB All rights reserved 8.1. 17
  • 18. Foreign Data Wrappers Co-Existence Platform FDW implements SQL/MED ("SQL Management of External Data") FDW Makes data on other servers (or services) look like tables in Postgres available for databases (MongoDB, MySQL, Oracle, ), files, services (Twitter, ) MongoDB FDW: (a collaboration with CitusDB) The list: 2013 EDB All rights reserved 8.1. 18
  • 19. Conclusion Its happening Has great deploymen