MySQL

AdministratorsBible

MySQL

AdministratorsBible

Sheeri CabralKeith Murphy

Wiley Publishing, Inc.

MySQLAdministrators Bible

Published byWiley Publishing, Inc.10475 Crosspoint BoulevardIndianapolis, IN 46256www.wiley.com

Copyright 2009 by Wiley Publishing, Inc., Indianapolis, Indiana

Published simultaneously in Canada

ISBN: 978-0-470-41691-4

Manufactured in the United States of America

10 9 8 7 6 5 4 3 2 1

Library of Congress Cataloging-in-Publication Data:

Cabral, Sheeri, 1978-MySQL administrators bible / Sheeri Cabral, Keith Murphy.

p. cm.Includes index.ISBN 978-0-470-41691-4 (paper/website)1. MySQL (Electronic resource) 2. Database management. 3. Relational databases. I. Cabral, Sheeri,

1970- II. Title.QA76.9.D3C3178 2009005.7565 dc22

2009005633

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form orby any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permit-ted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior writtenpermission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copy-right Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600.Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley& Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online athttp://www.wiley.com/go/permissions.

Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warrantieswith respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties,including without limitation warranties of fitness for a particular purpose. No warranty may be created orextended by sales or promotional materials. The advice and strategies contained herein may not be suitable forevery situation. This work is sold with the understanding that the publisher is not engaged in rendering legal,accounting, or other professional services. If professional assistance is required, the services of a competentprofessional person should be sought. Neither the publisher nor the author shall be liable for damages arisingherefrom. The fact that an organization or Web site is referred to in this work as a citation and/or a potentialsource of further information does not mean that the author or the publisher endorses the information theorganization or Web site may provide or recommendations it may make. Further, readers should be aware thatInternet Web sites listed in this work may have changed or disappeared between when this work was writtenand when it is read.

For general information on our other products and services please contact our Customer Care Departmentwithin the United States at (877) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.

Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc.and/or its affiliates, in the United States and other countries, and may not be used without written permission.MySQL is a registered trademark of MySQL AB Company. All other trademarks are the property of their respec-tive owners. Wiley Publishing, Inc. is not associated with any product or vendor mentioned in this book.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not beavailable in electronic books.

www.wiley.comwww.wiley.com/go/permissions

To Mitch Cherniack, who taught me databases and inspired mylove of them. In memory of Zvi Kramer and Yehoshua

Kritzer may I be able to echo their strength andspirit. Sheeri Cabral

For Rachel, the love of my life. Keith Murphy

About the AuthorsSheeri K. Cabral is a Database Team Lead at the Pythian Group, a remote database administra-tion solution. She has a masters degree in computer science specializing in databases from Bran-deis University and a career background in systems administration. Unstoppable as a volunteerand activist since she was 14, Sheeri founded and organizes the Boston MySQL User Group. Shealso produces OurSQL: The MySQL Database Podcast for the Community, By the Communityand many workshop videos. These resources are freely available at www.technocation.org,the website of Technocation, Inc. Sheeri is the Treasurer of Technocation, Inc, a not-for-profitorganization providing resources and educational grants for IT professionals. Sheeri currentlylives in the Boston area with her husband Tony Cabral and a myriad assortment of Muppets.When shes not working or volunteering, she reads, knits, listens to podcasts, and enjoys timewith her nieces and nephews.

Keith Murphy is a MySQL database administrator who has been using MySQL serversince 1998. A long-time Linux system administrator, he worked with a number of Internetcompanies solving systems issues. More recently he formed Paragon Consulting Services(http://www.paragon-cs.com) to provide consulting services for companies seeking trainingfor MySQL and help with MySQL solutions ranging from everyday database administrationtasks to utilizing cloud computing services, performance tuning, and scaling. In addition toinformation about the company, Keith blogs at http://blog.paragon-cs.com, which heupdates as frequently as possible with tricks, tips, and general information about MySQL server.

In addition he is the editor of MySQL Magazine (http://www.mysqlzine.net), the MySQLcommunitys digital magazine about all things MySQL. It is his pleasure to work with some ofthe finest people in the community in this collaborative effort to increase both the amount andquality of the available information on MySQL server.

Readers are invited to contact Keith by email at bmurphy@pargon-cs.com.

Credits

Executive EditorRobert Elliott

Development EditorEd Connor

Technical EditorsBaron SchwartzMorgan TockerNicklas Westerlund

Production EditorLiz Britten

Copy EditorsKim CoferFoxxe Editorial Services

Editorial ManagerMary Beth Wakefield

Production ManagerTim Tate

Vice President and Executive GroupPublisherRichard Swadley

Vice President and Executive PublisherBarry Pruett

Associate PublisherJim Minatel

Project Coordinator, CoverLynsey Stanford

ProofreaderJen Larsen, Word One New York

IndexerJack Lewis

Sheeri Cabral

When I was four years old, I refused to let anyone else tie my shoes, because I wanted to do itmyself. There was only one problem I did not know how to tie my shoes yet. I went aroundfor months with untied shoes because I made it very clear that I was going to do it. Eventually Ilearned to tie my shoes. I also learned that I could do anything I put my mind to.

I could not have accomplished the herculean task of writing a book had I not had the encour-agement of my stubbornness. To Mom, Dad, Lymor, Noam, Josh, and Elinor, thanks for puttingup with me all these years, letting me try my way so that my failures and successes were myown, and teaching me to always do a good job so I can be proud of my work. My love ofdatabases was inspired by the man who taught me the subject at Brandeis University, ProfessorMitch Cherniack. Professionally, I would not be the person I am today without the guidanceand example of Tom Limoncelli, an inspirational role model, great mentor, and good friend; andTracy Gangwer, who taught me to be a super DBA/sysadmin by confirming and triple-checkingeverything. A huge thank you goes to Keith for being the lead author on this project, and ofcourse for asking me to become involved. Of course, none of this could have been done withoutthe support of Wiley Publishing, particularly our editors Ed Connor and Bob Elliott.

I used many two-dimensional resources while writing this book, but theres a limit to how muchinformation is in manuals and books. Special thanks are due to Brian Aker for being a constantfountain of readily available knowledge and Martin Garden for software assistance. Jay Pipesprovided permission for material used in the Query Analysis chapter, Roland Bouman helpedme get a firm grip on the data dictionary, and Mark Atwood answered my questions about howstorage engines work. I was encouraged and supported by Carsten Turner, Ronald Bradford,Patrick Galbraith, and many fans of the OurSQL Podcast who wrote to me asking when it wascoming back.

Keith Murphy

There are many people involved in any book project. Without the crew of people from Wiley wecould never have finished this book. Thank you, Bob Elliott (our Executive Editor) for taking achance on me. I hope you werent disappointed. Without our Development Editor, Ed Connor,I suspect we would have ended up in the ditch never to get out. And thanks to all the otherpeople at Wiley who I didnt have the chance to meet who help make this happen.

xi

Acknowledgments

Sheeri, I could not have done this alone. I am grateful for all the long hours and the meticulousattention to detail that you brought to the project. If I ever get the notion to write another bookyou will be the first person I call. Of course you dont have to answer!

And a big thanks for all those in the MySQL community. It is you who help power this crazything we call Open Source. Thank you for taking the time help others learn and become better.

xii

Introduction .................................................................................................................................xxvii

Part I First Steps with MySQLChapter 1: Introduction to MySQL ..................................................................................................3Chapter 2: Installing and Upgrading MySQL Server ....................................................................... 9Chapter 3: Accessing MySQL ..........................................................................................................49

Part II Developing with MySQLChapter 4: How MySQL Extends and Deviates from SQL ............................................................87Chapter 5: MySQL Data Types .....................................................................................................159Chapter 6: MySQL Index Types ...................................................................................................219Chapter 7: Stored Routines, Triggers, and Events .......................................................................241Chapter 8: MySQL Views ..............................................................................................................301Chapter 9: Transactions in MySQL ..............................................................................................319

Part III Core MySQL AdministrationChapter 10: MySQL Server Tuning ..............................................................................................349Chapter 11: Storage Engines .........................................................................................................375Chapter 12: Caching with MySQL ...............................................................................................423Chapter 13: Backups and Recovery ..............................................................................................439Chapter 14: User Management .....................................................................................................473Chapter 15: Partitioning ................................................................................................................495Chapter 16: Logging and Replication ...........................................................................................517Chapter 17: Measuring Performance ............................................................................................545

Part IV Extending Your SkillsChapter 18: Query Analysis and Index Tuning ...........................................................................589Chapter 19: Monitoring Your Systems .........................................................................................633Chapter 20: Securing MySQL .......................................................................................................649Chapter 21: The MySQL Data Dictionary ....................................................................................667Chapter 22: Scaling and High Availability Architectures .............................................................727Appendix A: MySQL Proxy ...........................................................................................................749Appendix B: Functions and Operators .........................................................................................783Appendix C: Resources .................................................................................................................813

Index ..............................................................................................................................................821

xiii

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii

Part I First Steps with MySQL

Chapter 1: Introduction to MySQL . . . . . . . . . . . . . . . . . . . . . 3MySQL Mission Speed, Reliability, and Ease of Use ................................................3

Company background ...................................................................................................4Community and Enterprise server versions .................................................................5

The MySQL Community ......................................................................................................6How to contribute .........................................................................................................6Reasons to contribute ....................................................................................................7

Summary ................................................................................................................................7

Chapter 2: Installing and Upgrading MySQL Server . . . . . . . . . . . . . 9Before Installation ................................................................................................................9

Choosing the MySQL version .....................................................................................11MySQL support ...........................................................................................................12Downloads ...................................................................................................................12

Installation ..........................................................................................................................12MySQL Server installations on Unix ...........................................................................13MySQL Server Installation on Windows ....................................................................20Installing MySQL from a Noinstall Zip Archive .........................................................24Starting and stopping MySQL from the Windows command line ............................25Starting and stopping MySQL as a Windows service ................................................26

Initial Configuration ..........................................................................................................29Unix configuration file ................................................................................................31Windows configuration file .........................................................................................31

MySQL Configuration Wizard on Windows ..................................................................31Detailed Configuration ................................................................................................32The Server Type screen ...............................................................................................33Database Usage screen ................................................................................................33InnoDB Tablespace screen ..........................................................................................34Concurrent Connections screen ..................................................................................34Networking Options and Strict Mode Options screen ..............................................34Character Set screen ....................................................................................................35Service Options screen ................................................................................................35Security Options screen ..............................................................................................35Confirmation screen ....................................................................................................36

xv

Contents

MySQL Post-Install Configuration on Unix ...................................................................36Initializing the system tables .......................................................................................36Setting initial passwords ..............................................................................................37Root user password assignment ..................................................................................37Anonymous users ........................................................................................................39

Securing Your System ........................................................................................................40Windows PATH Variable Configuration ........................................................................42

Automated startup .......................................................................................................42Starting and stopping mysqld on System V-based Unix ........................................... 42System V run levels .....................................................................................................43

Upgrading mysqld ..............................................................................................................45The MySQL changelog ................................................................................................45Upgrading MySQL on Windows ................................................................................46

Troubleshooting ..................................................................................................................47Summary ..............................................................................................................................48

Chapter 3: Accessing MySQL . . . . . . . . . . . . . . . . . . . . . . . 49Accessing mysqld with Command-Line Tools ...............................................................49

Frequently used options ..............................................................................................50Using the command-line mysql client ........................................................................52mysqladmin Client for administering a server ......................................................62

GUI Tools ............................................................................................................................66SQLyog ........................................................................................................................66phpMyAdmin .............................................................................................................. 69MySQL Query Browser ...............................................................................................71MySQL Administrator .................................................................................................74MySQL Workbench .....................................................................................................80

Summary ..............................................................................................................................83

Part II Developing with MySQL

Chapter 4: How MySQL Extends and Deviates from SQL . . . . . . . . . 87Learning MySQL Language Structure .............................................................................88

Comments and portability ..........................................................................................88Case-sensitivity ............................................................................................................90Escape characters .........................................................................................................91Naming limitations and quoting .................................................................................93Dot notation ................................................................................................................95Time zones ..................................................................................................................97Character sets and collations ......................................................................................98

Understanding MySQL Deviations ................................................................................105Privileges and permissions ........................................................................................110Transaction management ..........................................................................................110Check constraints ......................................................................................................111Upsert statements ......................................................................................................112

xvi

Contents

Using MySQL Extensions ................................................................................................114Aliases ........................................................................................................................115ALTER TABLE extensions .........................................................................................115CREATE extensions ...................................................................................................118DML extensions .........................................................................................................119DROP extensions .......................................................................................................124The LIMIT extension .................................................................................................125SELECT extensions ................................................................................................... 126SELECT . . . INTO OUTFILE/SELECT . . . INTO DUMPFILE ...............................126SQL_SMALL_RESULT/SQL_BIG_RESULT ...............................................................127UNION . . . ORDER BY ............................................................................................127SELECT . . . FOR UPDATE ......................................................................................127SELECT . . . LOCK IN SHARE MODE ....................................................................128DISTINCTROW .........................................................................................................128SQL_BUFFER_RESULT .............................................................................................129HIGH_PRIORITY/LOW_PRIORITY ..........................................................................129Server maintenance extensions .................................................................................129The SET extension and user-defined variables ........................................................131The SHOW extension ...............................................................................................135Table definition extensions .......................................................................................147Table maintenance extensions ..................................................................................150Transactional statement extensions .......................................................................... 156

Summary ............................................................................................................................158

Chapter 5: MySQL Data Types . . . . . . . . . . . . . . . . . . . . . . 159Looking at MySQL Data Types ......................................................................................159Character String Types ...................................................................................................160

Length ........................................................................................................................162Character string type attributes ................................................................................164

National Character String Types ...................................................................................166Binary Large Object String Types .................................................................................168

BLOB values ..............................................................................................................169BINARY values ...........................................................................................................169BINARY length ..........................................................................................................169VARBINARY length ...................................................................................................170

Numeric Types ..................................................................................................................170Numeric data sizes and ranges .................................................................................172Numeric data type attributes ....................................................................................177

Boolean Types ...................................................................................................................180Datetime Types .................................................................................................................183

Allowed input values .................................................................................................185Microsecond input ....................................................................................................186Automatic updates .....................................................................................................187Conversion issues ......................................................................................................188Numeric functions and DATETIME types ................................................................188

xvii

Contents

Other conversion issues ............................................................................................190Datetime data type attributes ....................................................................................191The effect of time zones ............................................................................................192

Interval Types ...................................................................................................................193ENUM and SET Types .....................................................................................................195

Enumerations .............................................................................................................195ENUM and SET data type attributes ........................................................................198

Choosing SQL Modes ...................................................................................................... 201Invalid data ................................................................................................................201SQL modes ................................................................................................................203

Using NULL Values ..........................................................................................................211Finding an Optimal Data Type for Existing Data ......................................................212

Small data samples and PROCEDURE ANALYSE() .................................................215Summary ............................................................................................................................217

Chapter 6: MySQL Index Types . . . . . . . . . . . . . . . . . . . . . . 219

Looking at Keys and Indexes .........................................................................................219Using Indexes to Speed Up Lookups ............................................................................221

Creating and dropping indexes ................................................................................223Index order ................................................................................................................225Index length ..............................................................................................................226Index types ................................................................................................................228Redundant indexes ....................................................................................................230

Creating and Dropping Key Constraints ......................................................................231Creating and dropping unique key constraints ........................................................231Creating and dropping foreign key constraints ........................................................232Foreign key constraints and data changes ................................................................234Requirements for foreign key constraints .................................................................235

Using FULLTEXT Indexes ...............................................................................................237Summary ............................................................................................................................239

Chapter 7: Stored Routines, Triggers, and Events . . . . . . . . . . . . . 241

Comparing Stored Routines, Triggers, and Events ....................................................241Using Triggers ...................................................................................................................242

Creating a trigger .......................................................................................................243Dropping a trigger .....................................................................................................244Multiple SQL statements in triggers .........................................................................245Changing a trigger .....................................................................................................246Triggers on views and temporary tables ...................................................................247Trigger runtime behavior ..........................................................................................248Finding all triggers ....................................................................................................252Trigger storage and backup ......................................................................................252Triggers and replication ............................................................................................254Trigger limitations .....................................................................................................254

xviii

Contents

Using Stored Routines .....................................................................................................255Performance implications of stored routines ............................................................256Stored procedures vs. stored functions ....................................................................256Creating a stored routine ..........................................................................................256Invoking a stored procedure .....................................................................................259Dropping a stored routine ........................................................................................261Multiple SQL statements in stored routines .............................................................261INOUT arguments to a stored procedure ................................................................261Local variables ...........................................................................................................262Stored routine runtime behavior ..............................................................................264Options when creating routines ...............................................................................265Creating a basic stored function ...............................................................................268Full CREATE FUNCTION syntax .............................................................................269Invoking a stored function ........................................................................................269Changing a stored routine ........................................................................................270Naming: stored routines ...........................................................................................271Stored procedure result sets ..................................................................................... 273Stored routine errors and warnings ..........................................................................274Conditions and handlers ...........................................................................................275Stored routine flow control .......................................................................................282Recursion ...................................................................................................................284Stored routines and replication ................................................................................285Stored function limitations ....................................................................................... 285Stored routine backup and storage ...........................................................................286

Using Cursors ...................................................................................................................287Using Events ......................................................................................................................289

Turning on the event scheduler ...............................................................................289Creating an event ......................................................................................................291Dropping an event ....................................................................................................292Multiple SQL statements in events ...........................................................................293Start and end times for periodic events ...................................................................293Event status ...............................................................................................................294Finding all events ......................................................................................................295Changing an event ....................................................................................................295After the last execution of an event ..........................................................................296Event logging .............................................................................................................297Event runtime behavior ............................................................................................ 298Event limitations ....................................................................................................... 299Event backup and storage .........................................................................................300

Summary ............................................................................................................................300

Chapter 8: MySQL Views . . . . . . . . . . . . . . . . . . . . . . . . . 301Defining Views ..................................................................................................................302

View definition limitations and unexpected behavior .............................................304Security and privacy ..................................................................................................305

xix

Contents

Specify a views definer .............................................................................................306Abstraction and simplification ..................................................................................307Performance ...............................................................................................................308Updatable views ........................................................................................................313

Changing a View Definition ...........................................................................................317Replication and Views .....................................................................................................317Summary ............................................................................................................................318

Chapter 9: Transactions in MySQL . . . . . . . . . . . . . . . . . . . . 319Understanding ACID Compliance .................................................................................320

Atomicity ...................................................................................................................321Consistency ................................................................................................................321Isolation .....................................................................................................................321Durability ...................................................................................................................321

Using Transactional Statements ....................................................................................322BEGIN, BEGIN WORK, and START TRANSACTION .............................................322COMMIT ...................................................................................................................322ROLLBACK ................................................................................................................322Savepoints ..................................................................................................................323AUTOCOMMIT .........................................................................................................324

Using Isolation Levels .....................................................................................................325READ UNCOMMITED ..............................................................................................329READ COMMITTED .................................................................................................331REPEATABLE READ .................................................................................................332SERIALIZABLE ..........................................................................................................334Multi-version concurrency control ...........................................................................335

Explaining Locking and Deadlocks ...............................................................................336Table-level locks ........................................................................................................338Page-level locks .........................................................................................................341Row-level locks ..........................................................................................................341

Recovering MySQL Transactions ...................................................................................343Summary ............................................................................................................................344

Part III Core MySQL Administration

Chapter 10: MySQL Server Tuning . . . . . . . . . . . . . . . . . . . . 349Choosing Optimal Hardware .........................................................................................349Tuning the Operating System ........................................................................................352

Operating system architecture ..................................................................................352File systems and partitions .......................................................................................353Buffers ........................................................................................................................356Kernel parameters .....................................................................................................357Linux ..........................................................................................................................357Other daemons ..........................................................................................................360

xx

Contents

Tuning MySQL Server ..................................................................................................... 360Status variables ..........................................................................................................360System variables ........................................................................................................361Option file .................................................................................................................361Dynamic variables .....................................................................................................371

Summary ............................................................................................................................373

Chapter 11: Storage Engines . . . . . . . . . . . . . . . . . . . . . . . 375Understanding Storage Engines .....................................................................................375

Storage engines as plugins ........................................................................................376Storage engine comparison .......................................................................................376

Using Different Storage Engines ....................................................................................378MyISAM storage engine ............................................................................................378InnoDB storage engine ..............................................................................................384MEMORY storage engine ..........................................................................................394Maria storage engine .................................................................................................396Falcon storage engine ................................................................................................401PBXT storage engine .................................................................................................410FEDERATED storage engine .....................................................................................415NDB storage engine ...................................................................................................417Archive storage engine ..............................................................................................417Blackhole storage engine ...........................................................................................419CSV storage engine ...................................................................................................420

Working with Storage Engines ......................................................................................421CREATE TABLE ........................................................................................................421ALTER TABLE ...........................................................................................................421DROP TABLE ............................................................................................................422

Summary ............................................................................................................................422

Chapter 12: Caching with MySQL . . . . . . . . . . . . . . . . . . . . 423Implementing Cache Tables ...........................................................................................424Working with the Query Cache .....................................................................................427

What gets stored in the query cache? ......................................................................427Query cache memory usage and tuning ...................................................................429Query cache fragmentation .......................................................................................433

Utilizing memcached .......................................................................................................434Summary ............................................................................................................................438

Chapter 13: Backups and Recovery . . . . . . . . . . . . . . . . . . . . 439Backing Up MySQL ..........................................................................................................439

Uses for backups .......................................................................................................441Backup frequency ......................................................................................................443What to back up .......................................................................................................445Backup locations .......................................................................................................445Backup methods ........................................................................................................445

xxi

Contents

Online backup ...........................................................................................................460mysqlhotcopy ............................................................................................................462Commercial options ..................................................................................................464

Copying Databases to Another Machine ......................................................................467Recovering from Crashes ................................................................................................468Planning for Disasters .....................................................................................................471Summary ............................................................................................................................472

Chapter 14: User Management . . . . . . . . . . . . . . . . . . . . . . 473

Learning about MySQL Users ........................................................................................473Access Control Lists ..................................................................................................474Wildcards ..................................................................................................................475System tables .............................................................................................................476

Managing User Accounts ................................................................................................ 478GRANT and REVOKE commands ............................................................................481SHOW GRANTS and mk-show-grants .....................................................................485

Resetting the Root Password ..........................................................................................487Windows server .........................................................................................................488Unix-based server ......................................................................................................489

Debugging User Account Problems ...............................................................................490Bad password ............................................................................................................490Access issues ..............................................................................................................491Client does not support authentication protocol .....................................................491Cant connect to local mysqld through socket /path/to/mysqld.sock ....................492I do not have the right permissions! ........................................................................493

Summary ............................................................................................................................494

Chapter 15: Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . 495

Learning about Partitioning ...........................................................................................495Partitioning Tables .......................................................................................................... 496

RANGE partitioning ..................................................................................................497LIST partitioning .......................................................................................................502HASH partitioning .....................................................................................................503KEY partitioning ........................................................................................................504Composite partitioning .............................................................................................504Partition management commands ............................................................................507Restrictions of partitioning ........................................................................................510

MERGE Tables ..................................................................................................................510Creating a MERGE table ...........................................................................................511Changing a MERGE table .........................................................................................512Advantages of MERGE tables ....................................................................................513

Partitioning with MySQL Cluster ..................................................................................513Programmatic Partitioning .............................................................................................514Summary ............................................................................................................................514

xxii

Contents

Chapter 16: Logging and Replication . . . . . . . . . . . . . . . . . . . 517Log Files .............................................................................................................................517

Error log ....................................................................................................................517Binary logs .................................................................................................................518Relay logs ...................................................................................................................520General and slow query logs ....................................................................................520Rotating logs ..............................................................................................................522Other methods of rotating ........................................................................................523

Replication ........................................................................................................................ 524Setting up semisynchronous replication ...................................................................525Statement-based, row-based, and mixed-based replication .....................................527

Replication Configurations .............................................................................................529Simple replication ..................................................................................................... 529CHANGE MASTER statement ...................................................................................534More complex setups ................................................................................................534Additional replication configuration options ............................................................539

Correcting Data Drift ......................................................................................................540mk-table-checksum overview ................................................................................... 540mk-table-sync overview ............................................................................................542Putting this together ..................................................................................................542

Summary ............................................................................................................................543

Chapter 17: Measuring Performance . . . . . . . . . . . . . . . . . . . 545Benchmarking ...................................................................................................................546

mysqlslap ...................................................................................................................547SysBench ....................................................................................................................552Benchmarking recommendations ..............................................................................565

Profiling .............................................................................................................................566SHOW GLOBAL STATUS .........................................................................................566mysqltuner .................................................................................................................568mysqlreport ...............................................................................................................572mk-query-profiler ......................................................................................................580mysqldumpslow ........................................................................................................583

Capacity Planning ............................................................................................................585Summary ............................................................................................................................585

Part IV Extending Your Skills

Chapter 18: Query Analysis and Index Tuning . . . . . . . . . . . . . . 589Using EXPLAIN .................................................................................................................590

EXPLAIN plan basics ................................................................................................590Data access strategy ...................................................................................................596EXPLAIN plan indexes ..............................................................................................606Rows ..........................................................................................................................607

xxiii

Contents

Extra .......................................................................................................................... 608Subqueries and EXPLAIN .........................................................................................611EXPLAIN EXTENDED ..............................................................................................612

EXPLAIN on Non-SELECT Statements .........................................................................614Other Query Analysis Tools ...........................................................................................614Optimizing Queries ..........................................................................................................615

Factors affecting key usage .......................................................................................615Optimizer hints .........................................................................................................616Adding an Index ........................................................................................................616Optimizing away Using temporary ...........................................................................620Using an index by eliminating functions .................................................................623Non-index schema changes ......................................................................................626Batching expensive operations ..................................................................................628Optimizing frequent operations ................................................................................629

Summary ............................................................................................................................631

Chapter 19: Monitoring Your Systems . . . . . . . . . . . . . . . . . . 633

Deciding What to Monitor ..............................................................................................634Examining Open Source Monitoring .............................................................................636

Nagios ........................................................................................................................636Cacti ...........................................................................................................................637Hyperic HQ ...............................................................................................................638OpenNMS ..................................................................................................................640Zenoss Core ...............................................................................................................641Munin ........................................................................................................................642Monit .........................................................................................................................643

Examining Commercial Monitoring ..............................................................................644MySQL enterprise monitor .......................................................................................644MONyog ....................................................................................................................645

Summary ............................................................................................................................646

Chapter 20: Securing MySQL . . . . . . . . . . . . . . . . . . . . . . . 649

Access Control Lists ........................................................................................................649Wildcards and blank values ......................................................................................650Privilege and privilege levels .....................................................................................651

Accessing the Operating System ....................................................................................654Database access ..........................................................................................................654Changing MySQL connectivity defaults ...................................................................654Operating system login .............................................................................................654

Securing Backups and Logs ............................................................................................656Data Security ....................................................................................................................656

Data flow ...................................................................................................................657Encrypted connectivity ..............................................................................................659Data security using MySQL objects ..........................................................................664

xxiv

Contents

Creating Security Policies ...............................................................................................665Summary ............................................................................................................................666

Chapter 21: The MySQL Data Dictionary . . . . . . . . . . . . . . . . . 667Object Catalog ..................................................................................................................668

SCHEMATA ...............................................................................................................668TABLES ......................................................................................................................670VIEWS .......................................................................................................................674COLUMNS ................................................................................................................ 676STATISTICS ...............................................................................................................679TABLE_CONSTRAINTS ............................................................................................681KEY_COLUMN_USAGE ...........................................................................................682REFERENTIAL_CONSTRAINTS ...............................................................................684TRIGGERS .................................................................................................................685ROUTINES ................................................................................................................686PARAMETERS ...........................................................................................................690EVENTS .....................................................................................................................691PARTITIONS .............................................................................................................693

System Information .........................................................................................................695CHARACTER_SETS .................................................................................................. 695COLLATIONS ...........................................................................................................696COLLATION_CHARACTER_SET_APPLICABILITY .................................................696ENGINES ...................................................................................................................697PLUGINS ...................................................................................................................697PROCESSLIST ...........................................................................................................698PROFILING ...............................................................................................................709GLOBAL_VARIABLES ...............................................................................................710SESSION_VARIABLES ...............................................................................................710GLOBAL_STATUS .....................................................................................................711SESSION_STATUS ....................................................................................................711

Displaying Permissions ...................................................................................................711COLUMN_PRIVILEGES ............................................................................................712TABLE_PRIVILEGES .................................................................................................713SCHEMA_PRIVILEGES .............................................................................................714USER_PRIVILEGES ...................................................................................................715

Storage Engine-Specific Metadata .................................................................................716Custom Metadata .............................................................................................................716

Defining the plugin ...................................................................................................716Compiling the plugin ................................................................................................722Installing the plugin ..................................................................................................724

Summary ............................................................................................................................725

Chapter 22: Scaling and High Availability Architectures . . . . . . . . . 727Replication ........................................................................................................................ 728

One read slave ...........................................................................................................729Promoting a new master ...........................................................................................729

xxv

Contents

Many read slaves .......................................................................................................734Master/master replication ..........................................................................................735Circular replication ...................................................................................................736

SAN .....................................................................................................................................737DRBD ..................................................................................................................................738

MySQL and DRBD setup ..........................................................................................738MySQL Proxy ....................................................................................................................739

Scaling read queries ..................................................................................................740Automated failover ....................................................................................................740Read/write splitting ...................................................................................................742Sharding .....................................................................................................................742

Linux-HA Heartbeat .........................................................................................................742MySQL Cluster ................................................................................................................. 744Connection Pooling ..........................................................................................................746memcached ........................................................................................................................747Summary ............................................................................................................................748

Appendix A: MySQL Proxy . . . . . . . . . . . . . . . . . . . . . . . . 749

Appendix B: Functions and Operators . . . . . . . . . . . . . . . . . . 783

Appendix C: Resources . . . . . . . . . . . . . . . . . . . . . . . . . . 813

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 821

xxvi

Welcome to the MySQL Administrators Bible. Like all books in the Bible series, youcan expect to find both hands-on tutorials and real-world practical application infor-mation, as well as reference and background information that provide a context forwhat you are learning. This book is a comprehensive resource on MySQL database administra-tion. It covers almost every conceivable topic of database administration from the very basics,including server installation and user management, to some fairly complex topics such as securityand high availability. By the time you have completed the MySQL Administrators Bible you willbe well-prepared to perform as a database administrator.

This book fully covers MySQL 5.1 plus early features of the 6.0 release (as of the time of writing).All of the examples work in both MySQL 5.1 and MySQL 6.0, except where noted.

Who Should Read This BookThis book is designed for two groups of people. The first group of people is those who havenever worked on a database server and want to learn the fundamentals of database managementusing the worlds number one open source database server. The second group of people is thosewho are currently using another database server and are looking to make a transition to MySQLserver. They may be experienced database administrators, but are not experienced with MySQL.

How This Book Is OrganizedThe MySQL Administrators Bible is divided into four parts.

Part I makes no assumptions on the knowledge level of the reader. Chapter 1 is a brief introduc-tion to MySQLs history. After this introduction Part I covers:

MySQL server installation (Chapter 2)

Accessing MySQL server (Chapter 3)

Part II covers how MySQL compares to standards and MySQLs SQL syntax:

How MySQL extends and deviates from standard SQL (Chapter 4)

MySQL data types (Chapter 5)

MySQL index types (Chapter 6)

xxvii

Introduction

Stored routines, triggers and events (Chapter 7)

MySQL Views (Chapter 8)

Transactions in MySQL (Chapter 9)

Part III is all about MySQL administration:

Tuning the server (Chapter 10)

MySQL storage engines (Chapter 11)

Caching with MySQL (Chapter 12)

Backups, recovery and disaster planning (Chapter 13)

Managing MySQL users (Chapter 14)

Partitioning in MySQL server (Chapter 15)

Logging and replication (Chapter 16)

Measuring performance (Chapter 17)

Part IV highlights intermediate topics ranging from further administrative tasks to performancetuning:

Query analysis and index tuning (Chapter 18)

Monitoring MySQL servers (Chapter 19)

Securing MySQL (Chapter 20)

The MySQL data dictionary (Chapter 21)

Scaling MySQL server and high availability (Chapter 22)

In addition to the two parts there are three appendixes of additional material:

Using the MySQL Proxy (Appendix A)

MySQL Functions reference (Appendix B)

Additional resources for MySQL (Appendix C)

Conventions and FeaturesThere are many different organizational and typographical features throughout this book designedto help you get the most of the information.

Tips, Notes, and CautionsWhenever the authors want to bring something important to your attention the information willappear in a Tip, Note, Caution or On the Website.

This information is important and is set off in a separate paragraph with a specialicon. Cautions provide information about things to watch out for, whether simply

inconvenient or potentially hazardous to your data or systems.

xxviii

Introduction

Tips generally are used to provide information that can make your workeasier special shortcuts or methods for doing something easier than the norm.

Notes provide additional, ancillary information that is helpful, but somewhat outsideof the current presentation of information.

ON the WEBSITEON the WEBSITE On the Website points the reader to the companion website (www.wiley.com/go/mysqladminbible). The website contains new material and up to-date infor-mation about evolving MySQL features.

Whats on the Companion WebsiteOn the companion website (www.wiley.com/go/mysqladminbible), you will find thefollowing:

Sample code each chapter has its own subfolder on the website and youll find all thecode output that was discussed in each chapter organized accordingly.

Examples that work in both MySQL 5.1 and MySQL 6.0 (as released at the time of writ-ing), except where noted.

Extra material such as a new SQL tutorial, information on spatial data and indexes, andup-to-date information on the new features in MySQL 6.0.

Errata updates.

Where To Go From HereWhen you finish reading MySQL Administrators Bible you will have a solid foundation for work-ing as a MySQL database administrator. While it takes experience to produce a quality databaseadministrator, knowledge and understanding of foundational principals is critical. Our goal inthis book is to provide those two parts of the equation.

You will definitely want to check out the companion website (www.wiley.com/go/mysqladminbible). MySQL 6.0 is still new technology at the time of publication of this bookand it will continue to evolve for some time. The world of MySQL changes quickly and theinformation in the book is up-to-date at time of publication. However, as more changes occur,we will write about the latest changes on the website.

In addition there are many other resources available as you grow in your knowledge of MySQLserver. Many of these resources are listed in Appendix C of the book but we would point out inparticular Planet MySQL (http://www.planetmysql.org), a blog aggregation site, and MySQLMagazine (http://www.mysqlzine.net), a free digital magazine with which the co-authors areinvolved.

xxix

MySQL

AdministratorsBible

First Steps withMySQL

IN THIS PART

Chapter 1Introduction to MySQL

Chapter 2Installing and Upgrading MySQLServer

Chapter 3Accessing MySQL

Introduction to MySQL

IN THIS CHAPTERLearning MySQLs history

Finding the MySQL community

Contributing to MySQL

Everyone who has been involved with IT for more than a fewmonths has at least heard of MySQL. The acquisition of MySQL ABby Sun Microsystems brought a great deal of additional attentionto MySQLs database management system (DBMS). Even so, there is oftenmore to MySQL than many people realize. They simply do not realize thefull capabilities of MySQL.

Recent versions of MySQL have brought a large feature set that covers justabout every imaginable need. This includes partitions, scheduled events,prepared statements, triggers, and views. MySQL has long been used indynamic websites and applications. Whether you program in Perl, PHP,ASP, .NET or Ruby you can integrate MySQL into your environment.

MySQL Mission Speed,Reliability, and Ease of UseThe driving force behind MySQL has been to provide a reliable,high-performance server that is easy to set up and use. These qualities arewhy many Internet companies in the late 1990s chose MySQL to powertheir websites. These same qualities are why MySQL is making stronginroads into the internal database servers of Fortune 1000 companies thathave traditionally used commercial databases. MySQL did not corner theexisting database market; instead, MySQL allowed the database market toexpand and grow to include people all over the world. MySQL created aniche by creating the opportunity for almost anyone to be able use a

3

Part I First Steps with MySQL

database. Had there not been an easy-to-use, fast, reliable database server such as MySQL, therewould be a dearth of the user-provided content and collaborative efforts that are what we expectfrom the Internet. Without MySQL, there would be far fewer articles, podcasts, online zines,tutorials, photos, forums, videos, collaborative reference material, and search engines. The World(Wide Web) as we know it would be completely different.

MySQL is not the only free database management system; it also is not the only open sourcedatabase management system. One of the largest differences is the user friendliness thatpervades MySQL. The friendliness, starting with the cost free unless embedded in anotherproduct shines through the quick installation and setup, and pleases the new database userwith SQL language extensions that are nearly intuitive. For example, the SHOW DATABASEScommand shows a list of databases you have permission to see.

Experienced database administrators can install, configure, and bring a MySQL server online inless than fifteen minutes. If the installation process is packaged it can be done in five minutes.We look at the installation process in more detail in Chapter 2.

The reliability of MySQL played a role in MySQLs rise to become, as stated at www.mysql.com,the worlds most popular open source database. In addition to this reliability, MySQL operateson a wide range of hardware and operating systems from laptops to multi-core servers. Thoughbenchmark comparisons can be slanted to benefit one server or another, the various benchmarksavailable show that MySQL competes with and frequently beats competing database servers.

Company backgroundMySQL server has been downloaded more than 100 million times. MySQL is open sourcesoftware. An admittedly simple definition of open source software is software that is freelyavailable (including source code) with free redistribution. Source code is the source of aprogram the file(s) containing the original programming language code, which can be read,discussed, and learned from, just as the words of a book can. The roots of MySQL server arefound in a database system called Unireg that was developed by Michael Monty Wideniusfor a Swedish company called TcX during the 1980s. The initial release of MySQL server wascreated in 1995 when Monty added an SQL interface to Unireg. Shortly after, David Axmarkrecommended MySQL server be released under a dual licensing model, where it would beavailable for widespread free use but could also be used in situations that require a morerestrictive licensing use (such as in embedded devices). David and Monty, together with AllanLarsson, founded MySQL AB in 1995. MySQL AB was the company that, until its acquisitionby Sun Microsystems in January 2008, provided support and service for the MySQL database inaddition to developing most of the code.

In 2001 MySQL began supporting transactions with the integration of the BDB and InnoDBengines. This allowed for safer handling of concurrent write operations, which began the trendof adding features needed by enterprise environments.

Over the years the feature set of the MySQL server has grown to cover almost any featureneeded in an enterprise database server. Some might even argue that it has too many features!Innovation has continued; within months of readily accessible cloud computing environments

4

Introduction to MySQL 1

such as Amazon Web Services (http://aws.amzon.com), hobbyists and companies alike aredeploying and using MySQL in creative environments.

Table 1-1 shows an overview of MySQL servers release history, including important pre-release(Alpha and Beta) unstable versions as well as public stable releases that were determined to begenerally available (GA). In Table 1-1, only the first release notes the new features, though allfeatures mentioned in the pre-release Beta version were also released in the GA version.

TABLE 1-1

MySQL Server Timeline

Version Release Date Release Level New Features

Internal release May 23, 1995

Public Release August 31, 1996 Binaries for Solaris

Windows Release January 8, 1998 Windows 95 & Windows NT

3.23 June 2000 Beta BDB and InnoDB

3.23 January 2001 GA

4.0 August 2002 Beta Unions

4.0 March 2003 GA

4.1 June 2004 Beta Subqueries, prepared statements,B-tree and R-tree indexes

4.1 October 2004 GA

5.0 March 2005 Beta Stored procedure, cursors, views,triggers, and XA transactions.

5.0 October 2005 GA

5.1 May 2006 Beta Partitioning, row-based replication,plug-in storage engine API, and eventscheduler

5.1 November 2008 GA

6.0 April 2007 Alpha Falcon and Maria storage engines andonline backup

Community and Enterprise server versionsIn August of 2007 MySQL AB began offering the MySQL server in two different versions:MySQL Community and MySQL Enterprise. Although the code base of the two servers is sim-ilar, there are different levels of support for the servers. With MySQL Enterprise you purchase

5

Part I First Steps with MySQL

one of four support packages in addition to receiving the MySQL server binaries and sourcecode. Sun has a table of available options at www.mysql.com/products/enterprise/features.html.

MySQL Community offers freely downloadable binaries with no support guarantee. When adatabase experiences problems the user is on his or her own to resolve those problems.

MySQL Enterprise has more frequent releases than MySQL Community Server. Owners of thesource code must be allowed to redistribute the source code in full this is required under theprovisions of the GPLv2 license that governs MySQL distributions.

The MySQL CommunityThe MySQL community is a diverse population of both users of and contributors to MySQL,spread across the world. The large user community brings many opinions about what featuresare needed and discussions about the best methods to implement those features. Though thiscan sometimes bring about somewhat heated discussions, it also allows for honest and directfeedback that many companies may not hear.

How to contributeYou can contribute to MySQL server in a number of ways:

Blogging: Many people who work on MySQL either full or part time write about theirexperiences online. These blogs can be valuable for learning more about the ins and outsof working with MySQL. Many of these blogs are aggregated at the Planet MySQL website(www.planetmysql.org).

Mailing lists: The official public MySQL mailing lists are available at http://lists.mysql.com. The mailing lists are a great way to both contribute your knowledge andlearn something new. The General Discussion list, in particular, has a wide range of topics.Dive in and take a look.

IRC channels: Various IRC channels are available. These can often provide instant feed-back and help with problems. One of the most popular is the #mysql channel on theFreenode IRC network (irc.freenode.net, or visit http://freenode.net for moreinformation).

User groups: Many active user groups exist around the world. It is a great experience toget together with other people who share a similar passion. Each month the user groupswill have someone present on a topic relating to MySQL. One month it might be backupsand recovery. The next month it might be about how to scale an application effectively.The place to look for a user group near you is www.meetup.com. If there is not a usergroup near you, consider starting one!

Documentation: MySQL has some of the best software documentation available, locatedat http://dev.mysql.com/doc. Working on the documentation is the easiest way todirectly contribute to the MySQL project. As MySQL continues to change it is necessary tokeep the documentation up to date, so this is always an ongoing project.

6

Introduction to MySQL 1

Code: The MySQL server code is complex software. However, it is somewhat modularand there are areas where an experienced C or C++ coder can help out. For smallerprojects you might want to investigate the MySQL forge (http://forge.mysql.com)where many projects find a home. These projects are in many coding languages including(among others) Perl, Python, PHP, bash, C, and even Ruby.

MySQL Magazine: Do you like to write? Then MySQL Magazine (www.mysqlzine.net)might be your venue. Articles cover anything from coding to how-to articles on normalDBA activities. The audience is a large group of the best database administrators and devel-opers in the world.

Reasons to contributeMany people never do anything other than download MySQL server, read a tutorial or two, andinstall the software and use it. They never contribute in any manner. That is fine, but we thinkthat you can have a far more positive experience by becoming part of the community and con-tributing what you can while learning about this fascinating server software.

Though it is beyond the scope of this book to discuss the open source software philosophy indepth, openly and freely giving and receiving knowledge is the basis of this philosophy. Bothcode and knowledge are free to be shared. By doing so, everyone benefits from the exchange.

SummaryMySQL server has a long history of innovation. The community that has grown around MySQLprovides learning experiences and teaching opportunities for database administrators old andyoung, veteran and newbie alike.

This chapter covered:

MySQL company history

The original and continuing mission of the MySQL database

MySQL server version and feature overview

Where to find more information

How you can contribute to MySQL server

The MySQL Administrators Bible covers all the topics you need to understand as a beginningMySQL database administrator. If you are a more advanced administrator switching fromanother database system, this book offers a unique look at where MySQL is different from theISO SQL:2003 standard. After advancing to an intermediate level of familiarity with MySQL, thematerial on scaling and high availability, replication, metadata, and server tuning will continueto provide guidance.

7

Installing and UpgradingMySQL Server

IN THIS CHAPTERPre-installation

Installation

Initial Configuration

Upgrading

Troubleshooting Installation

The MySQL server has two installation formats, and the MySQLserver itself runs on more than sixteen different operating systemplatforms. The top three operating system platforms for MySQL arecovered in some depth: GNU/Linux, Microsoft Windows, and Sun Solaris.

For information on how to install the MySQL serveron Mac OS X, see the MySQL manual page at

http://dev.mysql.com/doc/refman/6.0/en/mac-os-x-installation.html.

This chapter takes you through the bare bones of what you need to get theMySQL server (also known as mysqld) up and running, including the ini-tial configuration. For more complete configuration tuning, see Chapter 10.It then delves into upgrading mysqld, which is very similar to installation.The chapter ends with troubleshooting assistance.

Before InstallationBefore installing, you must choose what to install. You have twobasic installa