Оптимизация MySQL. Что должен знать каждый разработчик

MySQL:Основы оптимизации

илиЧто должен знать каждый

разработчик

Агнислав Онуфрийчук

Обо мне

Агнислав Онуфрийчук Веб-разработчик

Обо мне

Агнислав Онуфрийчук Веб-разработчик Люблю девушек

Обо мне

Агнислав Онуфрийчук Веб-разработчик Люблю девушек Не люблю MS Internet Exporer и Opera

Обо мне

Кто работает с MySQL?

О вас

Кто работает с MySQL? Кто использует MyISAM?

О вас

Кто работает с MySQL? Кто использует MyISAM? Кто ещё не использует InnoDB?

О вас

Кто работает с MySQL? Кто использует MyISAM? Кто ещё не использует InnoDB? Кто не знает про my.ini/my.cnf?

О вас

Здесь знают толк в оптимизации MySQL

О PortaOne

Здесь знают толк в оптимизации MySQL Здесь знают толк в девушках

О PortaOne

Здесь знают толк в оптимизации MySQL Здесь знают толк в девушках Здесь знают толк в программистах

О PortaOne

Здесь знают толк в оптимизации MySQL Здесь знают толк в девушках Здесь знают толк в программистах Здесь знают толк в after-parties.

О PortaOne

Здесь знают толк в оптимизации MySQL Здесь знают толк в девушках Здесь знают толк в программистах Здесь знают толк в after-parties.

О PortaOne

Начнём?

Оптимизация MySQL

Всё плохо.

Оптимизация MySQL

Сегодня мы не поговорим о…

Шардинге

Сегодня мы не поговорим о…

Шардинге Железе

Сегодня мы не поговорим о…

Шардинге Железе Операционных системах

Сегодня мы не поговорим о…

Шардинге Железе Операционных системах Файловых системах

Сегодня мы не поговорим о…

Сегодня мы поговорим о…

Движках (engines)

Сегодня мы поговорим о…

Движках (engines) Базовых настройках

Сегодня мы поговорим о…

Движках (engines) Базовых настройках Полезных утилитах

Сегодня мы поговорим о…

Движках (engines) Базовых настройках Полезных утилитах Запросах

Сегодня мы поговорим о…

Движках (engines) Базовых настройках Полезных утилитах Запросах И о чём-нибудь приятном

Сегодня мы поговорим о…

Engines Settings Utilities Queries Girls

Roadmap

Engines Settings Utilities Queries Girls

Roadmap

Engines Settings Utilities Queries

Roadmap

Engines Settings Utilities Queries

Roadmap

Engines

WTF?

Engines

WTF? It’s cool!

Engines

InnoDB/XtraDB MyISAM/Maria Memory NDB CSV Blackhole

Engines

Extremely fast Not-transactional Table-level locking No BLOB/TEXT data types

Engines: Memory

Extremely fast Transactional Row-level locking Good scaling

capabilities

Engines: NDB

Simple data exchange

No table partitioning No NULL values

(since 5.1.23)

Engines: CSV

Extremely fast (faster then Memory/NDB) Transactional All types of indexes Low-cost storage

Engines: Blackhole

Default for years (up to 5.5) Non-Transactional Table-level locking Full-text search indexes Fast read

Engines: MyISAM

Transactional Row-level locking Foreign keys Simple performance tuning

Engines: InnoDB

All InnoDB feature plus: “Out-of-box” better performance Most powerful performance tuning “Troubleshoot without guesswork”

Engines: XtraDB

Engines Settings Utilities Queries

Roadmap

There hundreds of them…

Settings

There hundreds of them… but a couple of them are enough for us

Settings

Using memory is better than using disk

The main point is…

query_cache_size

Does matter for SELECT-dedicated server

32M is generally enough

Settings

innodb_buffer_pool_size

Give all you can:

Servers: 50-75% of available memoryDev PC: 25-50% of available memory

Settings

innodb_log_file_size

Make your log files big.

128M/256M is a good value

Settings

innodb_log_buffer_size

How often we’ll flush log data to disk

4M is a good value unless you’re piping large blobs.

Settings

innodb_flush_log_at_trx_commit

0 – write log buffer to file and flush it to disk every second

1 – write&flush to disk for every transaction

2 – write for every transaction and flush every second

Settings

innodb_thread_concurrency=8

Just use this value

Settings

innodb_flush_method=O_DIRECT

Avoid double buffering

Settings

innodb_file_per_table

Use it if you don’t have tons of tables

Settings

Engines Settings Utilities Queries

Roadmap

tuning-primer.sh mysql-tuner.pl maatkit innotop

Utilities

http://www.day32.com/MySQL/

tuning-primer.sh

SLOW QUERIES

Current long_query_time = 10 sec.

You have 526 out of 36204146 that take longer than 10 sec. to complete

The slow query log is NOT enabled.

Your long_query_time may be too high, I typically set this under 5 sec.

tuning-primer.sh

QUERY CACHEQuery cache is enabledCurrent query_cache_size = 8 MCurrent query_cache_used = 7 MCurrent query_cach_limit = 1 MCurrent Query cache fill ratio = 89.38 %However, 254246 queries have been removed from the query cache due to lack of memoryPerhaps you should raise query_cache_sizeMySQL won't cache query results that are larger than query_cache_limit in size 

tuning-primer.sh

TEMP TABLES

Current max_heap_table_size = 16 M

Current tmp_table_size = 32 M

Of 35170 temp tables, 74% were created on disk

Effective in-memory tmp_table_size is limited to max_heap_table_size.

Perhaps you should increase your tmp_table_size and/or max_heap_table_size to reduce the number of disk-based temporary tables

Note! BLOB and TEXT columns are not allow in memory tables.

If you are using these columns raising these values might not impact your ratio of on disk temp tables.

tuning-primer.sh

http://mysql-tuner.pl

mysql-tuner.pl

-------- Storage Engine Statistics ---------------------

[--] Status: -Archive +BDB -Federated +InnoDB -ISAM -NDBCluster

[--] Data in MyISAM tables: 19M (Tables: 90)

[!!] InnoDB is enabled but isn't being used

[!!] BDB is enabled but isn't being used

[!!] Total fragmented tables: 18

mysql-tuner.pl

-------- Performance Metrics ---------------------------

[--] Up for: 16m 37s (6K q [6.059 qps], 146 conn, TX: 54M, RX: 665K)

[--] Reads / Writes: 62% / 38%

[--] Total buffers: 298.0M global + 6.3M per thread (100 max threads)

[OK] Maximum possible memory usage: 929.2M (26% of installed RAM)

[OK] Slow queries: 0% (0/6K)

[OK] Highest usage of available connections: 5% (5/100)

[OK] Key buffer size / total MyISAM indexes: 256.0M/2.3M

[!!] Key buffer hit rate: 91.3% (1K cached / 101 reads)

[OK] Query cache efficiency: 97.6% (5K cached / 5K selects)

mysql-tuner.pl

-------- Recommendations -------------------------------

General recommendations:

Add skip-innodb to MySQL configuration to disable InnoDB

Add skip-bdb to MySQL configuration to disable BDB

Run OPTIMIZE TABLE to defragment tables for better performance

MySQL started within last 24 hours - recommendations may be inaccurate

Enable the slow query log to troubleshoot bad queries

mysql-tuner.pl

maatkit http://www.maatkit.org/

maatkit mk-error-log mk-log-player mk-index-usage mk-query-advisor mk-query-digest mk-query-profiler … (>30)

maatkit: mk-error-log Count Level Message

===== ======= =======================================

5 info mysqld started

4 info mysqld version info

3 info InnoDB: Started

2 info mysqld ended

1 unknown Number of processes running now: 0

1 error [ERROR] /usr/sbin/mysqld: unknown variable 'ssl-ke

1 error [ERROR] Failed to initialize the master info struc

maatkit: mk-log-playermk-log-player does two things: it splits MySQL query logs into session files and it plays (executes) queries in session files on a MySQL server.

maatkit: mk-index-usageThis tool connects to a MySQL database server, reads through a query log, and uses EXPLAIN to ask MySQL how it will use each query. When it is finished, it prints out a report on indexes that the queries didn't use.

maatkit: mk-query-advisormk-query-advisor examines queries and applies rules to them, trying to find queries that look bad according to the rules. It reports on queries that match the rules, so you can find bad practices or hidden problems in your SQL.

maatkit: mk-query-digest # pct total min max avg 95% stddev median

# Count 0 2

# Exec time 13 1105s 552s 554s 553s 554s 2s 553s

# Lock time 0 216us 99us 117us 108us 117us 12us 108us

# Rows sent 20 6.26M 3.13M 3.13M 3.13M 3.13M 12.73 3.13M

# Rows exam 0 6.26M 3.13M 3.13M 3.13M 3.13M 12.73 3.13M

…

# Query_time distribution

# 1us

# 10us

# 100us

# 1ms

# 10ms

# 100ms

# 1s

# 10s+ #############################################################

maatkit: mk-query-profilermk-query-profiler reads a file containing one or more SQL statements or shell commands, executes them, and analyzes the output of SHOW STATUS afterwards. It then prints statistics about how the batch performed. For example, it can show how many table scans the batch caused, how many page reads, how many temporary tables, and so forth.

http://code.google.com/p/innotop/

innotop

Engines Settings Utilities Queries

Roadmap

Queries

use ORMs use Indexes be concrete (limit yourself) get a map!

Queries

why?

use ORMs

experience safety clear code speeding development portability

use ORMs

why?

use Indexes

CREATE TABLE employee ( employee_number char(10) NOT NULL, firstname varchar(40), surname varchar(40), address text, tel_no varchar(25), salary int(11), overtime_rate int(10) NOT NULL);

use Indexes

use Indexes

NOT using PRIMARY KEY Using PRIMARY KEY

id: 1

select_type: SIMPLE

table: employee

type: ALL

possible_keys: NULL

key: NULL

key_len: NULL

ref: NULL

rows: 10000

Extra: Using where

id: 1

select_type: SIMPLE

table: employee

type: const

possible_keys: PRIMARY

key: PRIMARY

key_len: 4

ref: const

rows: 1

Extra:

EXPLAIN SELECT employee_number, firstname, surname FROM employee WHERE id=1000\G

use IndexesALTER TABLE employee ADD INDEX(surname, firstname);

SELECT overtime_rate FROM employeeWHERE surname='Madida';

SELECT overtime_rate FROM employee WHERE firstname='Mpho';

SELECT overtime_rate FROM employee WHERE surname='Madida' and firstname="Mpho";

SELECT overtime_rate FROM employee WHERE firstname="Mpho" and surname='Madida';

use IndexesALTER TABLE employee ADD INDEX(surname, firstname);

SELECT overtime_rate FROM employeeWHERE surname='Madida';

SELECT overtime_rate FROM employee WHERE firstname='Mpho';

SELECT overtime_rate FROM employee WHERE surname='Madida' and firstname="Mpho";

SELECT overtime_rate FROM employee WHERE firstname="Mpho" and surname='Madida';

use Indexes and be concreteselect … WHERE year(my_date) > 2010select … where email LIKE “%@gmail.com”select * from table;select * from table limit 10000;select col1 from table where id > 0;

insert into table values(‘1’, ‘bla’);

create table `ids` ( `id` int(11) NOT NULL AUTO_INCREMENT, `id1` int(11) NOT NULL, `id2` int(11) DEFAULT NULL, …, PRIMARY KEY (`id`) ) ENGINE=InnoDB;

SELECT COUNT(*) FROM `ids`; SELECT COUNT(`id`) FROM `ids`; SELECT COUNT(`id1`) FROM `ids`; SELECT COUNT(`id2`) FROM `ids`;

get a map!

Engines Settings Utilities Queries

Roadmap

Questions?

Contacts

vcard agnislav @

gmailskypetwitter

facebookvkontaktelivejournal