implementing mysql security dublin 2017 percona live ... · #perconalive @ronaldbradford @bytebot...
TRANSCRIPT
#PerconaLive @RonaldBradford @bytebot
Implementing MySQL Security Features
Ronald Bradford, Colin CharlesPercona Live Europe
Dublin 2017
#PerconaLive @RonaldBradford @bytebot
License● Creative Commons BY-NC-SA 4.0● https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode
#PerconaLive @RonaldBradford @bytebot
About: Ronald Bradford● Senior MySQL Database Consultant at Pythian
● Author/Blogger/Speaker
● Oracle ACE Director 2010 - present
● MySQL Community Member of the Year Award winner 09, 13
● Formally MySQL Inc 06-08, Oracle Corporation 96-99
● http://ronaldbradford.com/presentations/
● http://effectivemysql.com
R
#PerconaLive @RonaldBradford @bytebot
About: Colin Charles● Chief Evangelist (in the CTO office), Percona Inc
● Founding team of MariaDB Server (2009-2016), previously at Monty Program
Ab, merged with SkySQL Ab, now MariaDB Corporation
● Formerly MySQL AB (exit: Sun Microsystems)
● Past lives include Fedora Project (FESCO), OpenOffice.org
● MySQL Community Contributor of the Year Award winner 2014
● http://bytebot.net/blog/
#PerconaLive @RonaldBradford @bytebot
Agenda● A Security Primer
● Available ANSI and MySQL constructs
● Evolution of defaults in MySQL Versions
● Basic security hardening
● Reviewing authentication modules
● Better/Stronger/Harder passwords
● Encryption options for variants Percona
Server, MariaDB, MySQL Enterprise
including LUKS, ezcrypt, Gazzang
● Key management options
● Practical OS security (user accounts,
sudo, iptables/ufw, SELinux/Appamor)
● Why auditing is important
● Understanding the role of Clouds,
Containers and Compliance
#PerconaLive @RonaldBradford @bytebot
Let’s get setup● Vagrant and Virtualbox is a good choice
○ https://github.com/ronaldbradford/mysql-security-tutorial
● Docker
● You can also naturally just have a MySQL instance on your computer or cloud
Vagrant/VirtualBox on USB
$ wget https://bit.ly/dockerhelper $ source dockerhelper $ docker_mysql | docker_percona | docker_mariadb
#PerconaLive @RonaldBradford @bytebot
MySQL Variants and Versions… quick history lesson● MySQL Community Edition
○ Won’t cover: 3.23, 4.0, 4.1, 5.0, 5.1○ Will focus on: 5.5, 5.6, 5.7, with present development 8.0 version
● MySQL Enterprise Edition● Percona Server for MySQL
○ 5.5, 5.6, 5.7
● MariaDB Server○ Won’t cover: 5.1, 5.2, 5.3○ 5.5, 10.0, 10.1, 10.2, with 10.3 as an alpha
● What we won’t cover: MySQL Cluster (NDBCLUSTER), Galera Cluster, Group Replication/InnoDB Cluster, X Protocol/mysqlsh (33060)
#PerconaLive @RonaldBradford @bytebot
Structured Query Language (SQL)● ISO/IEC 9075 (reviewed every 5 years)● SQL-86● SQL-89● SQL-92● SQL:1999● SQL:2003● SQL:2006● SQL:2008● SQL:2011● SQL:2016
select @@global.sql_mode;
● ANSI - come close to the SQL standard● STRICT_TRANS_TABLES - If a value
could not be inserted as given into a transactional table, abort the statement.
● TRADITIONAL - “give an error instead of a warning” when inserting an incorrect value into a column.
● https://dev.mysql.com/doc/refman/5.7/en/sql-mode.html
Deprecated - MariaDB Server has NO_AUTO_CREATE_USER but MySQL 5.7 has this in standard sql_mode
#PerconaLive @RonaldBradford @bytebot
MySQL Security by version● GRANT (3.23)● REVOKE (3.23)● SET PASSWORD (3.23)● SHOW GRANTS (3.23)● DROP USER (4.1)● SHOW PRIVILEGES (4.1)● CREATE USER (5.0)● RENAME USER (5.0)● ALTER USER (5.6)
● SHOW CREATE USER (5.7)● CREATE ROLE (8.0)● DROP ROLE (8.0)● SET ROLE (8.0)● SET DEFAULT ROLE (8.0)● N/B: ROLES came to MariaDB Server in
10.0, and the DEFAULT ROLE came in 10.1
#PerconaLive @RonaldBradford @bytebot
mysql.users table● host● user● password (removed in 5.7; still present in MariaDB)● plugin (5.5)● authentication_string (5.5)● password_expired (5.6)● account_locked (5.7)● Create_role_priv (8.0)● Drop_role_priv (8.0)
Comparing mysql.user between MariaDB Server 10.2 and MySQL 5.7
#PerconaLive @RonaldBradford @bytebot
Key Security Features by Version● 5.1 - McAfee Audit plugin● 5.5 - pluggable authentication (MariaDB 5.2
backport), proxy users, changes in mysql.user table, client password warning; Enterprise provided Audit and PAM authentication (present again in Percona Server for MySQL and MariaDB Server)
● 5.6 - encrypted client credentials (mysql_config_editor), sha256_password, password expiry, VALIDATE_PASSWORD_STRENGTH(), --random-passwords (optional random on install), mysql.user password_expired column; Enterprise Firewall
● 5.7 - grep for root password on installation, password expiry every ‘n’ days, user accounts can be locked/unlocked, mysql_ssl_rsa_setup, mysql.user.password removed, super_read_only, at rest tablespace encryption
● 8.0 - roles + mysql.user changes● MariaDB 10.0 - roles, userstats● MariaDB 10.1 - default roles, at rest
table/tablespace encryption, simple_password_check, cracklib_password_check, AWS Key Management plugin
● MariaDB 10.2 - user limits, ed25519 auth● Percona 5.5 - extended SHOW GRANTS, utility
user, userstats● Percona 5.6 - super_read_only
#PerconaLive @RonaldBradford @bytebot
Passwords
R
#PerconaLive @RonaldBradford @bytebot
Installation Default Passwords● 'root' user
○ Pre 5.7 no password○ 5.7 expired random password
● Anonymous users○ Removed in 5.7
R
#PerconaLive @RonaldBradford @bytebot
MySQL Native Password Hashmysql> SELECT PASSWORD('test123') AS passwd;+-------------------------------------------+| passwd |+-------------------------------------------+| *676243218923905CF94CB52A3C9D3EB30CE8E20D |+-------------------------------------------+1 row in set (0.00 sec)
mysql> SELECT CONCAT('*',UPPER( SHA1(UNHEX(SHA1('test123'))))) AS gen_passwd;+-------------------------------------------+| gen_passwd |+-------------------------------------------+| *676243218923905CF94CB52A3C9D3EB30CE8E20D |+-------------------------------------------+1 row in set (0.00 sec)
R
#PerconaLive @RonaldBradford @bytebot
What is a native password?mysql> SELECT CONCAT('*',UPPER(SHA1(UNHEX(SHA1('test123'))))) AS gen_passwd;+-------------------------------------------+| gen_passwd |+-------------------------------------------+| *676243218923905CF94CB52A3C9D3EB30CE8E20D |+-------------------------------------------+1 row in set (0.00 sec)
$ echo -n "test123" | sha1sum | cut -c1-40 | xxd -p -r | sha1sum | cut -c1-40 | tr '[a-z]' '[A-Z]'676243218923905CF94CB52A3C9D3EB30CE8E20D
$ sudo yum install -y vim-common # xxd
R
#PerconaLive @RonaldBradford @bytebot
mysql.user table on the filesystem$ cd /var/lib/mysql/mysql/$ strings user.MYD
localhostrootmysql_native_password...
mysql> SET PASSWORD = PASSWORD('test123');
$ strings user.MYDlocalhost
root*676243218923905CF94CB52A3C9D3EB30CE8E20D...
R
#PerconaLive @RonaldBradford @bytebot
What is SHA?● The Secure Hash Algorithms (SHA) are a family of cryptographic hash
functions● SHA/SHA-1 is 160 bits == 20 bytes == 40 hexadecimal characters
SHA2 is SHA-224, SHA-256, SHA-384, and SHA-512
https://en.wikipedia.org/wiki/Secure_Hash_Algorithms
R
#PerconaLive @RonaldBradford @bytebot
How are passwords stored in MySQL (5.5)mysql55 >SELECT /* 5.5 */ host, user, password, plugin, authentication_string FROM mysql.user;+-----------+------+----------+--------+-----------------------+| host | user | password | plugin | authentication_string |+-----------+------+----------+--------+-----------------------+| localhost | root | | | || mysql55 | root | | | || 127.0.0.1 | root | | | || ::1 | root | | | || localhost | | | | NULL || mysql55 | | | | NULL |+-----------+------+----------+--------+-----------------------+6 rows in set (0.00 sec)
plugin unspecified
R
#PerconaLive @RonaldBradford @bytebot
How are passwords stored in MySQL (5.6)mysql56 >SELECT /* 5.6 */ host, user, password, plugin, authentication_string, password_expired FROM mysql.user;+-----------+------+----------+-----------------------+-----------------------+------------------+| host | user | password | plugin | authentication_string | password_expired |+-----------+------+----------+-----------------------+-----------------------+------------------+| localhost | root | | mysql_native_password | | N || mysql56 | root | | mysql_native_password | | N || 127.0.0.1 | root | | mysql_native_password | | N || ::1 | root | | mysql_native_password | | N || localhost | | | mysql_native_password | NULL | N || mysql56 | | | mysql_native_password | NULL | N |+-----------+------+----------+-----------------------+-----------------------+------------------+6 rows in set (0.00 sec)
plugin specified
R
#PerconaLive @RonaldBradford @bytebot
How are passwords stored in MySQL (5.7)mysql57 >SELECT /* 5.7 */ host, user, plugin, authentication_string, password_expired, password_last_changed, password_lifetime, account_locked FROM mysql.user;+-----------+---------------+-----------------------+-------------------------------------------+------------------+-----------------------+-------------------+----------------+| host | user | plugin | authentication_string | password_expired | password_last_changed | password_lifetime | account_locked |+-----------+---------------+-----------------------+-------------------------------------------+------------------+-----------------------+-------------------+----------------+| localhost | root | mysql_native_password | *E89C1DBB80A00976B61D19025C3081E4B190D8BE | N | 2017-09-03 18:45:43 | NULL | N || localhost | mysql.session | mysql_native_password | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | N | 2017-09-03 18:42:33 | NULL | Y || localhost | mysql.sys | mysql_native_password | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | N | 2017-09-03 18:42:33 | NULL | Y |+-----------+---------------+-----------------------+-------------------------------------------+------------------+-----------------------+-------------------+----------------+3 rows in set (0.01 sec) Password stored in
authentication_string
R
#PerconaLive @RonaldBradford @bytebot
Minimum password policy (5.7)mysql57 > ALTER USER USER() IDENTIFIED BY 'percona';ERROR 1819 (HY000): Your password does not satisfy the current policy requirementsmysql57 > ALTER USER USER() IDENTIFIED BY 'percona123';ERROR 1819 (HY000): Your password does not satisfy the current policy requirementsmysql57 > ALTER USER USER() IDENTIFIED BY 'Percona123';ERROR 1819 (HY000): Your password does not satisfy the current policy requirementsmysql57 > ALTER USER USER() IDENTIFIED BY 'Percona.123';Query OK, 0 rows affected (0.00 sec)mysql57 >SELECT @@validate_password_policy;+----------------------------+| @@validate_password_policy |+----------------------------+| MEDIUM |+----------------------------+
https://dev.mysql.com/doc/refman/5.7/en/validate-password-options-variables.html#sysvar_validate_password_policy
R
#PerconaLive @RonaldBradford @bytebot
VALIDATE_PASSWORD_STRENGTH()mysql> SELECT VALIDATE_PASSWORD_STRENGTH('percona');+---------------------------------------+| validate_password_strength('percona') |+---------------------------------------+| 25 |+---------------------------------------+
mysql> SELECT VALIDATE_PASSWORD_STRENGTH('percona123');+------------------------------------------+| 50 |+------------------------------------------+
mysql> SELECT VALIDATE_PASSWORD_STRENGTH('Percona123');+------------------------------------------+| 50 |+------------------------------------------+
mysql> SELECT VALIDATE_PASSWORD_STRENGTH('Percona.123');+-------------------------------------------+| 100 |+-------------------------------------------+
https://dev.mysql.com/doc/refman/5.7/en/encryption-functions.html#function_validate-password-strength
R
#PerconaLive @RonaldBradford @bytebot
VALIDATE_PASSWORD_STRENGTH()
https://dev.mysql.com/doc/refman/5.7/en/validate-password-options-variables.html
mysql> SHOW GLOBAL VARIABLES LIKE 'validate%';+--------------------------------------+--------+| Variable_name | Value |+--------------------------------------+--------+| validate_password_check_user_name | OFF || validate_password_dictionary_file | || validate_password_length | 8 || validate_password_mixed_case_count | 1 || validate_password_number_count | 1 || validate_password_policy | MEDIUM || validate_password_special_char_count | 1 |+--------------------------------------+--------+7 rows in set (0.01 sec)
R
#PerconaLive @RonaldBradford @bytebot
mysql_native_password format (deprecated in 5.7)
R
mysql56 > SELECT PASSWORD('test123') AS pwd;+-------------------------------------------+| pwd |+-------------------------------------------+| *676243218923905CF94CB52A3C9D3EB30CE8E20D |+-------------------------------------------+1 row in set (0.00 sec)
mysql57 > SELECT PASSWORD('test123') AS pwd;+-------------------------------------------+| pwd |+-------------------------------------------+| *676243218923905CF94CB52A3C9D3EB30CE8E20D |+-------------------------------------------+1 row in set, 1 warning (0.00 sec)
https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_old_passwordshttps://dev.mysql.com/doc/refman/5.7/en/password-hashing.html
#PerconaLive @RonaldBradford @bytebot
Anonymous usersmysql55 $ mysql -e "SELECT VERSION(), USER(), CURRENT_USER()"+-----------+-------------------+----------------+| VERSION() | USER() | CURRENT_USER() |+-----------+-------------------+----------------+| 5.5.57 | vagrant@localhost | @localhost |+-----------+-------------------+----------------+
mysql56 $ mysql -e "SELECT VERSION(), USER(), CURRENT_USER()"+-----------+-------------------+----------------+| VERSION() | USER() | CURRENT_USER() |+-----------+-------------------+----------------+| 5.6.37 | vagrant@localhost | @localhost |+-----------+-------------------+----------------+
mysql57 $ mysql -e "SELECT VERSION(), USER(), CURRENT_USER()"ERROR 1045 (28000): Access denied for user 'vagrant'@'localhost' (using password: NO)
R
#PerconaLive @RonaldBradford @bytebot
What can anonymous users do?mysql55 >SHOW SCHEMAS;+--------------------+| Database |+--------------------+| information_schema || test |+--------------------+
mysql56 >SHOW SCHEMAS;+--------------------+| information_schema |+--------------------+
mysql55 >SELECT table_schema, table_name FROM information_schema.tables;...
mysql55 >SELECT host,user,password FROM mysql.user;ERROR 1142 (42000): SELECT command denied to user ''@'localhost' for table 'user'
R
#PerconaLive @RonaldBradford @bytebot
What can anonymous users do?mysql55 >USE test;mysql55 >CREATE TABLE t1(i1 INT NOT NULL AUTO_INCREMENT PRIMARY KEY, v1 VARCHAR(100) NOT NULL);mysql55 >INSERT INTO t1(i1, v1) VALUES (1, REPEAT('abcde',20));mysql55 >INSERT INTO t1(i1, v1) SELECT NULL, a.v1 FROM t1 a, t1 b, t1 c;Query OK, 1 row affected (0.00 sec)mysql55 >INSERT INTO t1(i1, v1) SELECT NULL, a.v1 FROM t1 a, t1 b, t1 c;Query OK, 8 rows affected (0.00 sec)mysql55 >INSERT INTO t1(i1, v1) SELECT NULL, a.v1 FROM t1 a, t1 b, t1 c;Query OK, 1000 rows affected (0.02 sec)# Does it return in your VM, or fill your host disk?
Be Destructive
R
#PerconaLive @RonaldBradford @bytebot
Anonymous users in mysql.usermysql56 >SELECT host,user,password FROM mysql.user WHERE user='';+-----------+------+----------+| host | user | password |+-----------+------+----------+| localhost | | || mysql56 | | |+-----------+------+----------+2 rows in set (0.00 sec)
mysql56 >DELETE FROM mysql.user WHERE user='';mysql56 >FLUSH PRIVILEGES;
R
#PerconaLive @RonaldBradford @bytebot
Security Hardening 101● Set 'root' password● Remove anonymous users● Remove 'test' schema
This is all taken care of with mysql_secure_installation
R
#PerconaLive @RonaldBradford @bytebot
mysql_secure_installation
$ mysql_secure_installation
VALIDATE PASSWORD PLUGIN can be used to test passwordsand improve security. It checks the strength of passwordand allows the users to set only those passwords which aresecure enough. Would you like to setup VALIDATE PASSWORD plugin?
There are three levels of password validation policy:
LOW Length >= 8MEDIUM Length >= 8, numeric, mixed case, and special charactersSTRONG Length >= 8, numeric, mixed case, special characters and dictionary file
Please enter 0 = LOW, 1 = MEDIUM and 2 = STRONG: ...Estimated strength of the password: 25Do you wish to continue with the password provided?(Press y|Y for Yes, any other key for No)...
5.7 Functionality
R
#PerconaLive @RonaldBradford @bytebot
Command Line options (non-interactive)
● What about gh-ost, pt-osc, other pt- tools etc
read MYSQL_USERread -s MYSQL_PWD
mysql -u${MYSQL_USER} -p${MYSQL_PWD}
mysql # using $HOME/.my.cnfmysql --defaults-file=/path/to/.my.cnf
ps rewrite
https://dev.mysql.com/doc/refman/5.6/en/mysql-logging.html
ps -ef | grep mysql.... mysql -udemo -px xxxxxxx
R
#PerconaLive @RonaldBradford @bytebot
Bad habits$ (mysql -uinsecure -pTest.123 -e "SELECT SLEEP(3)" > /dev/null &); ps -ef | grep mysqlvagrant 2923 1 0 00:08 pts/0 00:00:00 mysql -uinsecure -px xxxxxxxxx -e SELECT SLEEP(3)
$ (pt-show-grants -uinsecure -pTest.123 > /dev/null &); ps -ef | grep ptvagrant 2902 1 0 00:04 pts/0 00:00:00 perl /usr/bin/pt-show-grants -uinsecure -pTest.123
$ (gh-ost --user=insecure --password=Test.123 &); ps -ef | grep ghvagrant 2983 1 0 00:29 pts/0 00:00:00 gh-ost --user=insecure --password=Test.123
$ (xtrabackup -uinsecure -pTest.123 --backup >/dev/null &); ps -ef | grep xtravagrant 2917 1 0 00:08 pts/0 00:00:00 xtrabackup -uinsecure -px xxxxxxxxx --backup
R
Do not do this
#PerconaLive @RonaldBradford @bytebot
MySQL Config Editor$ mysql_secure_installation$ mysql -uroot -p -e "CREATE USER demo@localhost IDENTIFIED BY 'passw0rd1';"$ echo "[client]user=demopassword=passw0rd1" > $HOME/.my.cnf$ mysql -e "SELECT USER()"$ rm $HOME/.my.cnf$ mysql -e "SELECT USER()"
$ mysql_config_editor set --login-path=client --host=localhost --user=demo --password$ ls -al $HOME/.mylogin.cnf$ cat $HOME/.mylogin.cnf$ mysql_config_editor print
$ mysql -e "SELECT USER()"$ mysqldump ....
Since 5.6
R
#PerconaLive @RonaldBradford @bytebot
Password Improvements
#PerconaLive @RonaldBradford @bytebot
MySQL 5.6 improvements● Password expiry - ALTER USER 'foo'@'localhost' PASSWORD EXPIRE;
● Password validation plugin - VALIDATE_PASSWORD_STRENGTH()
● mysql_config_editor - store authentication credentials in an encrypted
login path file named .mylogin.cnf○ http://dev.mysql.com/doc/refman/5.6/en/mysql-config-editor.html
● Random ‘root’ password on install○ mysql_install_db —random-passwords
○ cat $HOME/.mysql_secret
#PerconaLive @RonaldBradford @bytebot
PASSWORD EXPIRE$ export MYSQL_PS1="\u@\h [\d]> "
$ mysql -uroot -proot@localhost [(none)]> CREATE USER demo IDENTIFIED BY 'passw0rd1';$ mysql -udemo -p #passw0rd1
$ mysql -urootroot@localhost [(none)]> ALTER USER demo PASSWORD EXPIRE;
$ mysql -udemo -p #passw0rd1demo@localhost [(none)]> # No issue connectingdemo@localhost [(none)]> USE test;ERROR 1820 (HY000): You must reset your password using ALTER USER statement before executing this statement.
# can reset password to current valuedemo@localhost [(none)]> ALTER USER demo IDENTIFIED BY 'passw0rd1';
#PerconaLive @RonaldBradford @bytebot
MySQL 5.7 improvements● Improved password expiry — automatic password expiration available, so set
default_password_lifetime in my.cnf
● You can also require password to be changed every n-days○ ALTER USER foo@localhost PASSWORD EXPIRE INTERVAL n DAY;
● PASSWORD EXPIRE DEFAULT | NEVER options
● There is also account locking/unlocking now○ ALTER USER foo@host ACCOUNT LOCK | UNLOCK;
https://dev.mysql.com/doc/refman/5.7/en/alter-user.html
#PerconaLive @RonaldBradford @bytebot
MariaDB Server passwords● Password validation plugin
○ https://mariadb.com/kb/en/mariadb/development/mariadb-internals-documentation/password-v
alidation/
● simple_password_check password validation plugin○ can enforce a minimum password length and guarantee that a password contains at least a
specified number of uppercase and lowercase letters, digits, and punctuation characters.
● cracklib_password_check password validation plugin○ Allows passwords that are strong enough to pass CrackLib test. This is the same test that
pam_cracklib.so does
○ SET PASSWORD FOR 'bob'@'%.loc.gov' = PASSWORD('abc');
#PerconaLive @RonaldBradford @bytebot
Authentication in MySQL / MariaDB Server● auth_socket - Authenticates against the Unix socket file, using so_peercred
● sha256_password - default-authentication-plugin=sha256_password,
passwords never exposed as cleartext when connecting; SSL or RSA auth
● ed25519 - Elliptic Curve Digital Signature Algorithm, same as OpenSSH
● Kerberos/GSSAPI/SSPI - User principals: <username>@<KERBEROS
REALM>
● Active Directory (Enterprise only)
● mysql_no_login ( MySQL 5.7 ) - prevents all client connections to an account
that uses it
#PerconaLive @RonaldBradford @bytebot
Users and Privileges
#PerconaLive @RonaldBradford @bytebot
Managing usersmysql> CREATE USER demo@localhost;Query OK, 0 rows affected (0.00 sec)
$ mysql -udemo -e "SELECT USER(), CURRENT_USER()"+----------------+----------------+| USER() | CURRENT_USER() |+----------------+----------------+| demo@localhost | demo@localhost |+----------------+----------------+
mysql> DROP USER demo@localhost;Query OK, 0 rows affected (0.00 sec)
mysql57 >CREATE USER demo@localhost;ERROR 1819 (HY000): Your password does not satisfy the current policy requirements
R
#PerconaLive @RonaldBradford @bytebot
User Connection Errors$ mysql -udemoERROR 1045 (28000): Access denied for user 'demo'@'localhost' (using password: NO)
$ mysql -udemo -pxxxmysql: [Warning] Using a password on the command line interface can be insecure.ERROR 1045 (28000): Access denied for user 'demo'@'localhost' (using password: YES)
$ tail -2 /var/log/mysqld.log2017-09-06T00:39:24.908224Z 16 [Note] Access denied for user 'demo'@'localhost' (using password: NO)2017-09-06T00:39:37.423691Z 17 [Note] Access denied for user 'demo'@'localhost' (using password: YES)
#PerconaLive @RonaldBradford @bytebot
User Connection Errors via userstats (Percona Server / MariaDB Server)percona56 >set global userstat=ON;percona56 >SELECT user,total_connections,denied_connections,total_ssl_connections FROM information_schema.user_statistics;percona56 >select host,user from mysql.user;
$ mysql -udemo -p...
$ mysql -uroot -ppercona56 >SELECT user,total_connections,denied_connections,total_ssl_connections FROM information_schema.user_statistics;+------+-------------------+--------------------+-----------------------+| user | total_connections | denied_connections | total_ssl_connections |+------+-------------------+--------------------+-----------------------+| demo | 1 | 0 | 0 || root | 3 | 0 | 0 |+------+-------------------+--------------------+-----------------------+2 rows in set (0.00 sec)
#PerconaLive @RonaldBradford @bytebot
SHOW CREATE USER (5.7)mysql57 >SHOW CREATE USER root@localhost\G*************************** 1. row ***************************… CREATE USER 'root'@'localhost' IDENTIFIED WITH 'mysql_native_password' AS '*E89C1DBB80A00976B61D19025C3081E4B190D8BE' REQUIRE NONE PASSWORD EXPIRE DEFAULT ACCOUNT UNLOCK
SHOW CREATE USER (Privileges needed)host >select USER();+-----------------------+| USER() |+-----------------------+| [email protected] |+-----------------------+
host >SHOW CREATE USER external@'%';...| CREATE USER 'external'@'%' IDENTIFIED WITH 'mysql_native_password' AS '*E89C1DBB80A00976B61D19025C3081E4B190D8BE' REQUIRE NONE PASSWORD EXPIRE DEFAULT ACCOUNT UNLOCK |
host >SHOW CREATE USER external@localhost;ERROR 1044 (42000): Access denied for user 'external'@'%' to database 'mysql'
#PerconaLive @RonaldBradford @bytebot
MySQL privilegesmysql -e "SHOW GRANTS"mysql -uroot -e "SHOW GRANTS"mysql57 >SHOW PRIVILEGES;--------------------------------------+| Privilege | Context | Comment |+-------------------------+---------------------------------------+-------------------------------------------------------+| Alter | Tables | To alter the table || Alter routine | Functions,Procedures | To alter or drop stored functions/procedures || Create | Databases,Tables,Indexes | To create new databases and tables || Create routine | Databases | To use CREATE FUNCTION/PROCEDURE || Create temporary tables | Databases | To use CREATE TEMPORARY TABLE || Create view | Tables | To create new views || Create user | Server Admin | To create new users
#PerconaLive @RonaldBradford @bytebot
MySQL 8.0 additional privileges (8.0.2 DMR)Create role Server Admin To create new rolesDrop role Server Admin To drop rolesSET_USER_ID Server AdminROLE_ADMIN Server AdminREPLICATION_SLAVE_ADMIN Server AdminXA_RECOVER_ADMIN Server AdminGROUP_REPLICATION_ADMIN Server AdminSYSTEM_VARIABLES_ADMIN Server AdminPERSIST_RO_VARIABLES_ADMIN Server AdminENCRYPTION_KEY_ADMIN Server AdminCONNECTION_ADMIN Server AdminBINLOG_ADMIN Server Admin
#PerconaLive @RonaldBradford @bytebot
Sane user creation practices● Use a least privileges model● Segregate responsibilities of access
○ application (not DDL)○ Read_only○ DBA
● Use host as a means of limitation combined with OS access● GRANT/REVOKE is not enough to return privileges to a defined state.● DROP USER/CREATE USER/GRANT is necessary
#PerconaLive @RonaldBradford @bytebot
Why being SUPER is bad (GRANT ALL ON *.*)● Bypasses read_only (why we need super_read_only)
● Bypasses init_connect
● Can disable binary logging
● Can change dynamic configuration
● Takes the reserved connection
http://ronaldbradford.com/blog/why-grant-all-is-bad-2010-08-06/
http://effectivemysql.com/presentation/mysql-idiosyncrasies-that-bite/
#PerconaLive @RonaldBradford @bytebot
Create users for specific purposesBe specific:
● 'grafana' not 'monitoring'● 'ghost'● 'backup'● Individual DBA user accounts
#PerconaLive @RonaldBradford @bytebot
Bypassing all privileges--skip-grant-tables
$ more /etc/my.cnf…[mysqld]skip-grant-tables
$ sudo systemctl restart mysqld.service
$ mysql -uroot # No password
https://dev.mysql.com/doc/refman/5.7/en/server-options.html#option_mysqld_skip-grant-tables
#PerconaLive @RonaldBradford @bytebot
Bypassing all privileges --init-file
$ more /etc/my.cnf…[mysqld]init-file=/tmp/password.sql
$ cat /tmp/password.sqlGRANT ALL PRIVILEGES ON *.* TO root@localhost IDENTIFIED BY '<clear-text-password>'' WITH GRANT OPTION;
$ sudo systemctl restart mysqld.service
$ mysql -uroot -pSee also: http://code.openark.org/blog/mysql/dangers-of-skip-grant-tables
#PerconaLive @RonaldBradford @bytebot
Bypassing all privileges (replacing the user.MYD file)$ cd /tmp $ echo "[mysqld] datadir=/tmp" > my.cnf
$ scripts/mysql_install_db --defaults-file=/tmp/my.cnf $ ls -l /tmp/mysql/user.* -rw-rw---- 1 320 Feb 13 11:27 /tmp/mysql/user.MYD -rw-rw---- 1 2048 Feb 13 11:27 /tmp/mysql/user.MYI -rw-rw---- 1 10630 Feb 13 11:27 /tmp/mysql/user.frm
#PerconaLive @RonaldBradford @bytebot
MySQL Socket File (Localhost)$ ps -ef | grep mysqlroot 3016 1 0 Sep04 ? 00:00:00 /bin/sh /usr/bin/mysqld_safemysql 3169 3016 0 Sep04 ? 00:05:07 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock
$ grep socket /etc/my.cnfsocket=/var/lib/mysql/mysql.sock
#PerconaLive @RonaldBradford @bytebot
MySQL TCP/IP (external access)$ grep 3306 /etc/servicesmysql 3306/tcp # MySQLmysql 3306/udp # MySQLmysql57 > SELECT @@bind_address;+----------------+| @@bind_address |+----------------+| * |+----------------+
mysql57$ netstat -an | grep 3306Active Internet connections (servers and established)Proto Recv-Q Send-Q Local Address Foreign Address State...tcp6 0 0 :::3306 :::* LISTEN
R
#PerconaLive @RonaldBradford @bytebot
Using an OS firewallhost$ mysql -uexternal -p -h192.168.42.17...
mysql$ sudo systemctl start firewalld
host$ mysql -uexternal -p -h192.168.42.17Enter password:ERROR 2003 (HY000): Can't connect to MySQL server on '192.168.42.17' (113)
#PerconaLive @RonaldBradford @bytebot
Enable access via the OS firewallmysql$ sudo firewall-cmd --permanent --add-service=mysqlmysql$ sudo firewall-cmd --permanent --list-allpublic target: default icmp-block-inversion: no interfaces: sources: services: dhcpv6-client mysql ssh...
mysql$ sudo firewall-cmd --reload
host$ mysql -uexternal -p -h192.168.42.17...
mysql$ sudo systemctl stop firewalld
#PerconaLive @RonaldBradford @bytebot
SSL
#PerconaLive @RonaldBradford @bytebot
Secure Communications● SSL for replication
● SSL for client connections
● SSL for admin connections
● Encryption on the wire
https://dev.mysql.com/doc/refman/5.6/en/secure-connections.htmlhttps://dev.mysql.com/doc/refman/5.7/en/secure-connections.html
#PerconaLive @RonaldBradford @bytebot
Default client connection traffic (5.6)
mysql56$ sudo ngrep -d eth1 -wi -P ' ' -W single -l port 3306interface: eth1 (192.168.42.0/255.255.255.0)filter: ( port 3306 ) and ((ip || ip6) || (vlan && (ip || ip6)))
host$ mysql -uexternal -p -h192.168.42.16
host> select 'unencrypted';
#T 192.168.42.1:47634 -> 192.168.42.16:3306 [AP] select 'unencrypted'#T 192.168.42.16:3306 -> 192.168.42.1:47634 [AP] ! def unencrypted ! ! unencrypted
https://wiki.christophchamp.com/index.php?title=Ngrephttp://infoheap.com/ngrep-quick-start-guide/
#PerconaLive @RonaldBradford @bytebot
Default client connection traffic (5.7)mysql57$ sudo ngrep -d eth1 -wi -P ' ' -W single -l port 3306interface: eth1 (192.168.42.0/255.255.255.0)filter: ( port 3306 ) and ((ip || ip6) || (vlan && (ip || ip6)))
host$ mysql -uexternal -p -h192.168.42.17
host> select 'encrypted';
T 192.168.42.1:36781 -> 192.168.42.17:3306 [AP] @ F l d iVr H b ^ s t Z ( 2d " ? | )#T 192.168.42.17:3306 -> 192.168.42.1:36781 [AP] p% s` 3u5!%P] v= r # x E a y '! )Z 8 Js z. \t (r H@ 0 2 5k\ < M @)E& b q|q@ h
#PerconaLive @RonaldBradford @bytebot
5.7 non-SSL client connection traffichost$ > mysql -uexternal -p -h192.168.42.17 --ssl=0
host >select '-ssl=0 unencrypted';
T 192.168.42.1:36785 -> 192.168.42.17:3306 [AP] select '-ssl=0 unencrypted'#T 192.168.42.17:3306 -> 192.168.42.1:36785 [AP] ' def -ssl=0 unencrypted ! 3 -ssl=0 unencrypted
#PerconaLive @RonaldBradford @bytebot
SSL system variablesmysql57 > SHOW GLOBAL VARIABLES LIKE '%ssl%';+---------------+-----------------+| Variable_name | Value |+---------------+-----------------+| have_openssl | YES || have_ssl | YES || ssl_ca | ca.pem || ssl_capath | || ssl_cert | server-cert.pem || ssl_cipher | || ssl_crl | || ssl_crlpath | || ssl_key | server-key.pem |+---------------+-----------------+9 rows in set (0.00 sec)
mysql56 >SHOW VARIABLES LIKE '%ssl%';+---------------+----------+| Variable_name | Value |+---------------+----------+| have_openssl | DISABLED || have_ssl | DISABLED || ssl_ca | || ssl_capath | || ssl_cert | || ssl_cipher | || ssl_crl | || ssl_crlpath | || ssl_key | |+---------------+----------+9 rows in set (0.02 sec)
#PerconaLive @RonaldBradford @bytebot
Secure Communications[mysqld]ssl-ca=ca.pemssl-cert=server-cert.pemssl-key=server-key.pem
#PerconaLive @RonaldBradford @bytebot
SSL Protocols and Ciphersmysql> SHOW SESSION STATUS LIKE 'Ssl_version';+---------------+-------+| Variable_name | Value |+---------------+-------+| Ssl_version | TLSv1 |+---------------+-------+mysql> SHOW SESSION STATUS LIKE 'Ssl_cipher';+---------------+---------------------------+| Variable_name | Value |+---------------+---------------------------+| Ssl_cipher | DHE-RSA-AES128-GCM-SHA256 |+---------------+---------------------------+
#PerconaLive @RonaldBradford @bytebot
SSL Client Connectionshttps://dev.mysql.com/doc/connector-python/en/connector-python-connectargs.html
import mysql.connectorfrom mysql.connector.constants import ClientFlag
config = { 'user': 'ssluser', 'password': 'asecret', 'host': '127.0.0.1', 'client_flags': [ClientFlag.SSL], 'ssl_ca': '/opt/mysql/ssl/ca.pem', 'ssl_cert': '/opt/mysql/ssl/client-cert.pem', 'ssl_key': '/opt/mysql/ssl/client-key.pem',}
https://dev.mysql.com/doc/connectors/en/connector-net-tutorials-ssl.html
#PerconaLive @RonaldBradford @bytebot
Secure Connections● mysql_ssl_rsa_setup in MySQL 5.7
○ This program creates the SSL certificate and key files and RSA key-pair
files required to support secure connections using SSL and secure
password exchange using RSA over unencrypted connections, if those
files are missing.
● use the openssl command
#PerconaLive @RonaldBradford @bytebot
Cloud SSL● Rackspace - MySQL has GRANT modifier, REQUIRE SSL
○ GRANT SELECT, INSERT, UPDATE, DELETE ON mydatabase.* TO 'user'@'%' REQUIRE SSL;
● Amazon - ssl-verify-server-cert so SSL connection verifies the DB instance endpoint against the endpoint in the SSL certificate
mysql \ -h myinstance.123456789012.us-east-1.rds.amazonaws.com \ --ssl-ca=rds-ca-2015-root.pem \ --ssl-verify-server-cert
● Google - max 10 certs/instance
mysql --ssl-ca=server-ca.pem --ssl-cert=client-cert.pem --ssl-key=client-key.pem \ --host=[INSTANCE_IP] --user=root --password
#PerconaLive @RonaldBradford @bytebot
SSL notes● MySQL clients shipped pre-5.7.3 consider the --ssl option just as advise. ● It silently falls back to unencrypted connections if the server doesn’t accept encrypted
connections!● You have to configure the instance so that only SSL connections can connect to it.● Also… Client-side --ssl option is deprecated as of MySQL 5.7.11 and is removed in
MySQL 8.0. For client programs, use --ssl-mode instead. So --ssl-mode=REQUIRED is preferred
● This matters - your “mysql” client could be from MariaDB Server which is different!● SSL libraries for MySQL Community and Enterprise are different; also worth noting
some are linked statically, some are dynamic
#PerconaLive @RonaldBradford @bytebot
Authentication Plugins
R
#PerconaLive @RonaldBradford @bytebot
Stronger passwords (sha256)
R
mysql> CREATE USER bernie@localhost IDENTIFIED WITH sha256_password;mysql> select host,user,password,plugin,authentication_string,password_expired from mysql.user where user='bernie'\G*************************** 1. row *************************** host: localhost user: bernie password: plugin: sha256_passwordauthentication_string: password_expired: N
mysql> SET old_passwords=2;mysql> SET PASSWORD FOR bernie@localhost = PASSWORD('<clear-text-password>'');mysql> select host,user,password,plugin,authentication_string,password_expired from mysql.user where user='bernie'\G*************************** 1. row *************************** host: localhost user: bernie password: plugin: sha256_passwordauthentication_string: $5$ln^h{z\d{JSj}FB$9TF1X0S.Ts4lBSOdsx7p86F4OxRL8ataFok6hyPgew/ password_expired: N
#PerconaLive @RonaldBradford @bytebot
PAM ● Percona PAM authentication plugin (5.6+)
○ Open Source, runs community also
● MySQL Enterprise PAM authentication (5.5+)○ Commercial license
● MariaDB PAM Authentication (5.2+)
https://www.percona.com/doc/percona-pam-for-mysql/index.htmlhttps://dev.mysql.com/doc/refman/5.5/en/pam-pluggable-authentication.html
https://mariadb.com/kb/en/the-mariadb-library/authentication-plugin-pam/
R
#PerconaLive @RonaldBradford @bytebot
Configuring PAM (Demo only)
R
echo "auth required pam_warn.soauth required pam_unix.so auditaccount required pam_unix.so audit" | sudo tee /etc/pam.d/mysqld
# DO NOT DO THIS sudo groupadd shadowsudo usermod -G shadow mysqlgrep shadow /etc/groupsudo chgrp shadow /etc/shadowsudo chmod 440 /etc/shadowsudo systemctl restart mysqld.service
DO NOT DO THIS
#PerconaLive @RonaldBradford @bytebot
Installing PAM Plugin
R
percona56 >SELECT plugin_name FROM information_schema.plugins WHERE plugin_type='AUTHENTICATION';+-----------------------+| mysql_native_password || mysql_old_password || sha256_password |+-----------------------+
percona56 >INSTALL PLUGIN auth_pam SONAME 'auth_pam.so';percona56 >SELECT plugin_name FROM information_schema.plugins WHERE plugin_type='AUTHENTICATION';+-----------------------+| plugin_name |+-----------------------+| mysql_native_password || mysql_old_password || sha256_password || auth_pam |+-----------------------+4 rows in set (0.00 sec)
#PerconaLive @RonaldBradford @bytebot
Demonstrating PAM
R
percona56 >CREATE USER external@localhost IDENTIFIED WITH auth_pam;
$ mysql -uexternalPassword:ERROR 1045 (28000): Access denied for user 'external'@'localhost' (using password: YES)
$ sudo useradd external$ sudo passwd external
$ mysql -uexternal -p -Ne "SELECT USER()"+--------------------+| external@localhost |+--------------------+
mysql > SELECT /* Use 5.6 */ host, user, password, plugin, authentication_string, password_expired FROM mysql.user WHERE user='external'\G*************************** 1. row *************************** host: localhost user: external password: plugin: auth_pamauthentication_string: password_expired: N1 row in set (0.00 sec)
#PerconaLive @RonaldBradford @bytebot
MariaDB Authentication Plugins● edd25519 Authentication (10.1)
Elliptic Curve Digital Signature Algorithm● GSSAPI Authentication (10.1)
usually synonymous with Kerberos (*NIX)● Named Pipe (10.1)
Windows● Unix Socket (10.1)
https://mariadb.com/kb/en/the-mariadb-library/password-authentication-and-encryption-plugins/
R
#PerconaLive @RonaldBradford @bytebot
ed25519 Authentication (MariaDB 10.1+)
R
maria101 > INSTALL SONAME 'auth_ed25519';maria101 > CREATE FUNCTION ed25519_password RETURNS STRING SONAME "auth_ed25519.so";
maria101 > SELECT plugin_name FROM information_schema.plugins WHERE plugin_type='AUTHENTICATION';+-----------------------+| mysql_native_password || mysql_old_password || ed25519 |+-----------------------+
maria101 > SELECT ed25519_password("<clear-text-passwd>") AS pwd;+---------------------------------------------+| pwd |+---------------------------------------------+| CobnFnV6aei4yy55u90XPFUeBRMBSPtZazzJq8kLHNE |+---------------------------------------------+
maria101 > CREATE USER demo@localhost IDENTIFIED VIA ed25519 USING 'CobnFnV6aei4yy55u90XPFUeBRMBSPtZazzJq8kLHNE';
#PerconaLive @RonaldBradford @bytebot
Unix Socket Authentication (MariaDB 10.1)
R
maria101 > INSTALL PLUGIN unix_socket SONAME 'auth_socket';maria101 > SELECT plugin_name FROM information_schema.plugins WHERE plugin_type='AUTHENTICATION';+-----------------------+| mysql_native_password || mysql_old_password || ed25519 || unix_socket |+-----------------------+
maria101 > CREATE USER vagrant IDENTIFIED VIA unix_socket;
$ whoamivagrant
$ mysql -e "SELECT USER(),CURRENT_USER()"+-------------------+----------------+| USER() | CURRENT_USER() |+-------------------+----------------+| vagrant@localhost | @localhost |+-------------------+----------------+
root@maria101 > CREATE USER notvagrant IDENTIFIED VIA unix_socket;
$ mysql -unotvagrantERROR 1698 (28000): Access denied for user 'notvagrant'@'localhost'
#PerconaLive @RonaldBradford @bytebot
Auditing
R
#PerconaLive @RonaldBradford @bytebot
Auditing CapabilitiesMySQL
- Logging account access
- Logging SQL statements
- Logging uncommon SQL patterns
OS
- Logging account logins
- Logging sudo commands
R
#PerconaLive @RonaldBradford @bytebot
Auditing Plugin Options● McAfee (5.1+)● Percona (5.5)● MariaDB (10.0)● MySQL Enterprise (5.5)
● Installation● Configuration● Log format
R
#PerconaLive @RonaldBradford @bytebot
Installing McAfee Plugin (on Percona 5.6)
https://github.com/mcafee/mysql-audit/wiki/Installation
wget https://dl.bintray.com/mcafee/mysql-audit-plugin/:audit-plugin-mysql-5.6-1.1.5-742-linux-x86_64.zipunzip audit-plugin-mysql-5.6-1.1.5-742-linux-x86_64.zipcd audit-plugin-mysql-5.6-1.1.5-742/sudo cp lib/libaudit_plugin.so /usr/lib64/mysql/plugin/sudo chmod +w /usr/lib64/mysql/plugin/libaudit_plugin.sowget https://raw.github.com/mcafee/mysql-audit/master/offset-extract/offset-extract.shchmod +x offset-extract.shsudo yum install -y gdb./offset-extract.sh `which mysqld`sudo vi /etc/my.cnf
plugin-load=AUDIT=libaudit_plugin.soaudit_offsets=6992, 7040, 4000, 4520, 72, 2704, 96, 0, 32, 104, 136, 7128, 4392, 2800, 2808, 2812, 536, 0, 0, 6360, 6384, 6368, 13048, 548, 516
sudo service mysqld restart
R
#PerconaLive @RonaldBradford @bytebot
Installing Audit plugin (Enterprise 5.7)enterprise57 > SHOW GLOBAL VARIABLES LIKE 'audit%';Empty set (0.01 sec)
enterprise57 $ mysql -uroot -p < /usr/share/mysql/audit_log_filter_linux_install.sqlResultOK
enterprise57 > SELECT PLUGIN_NAME, PLUGIN_STATUS FROM INFORMATION_SCHEMA.PLUGINS WHERE PLUGIN_NAME LIKE 'audit%';+-------------+---------------+| PLUGIN_NAME | PLUGIN_STATUS |+-------------+---------------+| audit_log | ACTIVE |+-------------+---------------+1 row in set (0.00 sec)
R
https://dev.mysql.com/doc/refman/5.5/en/audit-log.html
#PerconaLive @RonaldBradford @bytebot
Installing Percona Audit Pluginpercona56 $ mysql -uroot
percona56 >INSTALL PLUGIN audit_log SONAME 'audit_log.so';
percona56 >SELECT PLUGIN_NAME, PLUGIN_STATUS FROM INFORMATION_SCHEMA.PLUGINS WHERE PLUGIN_NAME LIKE 'audit%';+-------------+---------------+| PLUGIN_NAME | PLUGIN_STATUS |+-------------+---------------+| audit_log | ACTIVE |+-------------+---------------+
R
https://www.percona.com/doc/percona-server/5.5/management/audit_log_plugin.htmlhttps://www.percona.com/blog/2014/05/16/introduction-to-the-percona-mysql-audit-log-plugin/
#PerconaLive @RonaldBradford @bytebot
Installing MariaDB Audit Pluginmaria101 > INSTALL PLUGIN server_audit SONAME 'server_audit.so';
maria101 > SELECT PLUGIN_NAME, PLUGIN_STATUS FROM INFORMATION_SCHEMA.PLUGINS WHERE PLUGIN_TYPE='AUDIT';+--------------+---------------+| PLUGIN_NAME | PLUGIN_STATUS |+--------------+---------------+| SERVER_AUDIT | ACTIVE |+--------------+---------------+
maria101 > SET GLOBAL server_audit_events = 'CONNECT,QUERY,TABLE';maria101 > SET GLOBAL server_audit_logging=ON;
R
https://mariadb.com/kb/en/mariadb/about-the-mariadb-audit-plugin/
#PerconaLive @RonaldBradford @bytebot
Audit Log Formats● OLD (XML)● NEW (XML)● JSON● CSV
R
https://dev.mysql.com/doc/refman/5.5/en/audit-log-file.html
#PerconaLive @RonaldBradford @bytebot
OLD/NEW Format$ sudo tail /var/lib/mysql/audit.log<AUDIT_RECORD NAME="Connect" RECORD="9_2017-09-04T21:40:19" TIMESTAMP="2017-09-04T21:45:34 UTC" CONNECTION_ID="13" STATUS="1045" USER="root" PRIV_USER="root" OS_LOGIN="" PROXY_USER="" HOST="localhost" IP="" DB=""/>
R
$ sudo tail /var/lib/mysql/audit.log<AUDIT_RECORD> <NAME>Quit</NAME> <RECORD>1358_2017-09-25T12:59:46</RECORD> <TIMESTAMP>2017-09-25T12:59:49 UTC</TIMESTAMP> <CONNECTION_ID>1</CONNECTION_ID> <STATUS>0</STATUS> <USER>root</USER> <PRIV_USER>root</PRIV_USER> <OS_LOGIN></OS_LOGIN> <PROXY_USER></PROXY_USER> <HOST>localhost</HOST> <IP></IP> <DB></DB></AUDIT_RECORD>
#PerconaLive @RonaldBradford @bytebot
JSON/CSV Formatsudo tail /var/lib/mysql/audit.log
{"audit_record":{"name":"Ping","record":"3417_2017-09-25T13:37:27","timestamp":"2017-09-25T13:37:29 UTC","command_class":"error","connection_id":"1","status":0,"sqltext":"","user":"root[root] @ localhost []","host":"localhost","os_user":"","ip":"","db":""}}{"audit_record":{"name":"Quit","record":"3418_2017-09-25T13:37:27","timestamp":"2017-09-25T13:37:29 UTC","connection_id":"1","status":0,"user":"root","priv_user":"root","os_login":"","proxy_user":"","host":"localhost","ip":"","db":""}}
R
sudo tail /var/lib/mysql/audit.log
"Connect","4915_2017-09-25T13:39:34","2017-09-25T13:39:38 UTC","1",0,"root","root","","","localhost","","""Ping","4916_2017-09-25T13:39:34","2017-09-25T13:39:38 UTC","error","1",0,"","root[root] @ localhost []","localhost","","",""
#PerconaLive @RonaldBradford @bytebot
McAfee Audit Log Examplemysql56 >set global audit_json_file=ON;
$ mysql -udemo -pxWarning: Using a password on the command line interface can be insecure.ERROR 1045 (28000): Access denied for user 'demo'@'localhost' (using password: YES)
$ sudo cat /var/lib/mysql/mysql-audit.json{"msg-type":"activity","date":"1505267816852","thread-id":"13","query-id":"0","user":"demo","priv_user":"demo","ip":"","host":"localhost","connect_attrs":{"_os":"Linux","_client_name":"libmysql","_pid":"5944","_client_version":"5.6.37","_platform":"x86_64","program_name":"mysql"},"pid":"5944","os_user":"vagrant","appname":"mysql","status":"1045","cmd":"Failed Login","query":"Failed Login"}{"msg-type":"activity","date":"1505267816852","thread-id":"13","query-id":"0","user":"demo","priv_user":"demo","ip":"","host":"localhost","connect_attrs":{"_os":"Linux","_client_name":"libmysql","_pid":"5944","_client_version":"5.6.37","_platform":"x86_64","program_name":"mysql"},"pid":"5944","os_user":"vagrant","appname":"mysql","cmd":"Connect","query":"Connect"}
R
#PerconaLive @RonaldBradford @bytebot
McAfee Audit Log Example{
"msg-type": "activity","date": "1505267816852","thread-id": "13","query-id": "0","user": "demo","priv_user": "demo",
..."pid": "5944","os_user": "vagrant","appname": "mysql","status": "1045","cmd": "Failed Login","query": "Failed Login"
}
R
#PerconaLive @RonaldBradford @bytebot
MariaDB Audit Log Format$ sudo cat /var/lib/mysql/server_audit.log20170912 02:02:15,maria101,root,localhost,3,11,QUERY,,'SET GLOBAL server_audit_logging=ON',020170912 02:02:39,maria101,root,localhost,3,0,DISCONNECT,,,020170912 02:02:40,maria101,root,localhost,4,0,CONNECT,,,020170912 02:02:40,maria101,root,localhost,4,13,QUERY,,'select @@version_comment limit 1',020170912 02:02:40,maria101,root,localhost,4,14,QUERY,,'select USER()',020170912 02:02:42,maria101,root,localhost,4,0,DISCONNECT,,,020170912 02:02:45,maria101,root,localhost,5,0,FAILED_CONNECT,,,104520170912 02:02:45,maria101,root,localhost,5,0,DISCONNECT,,,0
R
https://mariadb.com/kb/en/library/mariadb-audit-plugin-log-format/
#PerconaLive @RonaldBradford @bytebot
Auditing Implementation● MariaDB Server
○ User filtering as an additional feature via audit API extensions○ Query cache enabled? No table records
● Percona○ Multiple output formats: OLD, NEW, JSON, CSV○ Filter by user, SQL command type, database, ○ Auditing can be expensive, so asynchronous/performance/semisynchronous/synchronous
modes for logging - e.g. log using memory buffers, drop messages if buffers are full, or log directly to file, flush and sync at events, etc.
● McAfee Audit plugin○ Uses offsets
● MySQL Enterprise Audit Plugin (utility: mysqlauditgrep)
R
#PerconaLive @RonaldBradford @bytebot
Encryption
#PerconaLive @RonaldBradford @bytebot
Secure Storage● Encryption of data at rest
○ Data (table vs tablespace)
○ Binary Logs
○ Other Logs
● Key management
#PerconaLive @RonaldBradford @bytebot
Encryption in MariaDB Server● Encryption: tablespace OR table level encryption with support for rolling keys
using the AES algorithm (only with a keyserver)○ table encryption — PAGE_ENCRYPTION=1○ tablespace encryption — encrypts everything including log files
● file_key_management_filename, file_key_management_filekey, file_key_management_encryption_algorithm
● Documented — https://mariadb.com/kb/en/mariadb/data-at-rest-encryption/ ● Tablespace/logs scrubbing: background process that regularly scans through
the tables and upgrades the encryption keys● --encrypt-tmp-files & --encrypt-binlog
#PerconaLive @RonaldBradford @bytebot
Encryption in MariaDB Server II[mysqld]plugin-load-add=file_key_management.sofile-key-managementfile-key-management-filename = /home/mdb/keys.encinnodb-encrypt-tablesinnodb-encrypt-loginnodb-encryption-threads=4aria-encrypt-tables=1 # PAGE row formatencrypt-tmp-disk-tables=1 # this is for Aria
CREATE TABLE customer (customer_id bigint not null primary
key,customer_name varchar(80),customer_creditcard varchar(20))
ENGINE=InnoDBpage_encryption=1page_encryption_key=1;
#PerconaLive @RonaldBradford @bytebot
Encryption in MariaDB Server III● Use the preset! - /etc/my.cnf.d/enable_encryption.preset
● A plugin for Amazon Key Management Server (KMS)
● mysqlbinlog has no way to read (i.e. decrypt) an encrypted binlog
● This does not work with MariaDB Galera Cluster yet (gcache is not encrypted
yet), and also Percona XtraBackup needs additional work (i.e. if you encrypt
the redo log)○ Recommended backup method: MariaDB Backup (fork of Percona XtraBackup)
#PerconaLive @RonaldBradford @bytebot
Encryption in MySQL● MySQL 5.7.11 introduces InnoDB tablespace encryption
● early-plugin-load=keyring_file.so in my.cnf
● Must use innodb_file_per_table
● Convert via ALTER TABLE table ENCRYPTION=‘Y’
● Data is not encrypted in the redo/undo/binary logs
● Has external key management (Oracle Key Vault)
#PerconaLive @RonaldBradford @bytebot
Key management in MySQL● MySQL 5.7.11 and higher includes a keyring plugin, keyring_file, that stores
keyring data in a file local to the server host. ● MySQL 5.7.12 and higher includes keyring_okv, a KMIP 1.1 plugin for use
with KMIP-compatible back end keyring storage products such as Oracle Key Vault and Gemalto SafeNet KeySecure Appliance. (Enterprise)
● MySQL 5.7.19 and higher includes keyring_aws, a plugin that communicates with the Amazon Web Services Key Management Service for key generation and uses a local file for key storage. (Enterprise)
● MySQL 5.7.13 and higher includes an SQL interface for keyring key management, implemented as a set of user-defined functions (UDFs).
#PerconaLive @RonaldBradford @bytebot
Encryption conclusions● InnoDB Tablespace Encryption: MySQL & Percona Server 5.7
○ MySQL Enterprise 5.7: Transparent Data Encryption■ MySQL Enterprise TDE uses a two-tier encryption key architecture, consisting of a master encryption key and
tablespace keys, which provides easy key management and rotation. ● Google patches for InnoDB/XtraDB/Aria (tmptable) tablespace encryption: MariaDB Server 10.1+● Eperi patch for InnoDB/XtraDB table encryption: MariaDB Server 10.1+● AWS KMS: MariaDB Server 10.2+ (in 10.1 you have to compile it)● Linux Unified Key Setup (LUKS)● Column encryption doesn’t exist today● Other solutions include Penta Security’s MyDiamo● Cloud? Amazon has its own encryption + KMS + IAM. Google uses AES-256, with symmetric keys.
Rackspace has a MariaDB Server encryption option.
#PerconaLive @RonaldBradford @bytebot
Linux Unified Key Setup (LUKS)Using crypt+LUKS, we can encrypt everything (data + logs) under one umbrella – provided that all files reside on the same disk. If you separate the various logs on to different partitions, you will have to repeat for each partition.
● Creating, Formatting and Mounting an Encrypted Disk● Creating a Backup of Encryption Information● Unmounting and Closing a Disk● Rotating Keys (Adding / Removing Keys)
https://www.percona.com/blog/2017/06/06/mysql-encryption-at-rest-part-1-luks/
#PerconaLive @RonaldBradford @bytebot
SQL Standard Roles● Bundles users together, with similar privileges - follows the SQL standard
● MariaDB Server 10.0 (10.1 adds that each user can have a DEFAULT ROLE)
● MySQL 8.0 DMR
CREATE ROLE audit_bean_counters;GRANT SELECT ON accounts.* to audit_bean_counters;GRANT audit_bean_counters to ceo;
https://mariadb.com/kb/en/mariadb/roles_overview/https://dev.mysql.com/doc/refman/8.0/en/roles.html
#PerconaLive @RonaldBradford @bytebot
Firewalls
#PerconaLive @RonaldBradford @bytebot
Firewall ImplementationMariaDB - MariaDB MaxScale Firewall Filter
● https://mariadb.com/resources/blog/maxscale-firewall-filter
ProxySQL Firewall http://www.proxysql.com/
MySQL - MySQL Enterprise Firewall
● https://www.mysql.com/products/enterprise/firewall.html
Datasunrise firewall for Percona
● https://www.datasunrise.com/firewall/percona/
#PerconaLive @RonaldBradford @bytebot
MariaDB MaxScale● Pre-2.0, GPLv2; post-2.0 BSL license (also, performance improvements;
SQLite parser)● The database firewall filter is used to block queries that match a set of rules. It
can be used to prevent harmful queries from reaching the backend database instances or to limit access to the database based on a more flexible set of rules compared to the traditional GRANT-based privilege system. Currently the filter does not support multi-statements.
● Needs configuration via whitelist/blacklist● Data masking - possible to obfuscate the returned value of a particular
column (e.g. don’t allow SSN numbers to go back)
#PerconaLive @RonaldBradford @bytebot
AirBnB MaxScale fork“The query blocklisting feature leverages the MySQL parse tree to look for existence of malformed predicate conditions in update and delete statements and reject such statements. For more protective coverage, we block MySQL update and delete statements without any predicate condition as well. We deployed Airbnb MaxScale with query blocklist feature and it has protected us from at least one instance of scary corrupted query that could have caused damage to one of our core database tables.”
https://medium.com/airbnb-engineering/unlocking-horizontal-scalability-in-our-web-serving-tier-d907449cdbcf
#PerconaLive @RonaldBradford @bytebot
ProxySQL● Has firewall and data masking● Query rules based on Google’s RE2-style regex● Query firewall - queries don’t have to hit the database● Validate query changes without impacting production database (“mirror”)
○ Match? Thread is cloned, inherits all attributes of original thread (can also redirect to a different hostgroup)
○ Upgrades? E.g. in 5.7 GENERATED is a reserved word; you can get it to rewrite queries
● Data masking to obfuscate the returned value of a particular column (e.g. don’t allow SSN numbers to go back)
#PerconaLive @RonaldBradford @bytebot
MySQL Enterprise Firewall● Arrived in MySQL 5.6● Block SQL injection attacks that don’t match the whitelist● Intrusion detection to detect what’s not matching a whitelist and alert DBA● Block out of policy SQL statements● Enforce the whitelist● Record approved incoming SQL statements; build whitelists on a per user
basis● Log SQL activity
#PerconaLive @RonaldBradford @bytebot
Firewall conclusions● Your only real choice is ProxySQL - GPLv3 friendly licensing● MySQL Enterprise Firewall will cost you money● MariaDB MaxScale is BSL licensed (which is versioned, currently 1.1)
○ Additional Use Grant: You may use the Licensed Work when your application uses the Licensed Work with a total of less than three server instances for any purpose.
○ The Licensor hereby grants you the right to copy, modify, create derivative works, redistribute, and make non-production use of the Licensed Work. The Licensor may make an Additional Use Grant, above, permitting limited production use.
○ https://github.com/mariadb-corporation/MaxScale/blob/2.1/LICENSE.TXT
#PerconaLive @RonaldBradford @bytebot
SQL Injection● Always using bind variables
● Escape input content
● Restricted "least" privileges○ Do not have GRANT ALL
#PerconaLive @RonaldBradford @bytebot
Other Security Considerations
#PerconaLive @RonaldBradford @bytebot
Improving Database Server Access● Restricting user access to your database server (login accounts)
○ Every physical person
■ dedicated OS login
■ dedicated database login
○ sudo restrictions (e.g. sudo su -)
■ Setup sudo group
■ Grant only specific commands to execute
○ Never share account details
○ Password expiry
● MFA
#PerconaLive @RonaldBradford @bytebot
Improving Database Server Access● Restricting traffic to your database server (open ports)
● Run a software firewall○ iptables, ufw
● You should use OS software meant to benefit security○ SELinux / Apparmor
#PerconaLive @RonaldBradford @bytebot
Improving Data Access● Restrict access to datadir
○ Don't put error log here
○ Don't put socket file here
○ Don't put pid file here
● Restrict access to view mysql.user table on filesystem○ Check out the examples of how to Hack MySQL
http://effectivemysql.com/downloads/MySQLSecurityEssentialsPerconaLive2015.pdf
#PerconaLive @RonaldBradford @bytebot
Installation● Using your Linux distribution… mostly gets you MariaDB when you ask for
mysql-server○ Except on Debian/Ubuntu
■ However, when you get mariadb-server, you get an authentication plugin — auth_socket for “automatic logins”
○ You are asked by debhelper to enter a password
● You can use the APT/YUM repositories from Oracle MySQL, Percona or MariaDB
● Don’t disable SELinux: system_u:system_r:mysqld_t:s0
#PerconaLive @RonaldBradford @bytebot
Update CadenceA security patch is so named because it improves security and generally addresses a means of attack of your system
● OS
● Database
● Application Stack
Why are you still running MySQL 5.5 or older?
#PerconaLive @RonaldBradford @bytebot
Deployment SecurityWho has control over running deployments?
i.e. deploying code that manipulates your data or structure
An application user SHOULD NOT have CREATE, ALTER, DROP privileges
● User to write data● User to read data● DBA to administer data (restricted to localhost)
#PerconaLive @RonaldBradford @bytebot
Use of Docker ContainersDocker shows a disregard for security with 'root' OS logins by default
● MySQL server installation approach via exposed passwords
● Configuration is contained with container○ Can you access the container via SSH
○ Can you copy and use container
#PerconaLive @RonaldBradford @bytebot
Reference Materialhttps://www.mysql.com/why-mysql/presentations/mysql-security-best-practices/
● MySQL Authentication and Password Policies
● MySQL Authorization and Privilege Management
● MySQL Encryption to secure sensitive data
● MySQL Enterprise Firewall to block database attacks such as an SQL
Injection
● MySQL Enterprise Audit to implement policy
#PerconaLive @RonaldBradford @bytebot
MySQL Manual Security Practiceshttp://dev.mysql.com/doc/refman/5.5/en/security.html
http://dev.mysql.com/doc/refman/5.6/en/security.html
http://dev.mysql.com/doc/refman/5.7/en/security.html
#PerconaLive @RonaldBradford @bytebot
(some of) Your questions answered :-)● Can you have SSL config options in mysql_config_editor?
○ Yes! Use login-paths. http://mysqlhighavailability.com/new-mysql-utilities-now-supports-ssl-and-configuration-files/
● In 8.0, you want to define mandatory roles with SET PERSIST?○ Read: https://dev.mysql.com/doc/refman/8.0/en/roles.html which saves the value for
subsequent server restarts
● Using Amazon AWS RDS and need the audit plugin via an option group?○ Read:
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.MySQL.Options.AuditPlugin.html (this works with MariaDB Server and MySQL)
121
Thank You Sponsors!
122
SAVE THE DATE!
CALL FOR PAPERS OPENING SOON!www.perconalive.com
April 23-25, 2018Santa Clara Convention Center
#PerconaLive @RonaldBradford @bytebot
Rate us! Thanks/Q&A
● Don’t forget to rate us in the app!● Ronald Bradford: @RonaldBradford /
http://ronaldbradford.com/blog● Colin Charles: @bytebot /
http://www.bytebot.net/blog/