undrop for innodb
TRANSCRIPT
UNDROP FOR INNODB Toolkit for MySQL Data Recovery https://twindb.com/
What’s InnoDB Table?
CREATE TABLE `actor` ( `actor_id` smallint(5) unsigned NOT NULL, `first_name` varchar(45) NOT NULL, `last_name` varchar(45) NOT NULL, `last_update` timestamp NOT NULL, PRIMARY KEY (`actor_id`), KEY `idx_actor_last_name` (`last_name`) ) ENGINE=InnoDB;
PRIMARY index -‐ B+ Tree Structure 1 100
page 2 page 3
1 50
page 2 page 3
100 150
page 2 page 3
actor_id 1 2 … 49
first_name A C E
last_name K D F
last_update X Y … Z
actor_id 50 51 … 99
first_name G I K
last_name H J L
last_update X Y … Z
Table in File System
# ls -‐la /var/lib/mysql/sakila/actor.* -‐rw-‐rw-‐-‐-‐-‐ 1 mysql mysql actor.frm -‐rw-‐rw-‐-‐-‐-‐ 1 mysql mysql actor.ibd
InnoDB Dictionary
mysql> SELECT * FROM SYS_TABLES WHERE NAME = 'sakila/actor'\G *************************** 1. row *************************** NAME: sakila/actor ID: 2642 N_COLS: 4 TYPE: 41 MIX_ID: 0 MIX_LEN: 80 CLUSTER_NAME: SPACE: 2337 1 row in set (0.00 sec)
SYS_TABLES
InnoDB Dictionary mysql> SELECT * FROM SYS_INDEXES WHERE TABLE_ID = '2642'\G *************************** 1. row *************************** TABLE_ID: 2642 ID: 6133 NAME: PRIMARY N_FIELDS: 1 TYPE: 3 SPACE: 2337 PAGE_NO: 3 *************************** 2. row *************************** TABLE_ID: 2642 ID: 6134 NAME: idx_actor_last_name N_FIELDS: 1 TYPE: 0 SPACE: 2337 PAGE_NO: 4 2 rows in set (0.00 sec)
SYS_INDEXES
Let’s DROP TABLE `actor`?
mysql> drop table sakila.actor; ERROR 1217 (23000): Cannot delete or update a parent row: a foreign key constraint fails mysql> SET FOREIGN_KEY_CHECKS=0; Query OK, 0 rows affected (0.00 sec) mysql> drop table sakila.actor; Query OK, 0 rows affected (0.01 sec)
actor.frm is removed
# ls -‐la /var/lib/mysql/sakila/actor.frm ls: cannot access /var/lib/mysql/sakila/actor.frm: No such file or directory
So is actor.ibd
# ls -‐la /var/lib/mysql/sakila/actor.ibd ls: cannot access /var/lib/mysql/sakila/actor.ibd: No such file or directory
Table’s removed from Dictionary
mysql> SELECT * FROM information_schema.INNODB_SYS_TABLES WHERE NAME = 'sakila/actor'; Empty set (0.00 sec)
Introducing stream_parser • Finds InnoDB pages in bytes stream
00000000 b7 da 39 28 00 00 00 0b ff ff ff ff ff ff ff ff |..9(............| 00000010 00 00 00 00 ff 98 e1 54 45 bf 00 00 00 00 00 00 |.......TE.......| 00000020 00 00 00 00 00 00 00 1c 25 cd 00 83 23 4f 04 4c |........%...#O.L| 00000030 00 00 00 02 00 01 00 73 00 00 00 00 00 00 00 00 |.......s........| 00000040 00 00 00 00 00 00 00 00 00 03 00 00 00 00 00 00 |................| 00000050 00 02 08 72 00 00 00 00 00 00 00 02 07 b2 08 01 |...r............| 00000060 00 00 03 00 8c 69 6e 66 69 6d 75 6d 00 09 05 00 |.....infimum....| 00000070 08 03 00 00 73 75 70 72 65 6d 75 6d 00 33 2f 2b |....supremum.3/+| 00000080 27 23 1d 16 10 08 00 00 10 13 00 ce 00 00 00 00 |'#..............| 00000090 00 00 00 0b 00 00 00 00 00 00 00 0b 00 00 00 00 |................|
stream_parser in action
# df -‐h /var/lib/mysql/ Filesystem Size Used Avail Use% Mounted on /dev/sdb 5.8G 146M 5.4G 3% /mnt/data # ./stream_parser -‐f /dev/sdb -‐t 5.8G Opening file: /dev/sdb … Size to process: 6227702784 (5.800 GiB) … Worker(0): 99.95% done. 2014-‐09-‐14 17:30:38 ETA(in 00:00:00). Processing speed: 808.000 MiB/sec All workers finished in 9 sec
stream_parser Result
# ls -‐la pages-‐sdb total 24 drwxr-‐xr-‐x 4 root root 4096 Sep 14 17:30 . drwxr-‐xr-‐x 7 root root 4096 Sep 14 17:30 .. drwxr-‐xr-‐x 2 root root 12288 Sep 14 17:30 FIL_PAGE_INDEX drwxr-‐xr-‐x 2 root root 4096 Sep 14 17:30 FIL_PAGE_TYPE_BLOB # ls -‐la pages-‐sdb/FIL_PAGE_INDEX | head -‐100| tail -‐5 -‐rw-‐r-‐-‐r-‐-‐ 1 root root 81920 Sep 14 17:30 0000000000000727.page -‐rw-‐r-‐-‐r-‐-‐ 1 root root 114688 Sep 14 17:30 0000000000000728.page -‐rw-‐r-‐-‐r-‐-‐ 1 root root 16384 Sep 14 17:30 0000000000000729.page -‐rw-‐r-‐-‐r-‐-‐ 1 root root 786432 Sep 14 17:30 0000000000000730.page -‐rw-‐r-‐-‐r-‐-‐ 1 root root 180224 Sep 14 17:30 0000000000000731.page
Where is my table? # grep -‐r PENELO pages-‐sdb/FIL_PAGE_INDEX Binary file pages-‐sdb/FIL_PAGE_INDEX/0000000000000022.page matches Binary file pages-‐sdb/FIL_PAGE_INDEX/0000000000000706.page matches Binary file pages-‐sdb/FIL_PAGE_INDEX/0000000000000747.page matches # hexdump -‐C pages-‐sdb/FIL_PAGE_INDEX/0000000000000022.page | head -‐15 00000000 ea a2 dd d0 00 00 00 03 ff ff ff ff ff ff ff ff |................| 00000010 00 00 00 00 00 1b 82 20 45 bf 00 00 00 00 00 00 |....... E.......| 00000020 00 00 00 00 00 06 00 33 1d cb 80 ca 00 00 00 00 |.......3........| 00000030 1d ad 00 02 00 c7 00 c8 00 00 00 00 00 00 00 00 |................| 00000040 00 00 00 00 00 00 00 00 00 16 00 00 00 06 00 00 |................| 00000050 00 02 00 f2 00 00 00 06 00 00 00 02 00 32 01 00 |.............2..| 00000060 02 00 1c 69 6e 66 69 6d 75 6d 00 05 00 0b 00 00 |...infimum......| 00000070 73 75 70 72 65 6d 75 6d 07 08 00 00 10 00 29 00 |supremum......).| 00000080 01 00 00 00 00 07 40 c0 00 00 01 63 01 10 50 45 |[email protected]| 00000090 4e 45 4c 4f 50 45 47 55 49 4e 45 53 53 43 f2 f5 |NELOPEGUINESSC..| 000000a0 a9 08 04 00 00 18 00 26 00 02 00 00 00 00 07 40 |.......&.......@| 000000b0 c0 00 00 01 63 01 1a 4e 49 43 4b 57 41 48 4c 42 |....c..NICKWAHLB| 000000c0 45 52 47 43 f2 f5 a9 05 02 00 00 20 00 21 00 03 |ERGC....... .!..| 000000d0 00 00 00 00 07 40 c0 00 00 01 63 01 24 45 44 43 |[email protected].$EDC| 000000e0 48 41 53 45 43 f2 f5 a9 05 08 04 00 28 00 27 00 |HASEC.......(.'.|
InnoDB Dictionary
• We need it for: a) index_id b) Table structure
Recover InnoDB Dictionary # ./stream_parser -‐f /var/lib/mysql/ibdata1 Opening file: /var/lib/mysql/ibdata1 Size to process: 12582912 (12.000 MiB) All workers finished in 0 sec # ./recover_dictionary.sh Generating dictionary tables dumps... OK Creating test database ... OK Creating dictionary tables in database test: SYS_TABLES ... OK SYS_COLUMNS ... OK SYS_INDEXES ... OK SYS_FIELDS ... OK All OK Loading dictionary tables data: SYS_TABLES ... 46 recs OK SYS_COLUMNS ... 304 recs OK SYS_INDEXES ... 96 recs OK SYS_FIELDS ... 118 recs OK All OK
Introducing sys_parser
# yum install mysql-‐community-‐devel.x86_64 # make sys_parser /usr/bin/mysql_config cc `mysql_config -‐-‐cflags` `mysql_config -‐-‐libs` -‐o sys_parser sys_parser.c
# ./sys_parser sakila/actor CREATE TABLE `actor`( `actor_id` SMALLINT UNSIGNED NOT NULL, `first_name` VARCHAR(45) CHARACTER SET 'utf8' COLLATE 'utf8_general_ci' NOT NULL, `last_name` VARCHAR(45) CHARACTER SET 'utf8' COLLATE 'utf8_general_ci' NOT NULL, `last_update` TIMESTAMP NOT NULL, PRIMARY KEY (`actor_id`) ) ENGINE=InnoDB;
Getting index_id from Dictionary
mysql> SELECT SYS_TABLES.NAME, SYS_INDEXES.NAME, SYS_INDEXES.ID FROM SYS_TABLES LEFT JOIN SYS_INDEXES ON SYS_TABLES.ID = SYS_INDEXES.TABLE_ID WHERE SYS_TABLES.NAME = 'sakila/actor'; +-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐+ | NAME | NAME | ID | +-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐+ | sakila/actor | PRIMARY | 22 | | sakila/actor | idx_actor_last_name | 23 | +-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐+ 2 rows in set (0.00 sec)
Checklist before Launch
ü InnoDB pages (credit to stream_parser) ü Table Structure (credit to sys_parser) ü index_id (InnoDB dictionary)
Introducing c_parser
# ./c_parser -‐5f \ pages-‐sdb/FIL_PAGE_INDEX/0000000000000022.page \ -‐t actor.sql \ > dumps/default/actor 2> dumps/default/actor.sql
c_parser Result
# head -‐5 dumps/default/actor -‐-‐ Page id: 3, Format: COMPACT, Records list: Valid, Expected records: (200 200) 000000000740 C0000001630110 actor 1 "PENELOPE" "GUINESS" "2006-‐02-‐15 04:34:33" 000000000740 C000000163011A actor 2 "NICK" "WAHLBERG" "2006-‐02-‐15 04:34:33" 000000000740 C0000001630124 actor 3 "ED" "CHASE" "2006-‐02-‐15 04:34:33" 000000000740 C000000163012E actor 4 "JENNIFER" "DAVIS" "2006-‐02-‐15 04:34:33"
# cat dumps/default/actor.sql SET FOREIGN_KEY_CHECKS=0; LOAD DATA LOCAL INFILE '/root/undrop-‐for-‐innodb/dumps/default/actor' REPLACE INTO TABLE `actor` FIELDS TERMINATED BY '\t' OPTIONALLY ENCLOSED BY '"' LINES STARTING BY 'actor\t' (`actor_id`, `first_name`, `last_name`, `last_update`);
Load Table Back
# mysql sakila < actor.sql
# mysql sakila < dumps/default/actor.sql
# mysql sakila -‐e "SELECT COUNT(*) FROM actor" +-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+ | COUNT(*) | +-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+ | 200 | +-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+
Follow Us ü https://twindb.com ü https://twitter.com/twindba ü https://www.facebook.com/twindba ü https://www.linkedin.com/company/twindb