MySQL down due to NFS bug

Team was in the middle of doing RAM upgrade for a host. Once the upgrade was done and mysql was started, it failed to come up with the following error :

InnoDB: Check that you do not already have another mysqld process
InnoDB: using the same InnoDB data or log files.
InnoDB: Error in opening ./ibdata1
050415 15:03:41 InnoDB: Operating system error number 11 in a file operation.
InnoDB: Error number 11 means ‘Resource temporarily unavailable’.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/mysql/en/Operating_System_error_codes.html

Access to the ibdata file was perfect with no issues. The filesystem was mounted on NFS. After some scratching figured out it might be a combination of an NFS bug and innodb advisory locking that is taking us down.

Just doing a cp -a on the file to a different filename and moving that new file to replace the locked file did the trick. This needs to be done to all innodb data files and the innodb log files. Whew!

3 Responses

  1. Do you made good experience with MySQL innoDB on NFS Filesystem?

  2. We had a 300 GB innodb database working well on NFS. But that was just a test done on one of our slaves. It was doing just fine.

  3. have you/anyone tried MySQL myisam on Hadoop DFS/Gluster FS?
    please let me know your feedback as i want to move from MySQL MyISAM on NFS to hadoop/gluster fs. thanks.

Leave a Reply