Monthly Archives: September 2018

Revisiting Embedded InnoDB

Many people these days don’t know InnoDB was originally developed as an independent database engine apart from MySQL. Its author, Heikki Tuuri, modeled InnoDB after Transaction Processing: Concepts and Techniques, the seminal transaction processing book authored by Turing Award laureate James “Jim” Gray and Andreas Reuter. It wasn’t until later InnoDB was integrated with MySQL.

While InnoDB contains a very simple, barebones, parser and executor, it’s major advantage was in its solid transactional storage engine implementation. The advantage to using MySQL for the high-level database functionality was that a full SQL parser, optimizer, et al didn’t have to be developed and InnoDB could focus on its strengths; this was especially true when compared to the two other popular storage engines of the day, MyISAM and the Berkeley DB (BDB) Storage Engine, which–when I met with Keith Bostic in 2005–it seemed Sleepycat had no interest in pursuing. Several years later, after InnoDB was already considered the defacto production-ready storage engine for MySQL, Innobase Oy released the Embedded InnoDB, which enabled developers to embed the open-source InnoDB database engine directly within their applications. As I had done some work years before to extract the storage engine from MySQL and use it in my own project, Prophet, this was a pretty awesome turn of events. Sadly, however, it ended-up being overlooked and died pretty quickly.

Given the death of the official Embedded InnoDB, developer Stewart Smith took up the reins, forked the codebase, and began a separate project he called HailDB. This continued for some time and resulted in a couple releases. Eventually, however, the same disinterest Innobase saw resulted in the project being canned as well.

For posterity, I’ve uploaded a copy of the original release of the official Embedded InnoDB source code as well as the documentation not included in the source distribution. Likewise, I’ve committed the code to the last official release of HailDB along with example code demonstrating how to develop against the embedded database engine.

You may ask, “what would it take to get the latest InnoDB working in embedded mode?” Well, I’ve done a cursory look and I believe it would take a week or so to re-work things from the current C++-based MySQL server codebase and swap out the items required for it to be standalone again. While I have several cool uses for this, including my old drop-in replacement libraries for Btrieve and Microsoft Jet/Extensible Storage Engine (ESENT), it’s not worth my free-time to work on.