TingoDB API and features designed to be close compatible with MongoDB. As an additional proof for this concept we managed to implement experimental driver for Mongoose.js ODM libary. Work is still in progress, but basic functions work, and we hope to make it usable for real applications. For TingoDB this means that it gets a lot of additional tests from complex tool and it will help to improve its quality.
It is trivial thing and we never used it. But others do, and now it is implemented. As usual it fully support indexes where appropriate. $exists operation is available starting from version 0.1.7
At least this was our story. Our two years old choice of AlfredDB was wrong. It looks promising, but as we start to work with real data-sets (>10k of records) it became nightmare. We spent months trying to fix it, but, no luck. The main problem is that buggy database engine prevent us from active development of our main application while it was a priority.
One day we decided that we can’t get such risks anymore. We decided to switch to MongoDB with goal to complete our application without worrying about bugs in database. Surprisingly migration was not easy. The fact that database uses MongoDB alike query syntax and has find and findOne methods do not means that it is easy to replace it with MongoDB. For those who saw The Pulp Fiction there are too much of little differences – “It’s the little differences. I mean, they got the same shit over there that we got here, but it’s just…it’s just, there it’s a little different”.
While one part of team work with application itself and didn’t worry about database issues another part of team was building embedded database that will be capable to transparently replace Mongodb. It was not easy but if you focus on this from beginning you can do it. First our decision was that we must have good test coverage and all tests must pass both with MongoDB and TingoDB. Looking back, this was very important decision, we was not need to invent something, we just need to reproduce existing functionality.
Once some core functionality was ready we come to idea that we can reuse tests from MongoDB Node.js driver project. It was not easy as well. We discover that original API is designed to be way too flexible. Many functions can work with or without callback. Some functions accept many different parameter combinations. We have surprised to found that find method support 7 different parameter combinations and on every its call there are dozens of code lines that just guess what you actually want to do. To be honest we was upset with that, but now we at least know how to call it to make this code run faster. Also tests verify use of many options or behaviours that we never used in real life and was not aware. No matter what we precisely duplicate all behaviours including exception error messages and even bugs (as we think about them).
Initially we didn’t implement it because it is kind of useless for in-proc implementation as you always can do any post filtering with same efficiency.
However based on user feedback we decided to implement it for compatibility purposes.
The $where clause is available starting from version 0.1.5
While implementing our app we come to situation when multiple indexes on same collection significantly slowdown search. In our example we have collection that store information about financial transactions. To speedup app we need to maintain separate helper collection that holds links between accounts and transaction splits. This helper collection holds account and transaction ids with some derivative information like date and sum. It has indexes for both account and transaction ids. During update we use search query that include both ids and by default TingoDB tries to use both indexes. It found ranges and then intersect them. With some experimentation we found that using only single transaction id index can speedup thing a lot. This happens because transactions usually consists from few splits (usually two). So, using only one transaction index to fetch those few records with additional check of account id fields appears to be much faster that using both indexes.
Probably this is good example why MongoDB support search hint option which you can use to point exact index you want to use. So it was easy to add this support and we did it.
Hints option is available in TingoDB since version 0.1.3
After about 3 months of work we proud to announce first public release of TingoDB.
Module is pushed to npm repository and some initial documentation is ready.
So far we successfully integrated it into our application and it works for us the same good as Mongodb.
Take a look, and give it a try !