Few years back we were working on a project where we needed real-time synchronization of data along with storing that same data into your run-of-the-mill database. Sounds like a simple enough request, right? But there is no out-of-the-box solution! Then, as now, developers and architects usually end up constructing a complex contraption with MySQL or some other database on one side and an in-memory solution capable of PUB-SUB on the other side, like e.g. Redis. And as is usually the case - things get out of hand in terms of maintenance, development and monitoring. You need to keep track of all your subscribers (be it users or other services), make sure you don’t exceed resources at hand, scale your services in peak times etc.
So, even though we ended up creating that same contraption out of messaging queues, mysql and Redis, we decided to make our lives simpler - and PubSubSQL was born!
A couple of us got together and decided to take the advantage of Go to write a PUB-SUB passthrough service that would at the same time allow SQL clients to get notified of new or updated data as well as use the usual SQL functionalities without needing to care about intricacies of distributing messages, payload or performance.
Due to our day-jobs the project was dormant for the last year but we revisited the idea and decided to revive it. Version that is currently in front of you is basic, updated to work with latest versions of Go and MySQL but it is not production ready yet. We are working hard to improve the in-memory components and better the coupling between PSS and MySQL.