Today is two years since I started working on pg_chameleon. Back in 2016 this commit changed the project’s license from GPL v2 to the 2 clause BSD and the project’s scope, which became a MySQL to PostgreSQL replica system.
Since then I learned a lot of lessons, made several mistakes and worked out solutions which resulted in a decent tool for bridging two different universes.
Writing a replica system is a very complex task.
Has been a while since I wrote a post on this blog.
I’ve been quite busy on coding pgchameleon’s version 2.
I finally managed to release an alpha1 the 11th of November but this release had several issues which prevented the users to have real tests.
However, after a week of debugging I’ve released pg_chameleon v2.0.0.alpha2 which is now quite usable.
For a complete command line reference and an usage example click here.
Today is one year since I started working seriously on pg_chameleon.
With this commit I changed the project’s license to the 2 clause BSD and the project’s scope, evolving the project into a MySQL to PostgreSQL replica system.
Initially this change was just a try. I needed to synchronise the data between MySQL and PostgreSQL and at that time the only option I had it was to use the MySQL foreign data wrapper, eventually to copy the data locally every now and then.
Bugfix emergency release v1.3.1 I discovered a regression when running the init_replica caused by a wrong handling of missing time in master coordinates. Sorry about that.
After another round of bug fixes I’ve released the version 1.3 of my pet project pg_chameleon.
The package is available on pypi as usual.
The changelog is available here.
If you have any question/issue to discussI created a community on gitter.
Please join!
I’ve also added a RELASE_NOTESfile to explain the changes.
Last week I announced the stable release of pg_chameleon as I felt quite confident the tool worked properly.
However the murphy law is always ready to teach us we live in an interesting universe. A couple of days after the release I noticed on a server which data were modified seldom a strange replica issue. For some reason at specific moments of the day the inserts replayed on the postgres tables failed with the primary key violation.
In 2013 I started my pet project pg_chameleon to discharge my frustration only to find the entire thing even more frustrating.
It’s fascinating how time and circumstances can fix the personal perspective revamping the passion for development that never left me.
This philosophical introduction is to tell I still can’t believe I’ve relased the RC1 of my pet project pg_chameleon.
The release is already available on pypi and includes several bug fixes and performance improvement.
After fixing few bugs I decided to release the first pg_chameleon beta.
The package details with the changelog are available here http://www.pgdba.co.uk/p/mysql-to-postgresql-replica.html
These are the notable changes.
Switch to psycopg2 2.7.x - Thanks to Daniele’s work now the requirement installs without need for python or postgresql headers, making the install in virtualenv much simpler Install system wide or in virtualenv follow the same rules. The configuration files and the package files are installed in the python’s site-packages/pg_chameleon.
Over the last week end I finally worked trough python’s setuptools and relased the alpha4 for pg_chameleon 1.0.
This is the final alpha version the version 1.0.
The system now can be installed in the virtualenv just typing
pip install pg_chameleon
It’s important pip is up to date so if there’s any error upgrade pip with
pip install pip –upgrade
The first install creates in the user’s home the directory .
Thanks everybody for listening. And massive thanks to the PostgreSQL Europe staff for organising an amazing conference.
The slides from my talk are available fordownload here.
The open source replica system is on github https://github.com/the4thdoctor/pg_chameleon and hopefully Transferwise will also release the customised obfuscator soon.
As mentioned in the presentation I did at the Brighton PostgreSQL Meetup, pg chameleon is a commute project.
In general I work on it when I travel. And thanks to the pgconf and other requirements I travelled a lot recently.
This is the presentation’s recording (sorry the audio is suboptimal).
So I was able to build a minimum viable product which seems to work properly.
The alpha1 release is available for download and testing here
