I used to manage database deployments via sourcing .sql text files
and it was problematic.
I love the ability in Liquibase to have error handling and
incremental idempotency in my SQL deployment operations, both
command-line and via Maven builds. Thanks to creative use of RunAlways
and Preconditions I am even able to create TDD unit tests for
liquibase operations that ensure certain operations run always and can
be ensured to remain true as part of each code build or database deployment!
However, the latest of the applications we've built is
trying to take our SQL data and push it to Hadoop (via Sqoop) for
predictive analysis there that is very computationally intensive.
So I am now also supporting Hive schema on our Hadoop cluster, at
least some of which mirrors our SQL schema very closely. (Hive is
an open-source SQL implementation that runs across a Hadoop
cluster and can be connected to with a standard JDBC driver.)
So after championing Liquibase amongst my technical peers I find
myself in the predicament that I have to return to the days of
sourcing some of my DDL in shell scripts with no error handling, and
I can't use Liquibase for all DDL changes related to our applications!
I don't really need Liquibase to generate "HiveQL DDL" via its tags, but I at
least need to be able to specify such sorts of DDL in a <sql>
tag like I do with our existing database-specific DDL in Liquibase.
It seems plausible/feasible at a high level to me because one
just needs to have Liquibase pass the DDL through a JDBC driver and
read back the responses, right? I am probably grossly oversimplifying.
Is deploying DDL for Hive/Hadoop via Liquibase possible via some
existing means? Is this even on the radar for the future?