Quantcast
Channel: Liquibase Forums
Viewing all 2993 articles
Browse latest View live

Re : Liquibase reverted grant after updating PLSQL function (DB - PostgreSQL)

$
0
0
Liquibase doesn't do anything with grants, my guess is that for some reason the database is dropping them. Are you doing drop/creates where you could be doing alter statements?

Nathan

Re : Informix generateChangeLog issue

Re : Table and column remarks => dbDoc

Re : Liquibase taking too long to execute changeset

Re : UpdateSQL ignores dbms type?

$
0
0
I'm using liquibase 3.3.2, run through maven.

I have some changesets that are ran on only one database type.  Sample:
<changeSet author="nthorn" id="1418685979642-17-hsql">
    <preConditions onFail="MARK_RAN">
        <dbms type="hsqldb" />
    </preConditions>
<createTable tableName="Message">
...
</createTable>
</changeSet>

They work just fine when liquibase runs against the database directly, but when I run updateSQL (with the properties setting "url : jdbc:sqlserver://localhost:1433;databaseName=my_db") I am getting this in the output SQL file:

-- Changeset ../ourApp/src/main/resources/changelogs/1.4.0-changelog.xml::1418685979642-17-hsql::nthorn
CREATE TABLE [Message] ( ... )
GO
INSERT INTO [DATABASECHANGELOG] (... , [COMMENTS], [EXECTYPE], [LIQUIBASE]) VALUES (... , 'createTable', '', 'EXECUTED', '3.3.2')
GO

So it looks to me like my precondition is failing, and I'm wondering if I might just not be telling liquibase the type of database it is addressing properly.


Thanks for the help Nathan,
Nate

Re : Understanding validCheckSum

$
0
0
Thanks for reminding me. I updated the docs, let me know what you think.

Nathan

Liquibase doesn't honor type modifiers

$
0
0
Code:

  1. AddColumn.Column col = new AddColumn.Column();
  2. ...
  3. col.setType("INT(10) ZEROFILL UNSIGNED");
  4. changeSet.getChangeSetChildren().add(addCol);

Output:

  1. ALTER TABLE tablename ADD testcolumn INT NULL;

... which means both the zerofill and unsigned modifiers are completely lost.

Is there a way to make Liquibase use the EXACT value passed to setType ?



The above targets MySQL, using the latest available driver, but I get the same problem with older drivers.


Re : Liquibase taking too long to execute changeset

$
0
0
Hi Nathan

Thanks a lot for your reply and creating a ticket in liquibase core forum. I am just curious to know did you find similar issue when you were doing performance improvements on version 3.2.0? I will definitely take your suggestion for upgrading liquibase to the newer version. However do you know is there any similar issue fixed in 3.3.2? If you can provide me any related details that will be great and I can share those with my team, take your suggestions further. I will eagerly wait for the response on the other ticket as well.

Thanks
Sayee

Re : Extending Liquibase to support Cassandra

$
0
0
I am also looking for same. From where i get this modified jar file.

Maximum csv file size for loadData

$
0
0
Hi, I'm trying to use Liquibase to import data into a large database table. I'm running with this command:

liquibase --driver=oracle.jdbc.OracleDriver ^
     --classpath="C:\tools\tomcat-ex\lib\oracle-jdbc-11.2.0.2.0.jar" ^
     --changeLogFile=abc-schema-base.xml ^
     --url="jdbc:oracle:thin:@abc.def:1521:hijklmnop" ^
     --username=abcdef ^
     --password=abcdef ^
     --logLevel=DEBUG ^
     update

The changeset is using loadData to import a csv file. The csv file I'm importing is 650Mb. Whenever I run the update, I get OutOfMemoryErrors (after a couple of hours of waiting).

Both the changeset and csv were generated with generateChangeLog. So full marks to Liquibase for at least being able to extract the large table!

I'm running with -Xmx2048m, and Liquibase 3.3.2.

I'm considering breaking the csv into multiple smaller files and multiple changesets, but I'm unsure how small the csv's would need to be. Does anyone have any info on the largest csv filesize Liquibase loadData can handle?

Has anyone else encountered and resolved this kind of problem?

Alternatively I can exclude this table from my Liquibase scripts as it's the only one causing issues.

Thanks in advance,
Adam.

Re : Maximum csv file size for loadData

$
0
0
Seems like something that should and could be fixed in Liquibase.

Just curious - why you are using Liquibase to load that much data? What is the use case? 

I would recommend that you use native tools if possible when loading that much data into a database. Liquibase is really intended for managing the structure of a database rather than the contents. It is able to work with the data also, but it is intended mainly for loading small sets of data - tables full of constants for example, or small test data. Since Liquibase works at the JDBC level, what it ends up doing is generating tons of INSERT statements, which is an extremely inefficient way to load bulk data. 

Steve Donie
Principal Software Engineer
Datical, Inc. http://www.datical.com/

Re : Extending Liquibase to support Cassandra

$
0
0
It should be at https://github.com/liquibase/liquibase-cassandra/releases but it looks like there have been no builds yet. It's a community-contributed extension so I haven't really looked at it much.

A quick look at the current state of the code appears that I have a few changes to make it work with 3.3.2. I'll look at getting that done quickly and and put a snapshot build up for testing.

Nathan

Re : Maximum csv file size for loadData

$
0
0
Thanks for your response Steve,

The reason for doing this is that I'm migrating a legacy application to Liquibase, and it would be convenient if I could build the entire database from liquibase, rather than using multiple tools. The main benefit Liquibase gives me is versioning with changesets, and running an import from another tool won't easily fit into the changset model.

Currently my legacy application has many unmanaged and unversioned database snapshots, which makes it very difficult to run repeatable CI cycles of: develop, build, test, release, deploy.

Like you say, there are more efficient ways to load large datasets, and I may need to take that approach instead.


generateChangeLog reports success but no changes listed

$
0
0
I'm trying to run generateChangeLog on an existing derby database. It has a number of tables in it.

c:\Projects\core\cams>java -jar lib\liquibase.jar --classpath=derby.jar;derbyclient.jar --driver=org.apache.derby.jdbc.EmbeddedDriver --url="jdbc:derby:CAMSDB" --changeLogFile=db-changelog.xml --logLevel=debug --logFile=log.txt generateChangelog

The resulting output indicates success:

INFO 13/03/15 14:27: liquibase: Can not use class liquibase.serializer.core
.json.JsonChangeLogSerializer as a Liquibase service because org.yaml.snake
yaml.representer.Representer is not in the classpath
INFO 13/03/15 14:27: liquibase: Can not use class liquibase.serializer.core
.yaml.YamlChangeLogSerializer as a Liquibase service because org.yaml.snake
yaml.representer.Representer is not in the classpath
Liquibase 'generateChangelog' Successful

The debug trace is: 

DEBUG 13/03/15 14:27: liquibase: Connected to APP@jdbc:derby:CAMSDB
DEBUG 13/03/15 14:27: liquibase: Setting auto commit to false from true
DEBUG 13/03/15 14:27: liquibase: Executing QUERY database command: select current schema from sysibm.sysdummy1
DEBUG 13/03/15 14:27: liquibase: Computed checksum for 1426256831398 as 0815fc1f3d5660bee0ff4ef70e42471d
INFO 13/03/15 14:27: liquibase: db-changelog.xml does not exist, creating
DEBUG 13/03/15 14:27: liquibase: MissingObjectChangeGenerator type order:     liquibase.structure.core.Catalog    liquibase.structure.core.Schema    liquibase.structure.core.Sequence    liquibase.structure.core.StoredProcedure    liquibase.structure.core.Table    liquibase.structure.core.Column    liquibase.structure.core.PrimaryKey    liquibase.structure.core.UniqueConstraint    liquibase.structure.core.ForeignKey    liquibase.structure.core.Index    liquibase.structure.core.View
DEBUG 13/03/15 14:27: liquibase: UnexpectedObjectChangeGenerator type order:     liquibase.structure.core.Catalog    liquibase.structure.core.ForeignKey    liquibase.structure.core.Schema    liquibase.structure.core.StoredProcedure    liquibase.structure.core.UniqueConstraint    liquibase.structure.core.View    liquibase.structure.core.Table    liquibase.structure.core.PrimaryKey    liquibase.structure.core.Column    liquibase.structure.core.Index    liquibase.structure.core.Sequence
DEBUG 13/03/15 14:27: liquibase: ChangedObjectChangeGenerator type order:     liquibase.structure.core.Catalog    liquibase.structure.core.ForeignKey    liquibase.structure.core.Schema    liquibase.structure.core.Sequence    liquibase.structure.core.StoredProcedure    liquibase.structure.core.Table    liquibase.structure.core.Column    liquibase.structure.core.PrimaryKey    liquibase.structure.core.Index    liquibase.structure.core.UniqueConstraint    liquibase.structure.core.View
INFO 13/03/15 14:27: liquibase: Shutting down derby connection: jdbc:derby:CAMSDB;shutdown=true

but the created xml file is substantially empty



Why is it not working?





Problem using HSQL in Test profile

$
0
0
Note: I am new to Liquibase and HSQL.

The application uses Oracle and JPA. Through liquibase performing DDL and DML to initialize the environment. Everything works fine at the time of deployment.

The issue is only when I want to use in-memory HSQL in place of Oracle.

At first, I just changed the datasource to HSQL. The Liquibase script ran successfully from JPA Integration test failed when calling sequence nextval. From Stackoverflow I found that I should use Oracle Dialect like
'jdbc:hsqldb:mem:testdb;sql.syntax_ora=true '

If I use this then the liquibase script fails for a DML that populates Date field from CSV.

<column name="EFFECTIVE_DATE" type="DATE"/>
The CSV has value in 'YYYY-MM-DD' format. When used with Oracle datasource, it uses to_date function and inserts correctly
but when used with HSQL Oracle dialect, this fails with invalid date format.

Need help in making this work with HSQL as well.

Thanks



Executing Stored Procs across platforms

$
0
0
The syntax for executing a stored proc is different across databases (call vs exec vs nothing, etc)

RIght now we would have to have different <sql> sections for a single stored proc call for each database we support. We actually setup quite a bit of data this way.

Does anyone know if there are plans to support this as part of core or if there is already an extension out there somewhere?
I'd be looking for something like:

<changeSet id="1" author="Zorax">
<execProc proc="foo(1, 'bar')"/>
</changeSet>

Support AutoIncrement in creating objects

$
0
0
Hi.

I would like to use liquibase not only to install or update a database schema for my application, but also to create some objects using the auto_increment feature of the database and create links between the objects, using their id's. As long as i understand, the current version of liquibase is not supporting this feature.

Let's say we have a person and an address table, with a 1:1 relationship. Both tables are using the auto_increment feature of the database and the table person contains a link to an entry in the adress table, represented by its id, which also has a not null and a foreign key constraint. When i want to create a person with an associated address using liquibase, the only way i see is to create an sql statement, that inserts the address, fetches the id of the newly created address and then inserts the person with the id fo the address or i depend on the execution order and guess the id's. Guessing the id's, the changeset might look like this:

    1. <createTable tableName="address">
    2.   <column name="id" type="int" autoIncrement="true">
    3.     <constraints primaryKey="true" nullable="false"/>
    4.   </column>
    5.   <column name="street" type="varchar(50)" />
    6. </createTable>
    7. <createTable tableName="person">
    8.   <column name="id" type="int" autoIncrement="true">
    9.     <constraints primaryKey="true" nullable="false"/>
    10.   </column>
    11.   <column name="name" type="varchar(50)" />
    12.   <column name="address_id" type="int">
    13.     <constraints references="address" foreignKeyName="id" nullable="false"/>
    14.   </column>
    15. </createTable>

    16. <insert tableName="address">
    17.   <column name="street" value="Sesame street" />
    18. </insert>
    19. <insert tableName="person">
    20.   <column name="name" value="Kermit" />
    21.   <column name="address_id" value="1" />
    22. </insert>
    This is bad code, i think. What if i would like to create entries in an update? There might be already data in these tables and i'm getting in trouble, guessing the wrong id's. Writing SQL statements would even be worse.

    Thinking about better solutions, i thought about creating the schema using liquibase and creating objects in the application, which uses hibernate. This would be pretty simple, because i'm already creating objects in my application. But what about updates? What about error handling and rollback? I'm also a fan of the Single-Responsibility-Principle, which is broken, by doing some database installation/updates using liquibase and some in the application code.

    No, the best solution would be, to create the objects using liquibase, but how can this be done? I thought about something like this:

    1. <insert id="sesame_street" tableName="address">
    2.   <column name="street" value="Sesame street" />
    3. </insert>
    4. <insert tableName="person">
    5.   <column name="name" value="Kermit" />
    6.   <column name="address_id" reference="address" id="sesame_street" />
    7. </insert>
    The first insert gets an optional attribute id, which should be unique inside the change set and according to the table. The second column element of the second insert gets the optional attributes reference and id, containing the referenced table and the given id of the first insert element. I think, this is what a user would like to have. There had to be some constraints, for example, that a reference to an insert statement is limited to the changeset. The reference and the source had to be in the same changeset, because the id can only be determined on creation. Another constraint would be the order of the insert statements. The source had to be inserted before the reference is used.

    Looking at the liquibase api, i thought that it might be possible, to change some sql statements, in order to get the newly created id of an insert statement. Then i have to associate this table id with the given id of the insert element in the changeset. Then i have to extend the column element, where i have to set the right value, if there is a reference to an id field of another table. This is not solving every problem, creating and updating objects, but i think it is a step in the right direction.

    Is this complete non-sense, what i'm thinking about? Is it possible, to write an extension for this? Any ideas, suggestions or comments are appreciated.

    Volker.

    Re : Support AutoIncrement in creating objects

    $
    0
    0
    I would humbly suggest that Liquibase is the wrong tool for the job. I see Liquibase's single responsibility as 'manage database schema changes' and loading data is not a part of that responsibility. If you are using it to load test data, that seems like a reasonable use, maybe, but in that case your tests should not be tightly coupled to implementation details like what the id is for a raw that was inserted. If you are using Liquibase to load 'constant' data (i.e. US States table that is used for many address-type tables) then that should be OK too - just hard-code the IDs and ensure that every instance of the database has the same ones.  

    Steve Donie
    Principal Software Engineer
    Datical, Inc. http://www.datical.com/

    validating if databasechangelog has more commits than in provided log file

    $
    0
    0
    Hi,

    Is there any way to tell liquiebase to validate if the provided changelog file has all  changesets already applied to database,
    and in case it doesn't - throw an exception?

    e.g
    there were 2 changes already applied to database,
    but now someone tries to launch an application which has only 1 changeset in the liquibase change log file. (i would like such application to fail to start, because it was designed for an older version of database)

    Regards,
    Vitaliy S

    Re : Removing autoincrement from the column.

    Viewing all 2993 articles
    Browse latest View live


    <script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>