Quantcast
Channel: Liquibase Forums
Viewing all 2993 articles
Browse latest View live

Re : maven generateChangeLog several files

$
0
0
Not automatically, no. If you wanted to do it automatically, you would need to write some sort of post-processor. If you're only doing it once I would just manually organize things. 

Steve Donie
Principal Software Engineer
Datical, Inc. http://www.datical.com/

Liquibase logfile leaves commandline empty

$
0
0
When I use the --logLevel parameter at the command-line liquibase prints the logging on the command-line. But when I additionally use the --logFile parameter then it prints them in a file without printing it in the command-line. Is there any way to let liquibase print it in the command-line AND in a logfile? I am using liquibase 3.5.5. and this is what I give liquibase as parameters:

liquibase --url=offline:oracle? `
--changeLogFile="C:\Users\Ferid\Documents\Box Sync\PRIVATE_Ferid\liquibase-3.5.5-bin\cl.xml" `
--outputFile="SQLoutput.sql" `
--logLevel="info" `
--logFile="log" updateSql

Re : Liquibase logfile leaves commandline empty

Re : Liquibase logfile leaves commandline empty

Introduction to Python Programming

$
0
0

Fortunately, an experienced programmer in any programming language (whatever it may be) can pick up Python very quickly. Indeed, it’s also easy for beginners to use and learn. So why are we being late?  Let’s get started.

Python is a dynamic, interpreted (bytecode-compiled) and very reliable, a high-level programming language that allows you to achieve more in lesser lines of code. There are no type declarations of variables, parameters, functions, or methods in the source code. This makes the code very short and flexible, and you lose the compile-time type checking of the source code. Python tracks the types of all values at runtime and flag code that does not make sense as it runs.

Mindmajix Python Programming Training course helps you learn the Python language. Enroll Now!

The programming model offered by Python allows you to carry out coding in various styles such as functional, procedural or object-oriented. It offers a huge library, carries out memory management automatically and has a dynamic type of system. It can be installed on various operating systems.

An excellent way to see how Python code works is to run the Python interpreter and type code right into it. If you ever have a question like, “What happens if I ever add an int to a list?” Just typing it into the Python interpreter is a fast and likely the best way to see what happens.

Another great feature of Python is dynamic name resolution, which binds methods and variable name when the program is executed, it is an amazing feature and you will realize this when you start working on it.

Here are some reasons why Python is preferred over other programming languages:

  • Python code is robust and easy to maintain. It is less prone to issues and is very easy to debug. There is no doubt that in today’s date it is one of the most powerful and easy to understand language.
  • Python is a flexible language. That is the main reason why it is being used to develop the web as well as desktop applications. It is also used for system administration and voice over IP.
  • The syntax used in Python programming is very simple, straightforward and easy to implement. For example, there is no need to define the type of variable; and it is automatically detected when the program is executed. This feature greatly reduces the number of syntax errors that a programmer makes.
  • Since Python is easier to understand and easy to implement, it allows you to develop an application in lesser time as compared to other languages. So, Python ultimately saves time.
  • Since it is an open source programming language, it is desirable to use this language to bring down the cost of a project.

Liquibase package compilation when spec and body in same file

$
0
0

I have a directory where the packages are kept in the below format ,both the package specification and body are kept in the same file ,the file name is abc.pkg

create or replace package abc
 IS 
  /* this package is utility to fetch employee related data*/


 END abc;
/
create or replace package body abc
 IS 

 END abc;
/

My changeset is

<changeSet author="curious" id="PKG_1" logicalFilePath= "package/abc.pkg" runOnChange="true">
    <sqlFile path="package/abc.pkg"
            relativeToChangelogFile="true"
            endDelimiter="/"
            splitStatements="true"
            encoding="utf8"
            />
</changeSet>

The build is successful but the package got error because of / , can anyone help me with this fix , as this must be the most common way of checkin the code into version control systems like GIT and SVN

generating a changelog file where each createTable tag is missing the column tags

$
0
0
*The following took place on a Windows 8.0 PC with MySQL Version 5.7.19 and liquibase 3.6.2 installed.

I have a MySQL database that I created from a liquibase change log file that myself and another developer are using.  My developer made some changes and generate the diff change log (the diff change log contained dropping columns, add tables, adding foreign keys) which I was able to successfully apply to my local MySQL database.  I then went to generate a changelog file off that updated MySQL database and the generateChangLog command completes successfully but when I look at the resulting changelog XML file; all the createTable tags in each changeSet tag were missing the column tags for each table.  Is this normal?  

I even tried dropping the 'databasechangelog' and 'databasechangeloglock' tables in the updated MySQL database before generating a changelog and that failed as well. So I am not sure why liquibase is excluding columns in the createTable tags.

To get around this I manually updated the original changelog XML file making all the changes by hand (dropping columns, adding tables, adding foreign keys) and then creating a new MySQL database (ChangeLogTestTwoDb).  From there I was able to successful generate a changelog file off the ChangeLogTestTwoDb database.

For some reason; after applying the diffChangeLog to an existing database (that was created by a liquibase changelog) I am not able to create a new changelog file from the current state of the database.

Has anyone experienced this before?  Is this expected behavior?  

There wasn't that many changes this time, so making manual edits to an existing changelog XML file was not too bad.  But going forward, I won't be able to do that.  I did find an old ticket (https://liquibase.jira.com/browse/CORE-2142) where the same behavior was found, but that was 4 years ago and I am already using the latest version of liquibase.

Re : generating a changelog file where each createTable tag is missing the column tags

$
0
0
I found this StackOverflow post (https://stackoverflow.com/questions/31684664/liquibase-generatechangelog-missing-columns-statements-in-mariadb) and after reading it it occurred to me that the updates being done involve dropping a column in a table that is used by the one view in the database.  

I tested this out, I create 2 databases from the original change log file and applied the diff changelog file to both databases.  I dropped the view on one database and left it in the other database.  I executed the generateChangeLog commands against both databases. I was able to generate a changelog file as expected from the database where the view was dropped.  With the database with the old version of the view still in place, I generated the changelog file where the createTable tags were missing the column tags.


So the old view referencing recently dropped columns caused the issue.  I am all good now.

Prototype: Automatic lock prolonging and removal of stale locks

$
0
0
Hey there,

automatic lock prolonging (and with this, removal of stale locks) is quite important in microservice environments, especially where developers don't have access to the production database and hence cannot manually remove locks there.

I started working on https://liquibase.jira.com/browse/CORE-2375 and got a prototype working, however only on an older branch.

I haven't been able to get the newer branches to work even without modifications, see https://www.zohodiscussions.com/liquibase#Topic/49382000001952005

My solution seems to work so far, only drawback is that I am creating two JDBC connections now, as the prolonging lock service asynchronously prolongs its lock and hence needs its own transaction (it would otherwise rollback or commit changes the actual updater is doing).

The prolonging lock service is not activated by default, but even when its deactivated, and the standard lock service is used, I'm creating two connections.

I'm not sure if that might cause problems in situations where people use a JDBC pool (with size 1). Maybe we should do another check and only create two connections in case the prolonging lock service got activated.

Looking forward to hear your feedback, or get some review comments (see pull request in the ticket).

Best

Bernhard



LiquiBase changeset is being picked up, but not showing up in databasechangelog or in the database

$
0
0

I created a new changeset to add a new column to an existing table as shown below. I also added the key-property to the hbm.xml file of the usergroup table. It does get picked up according to the logs, and ran successfully, but it does not show up on the databasechangelog table, and the changes are not being reflected in the usergroup table.

<changeSet author="changeTest" id="2018-10-25">
    <preConditions onFail="MARK_RAN">
        <not>
            <columnExists tableName="USERGROUP" columnName="USERROLE"/>
        </not>
    </preConditions>
    <addColumn tableName="USERGROUP">
        <column name="USERROLE" type="VARCHAR(50 ${varcharUnit})" />
    </addColumn>

    <update tableName="USERGROUP">
        <column name="USERROLE" type="VARCHAR(50 ${varcharUnit})" value="user"/>
        <where>USERNAME = 'user1'</where>
    </update>
</changeSet>

The key-property is under a composite-id element, with 2 keys before it.

<key-property name="userRoleInGroup" type="string" >
    <column name="userRole" default="user"/>
</key-property>

Am I missing some update process for this?

How to dump data as SQL from the liquibase Java API?

$
0
0
Platform: Java 8, liquibase, derby 10.11.1.1, liquibase 3.5.3

Here's what I want to do
  1. Write Java code that creates a derby database using a liquibase schema (already in place)
  2. Write Java code to parse CSV data from a legacy system and populate the derby database using plain JDBC
  3. Export the data from the derby database as SQL in a form that can be used to populate a PostgreSQL database initialized by the same liquibase schema
Since I already have Liquibase available, is there a way I could accomplish step 3 using liquibase? 

Or would it be less effort to just do selects in JDBC and walk the resultsets to output them as SQL in print statements?

Thanks!


- Steinar

Re : How to dump data as SQL from the liquibase Java API?

$
0
0
So far I've done this (liquibase is a Liquibase object that already has been used to set up the schema)
  1.         CatalogAndSchema catalogAndSchema = null;
  2.         DiffToChangeLog changeLogWriter = null;
  3.         PrintStream outputStream = new PrintStream("dbdump.sql");
  4.         Class<? extends DatabaseObject>[] snapshotTypes = (Class<? extends DatabaseObject>[])new Class[1];
  5.         snapshotTypes[0] = Data.class;
  6.         liquibase.generateChangeLog(catalogAndSchema, changeLogWriter, outputStream, snapshotTypes);
 but it NPEs in this line in SnapshotGeneratorFactory.createSnapshot() because examples is an array with a single element, containing a null:
  1.             examples[i] = examples[i].customize(database);
I had hoped that when using a null catalogAndSchema I would just use the database I already had set up and populated with data....?

Data dump

$
0
0
I am trying, programmatically in Java, to dump data to a changelog script. I can manage to dump the schema but cannot fathom how to just dump the data, resulting in the very same changelog script file if I had run this from the command line:

liquibase --driver=oracle.jdbc.OracleDriver --logLevel=flag \
      --classpath=/Users/mcutis/Projects/mcurtis/setup/ojdbc7.jar \
      --changeLogFile=db.changelog.data.xml \
      --url="jdbc:oracle:thin:@//localhost:1521/orcl" \
      --username=root \
      --password=root \
      --diffTypes="data" generateChangeLog

I am trying to use liquibase to add members to a role

$
0
0
I am trying to use liquibase to add members to a role, but the SQL is different from SQL Server 2008 to 2012.

SQL Server 2008:

exec sp_addrolemember db_datareader, MYUSER 

SQL Server 2012:

ALTER ROLE db_datawriter ADD MEMBER MYUSER

The following SQL when run via liquibase against SQL Server 2012 works correctly, but when ran against SQL Server 2008 it fails and says:

Incorrect syntax near the keyword 'ADD'

Code:

DECLARE @ver nvarchar(50),
        @intVer int,
        @ver2008 int = 10

SET @ver = CAST(serverproperty('ProductVersion') AS nvarchar)
SET @intVer = CAST(SUBSTRING(@ver, 1, CHARINDEX('.', @ver) - 1) AS int)

IF (@intVer = @ver2008)
    exec sp_addrolemember db_datawriter, MYUSER
ELSE
    ALTER ROLE db_datawriter ADD MEMBER MYUSER;

Can ColumnSnapshotGenerator handle MySQL datetime precision ?

$
0
0
Hi,

With liquibase-3.5.3,  I was trying to retrieve Database schema via liquibase and found out [ColumnSize] returned was always 19 but not the one I saw in MySQL, e.g. 3 for datatime(3). Therefore, could you advise what was going wrong ?
There is a comment I don't if it is helpful for the answer :
According to the description of DatabaseMetaData.getColumns, the content of the "COLUMN_SIZE" column is
         * pretty worthless for datetime/timestamp columns:
         *
         * "For datetime datatypes, this is the length in characters of the String representation
         * (assuming the maximum allowed precision of the fractional seconds component)." 


tmp files after "Run Migrator pulling changelogs from a .WAR file"

$
0
0
Hi all,

I'm using
  1. Run Migrator pulling changelogs from a .WAR file

    java -jar liquibase.jar \
          --driver=oracle.jdbc.OracleDriver \
          --classpath=website.war \
          --changeLogFile=com/example/db.changelog.xml \
          --url=jdbc:oracle:thin:@localhost:1521:oracle \
          --username=scott \
          --password=tiger \
          update

and I can see this logs:
INFO 11/21/18 12:54 PM: liquibase: adding 'jar:file:/tmp/liquibase.tmp7700727174965420685.tmp!/' to classpath
INFO 11/21/18 12:54 PM: liquibase: adding 'jar:file:/tmp/liquibase.tmp6452496865871492534.tmp!/' to classpath
INFO 11/21/18 12:54 PM: liquibase: adding 'jar:file:/tmp/liquibase.tmp4023308518781064752.tmp!/' to classpath
INFO 11/21/18 12:54 PM: liquibase: adding 'jar:file:/tmp/liquibase.tmp8499956868201722408.tmp!/' to classpath
INFO 11/21/18 12:54 PM: liquibase: adding 'jar:file:/tmp/liquibase.tmp7185863074002000436.tmp!/' to classpath
and indeed all these tmp files are created, but not removed (?)

so I'm wondering is there a way to clean all these tmp files? or what other options need to be specified?

liquibase version is 3.5.3

thanks in advance.

Why does runAll="true" validate checksum?

$
0
0
I have a changeset that applies some default data through SQL scripts.

Those scripts can be run always as they handle old and new data alike.
So we change the script to integrate new defaults.

The changeSet is defined with runAlways="true"

The problem is: if the script was changed, Liquibase complains about an invalid checksum.

Why does Liquibase (3.5.5) validate the checksum if the changeSet should be run always anyway?

Is this intended or a bug?

I know I can workaround this by using validCheckSum="1:any" but still, this feels like a bug to me.

Long time of Reading ChangeSet operation

$
0
0
We have a big sized changesets in our liquibase patches(sql format).
Changesets look like this and have about 30000 strings each:

INSERT INTO Table1 (c1, c2, c3, Name, Active, Properties, Domain) VALUES
('SOME_VALUE', 'SOME_VALUE_2', '', 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', true, '25472354725472534hsdfjhsgfjsd25472354725472534hsdfjhsgfjsd25472354725472534hsdfjhsgfjsd25472354725472534hsdfjhsgfjsd', null),
...
('SOME_VALUE', 'SOME_VALUE_2', '', 'adsasdasdasddddddddddddddddddddddddddddddddddddddddddddddddddaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaddddddddddddddddddddddddddddddddddddddddddddddd', true, '25472354725472534hsdfjhsgfjsd25472354725472534hsdfjhsgfjsd', null);

Execution of this changesets extremely slow:

DEBUG 11/28/18 3:15 PM: liquibase: 201811182215_RGS-28022.sql: 201811182215_RGS-28022.sql::8813::RGS-28022: Reading ChangeSet: 201811182215_RGS-28022.sql::8813::RGS-28022
DEBUG 11/28/18 3:54 PM: liquibase: 201811182215_RGS-28022.sql: 201811182215_RGS-28022.sql::8813::RGS-28022: Executing Statement: INSERT INTO Table1 (c1, c2, c3, Name, Active, Properties, Domain) VALUES
DEBUG 11/28/18 3:54 PM: liquibase: 201811182215_RGS-28022.sql: 201811182215_RGS-28022.sql::8813::RGS-28022: Executing EXECUTE database command: INSERT INTO Table1 (c1, c2, c3, Name, Active, Properties, Domain) VALUES

In this time we have 100% CPU load with java, and 1,2 - 1,6 Gb RAM used for this.

Is there ways exists for force liquibase use lover resources and execute such changesets faster? 3.5.3 and 3.5.5 versions affected

ValueComputed creates a null value,it works without the valueComputed bug.

$
0
0
    Trying to create a new row in the DB using a Clobfile.
    I'm using the Oracle database its valueComputed does not return a value but instead returns null. 
    Using the combination of ValueComputed. I have not tested it on MsSQL.

    How do I solve this, so I can use it with valueComputed? the ValueComputed query works in itself without the ClobFile.
    I think I have found a unique combination which seems to not work. 
    Version 3.6.2 of the liquibase-core.

    I am using this query:
    1. <insert tableName="TABLE_NAME">
    2. <column name="ID" type="NUMBER" valueComputed="(select COALESCE(max(id),0) +1 from TABLE_NAME)"/>
    3. <column name="SOME_TYPE" value="SOME_VALUE"/>
    4. <column name="TITLE" value="row with Title"/>
    5. <column name="THISISGREATCONTENT" valueClobFile="data/somefile.txt" type="CLOB" />
    6. <column name="SOMETEMPLATETYPE" value="HTML_EMAIL"/>
    7. </insert>

    Proven Working Query:
    1. <insert tableName="TABLE_NAME">
    2. <column name="ID" type="NUMBER" value="100" />
    3. <column name="SOME_TYPE" value="SOME_VALUE"/>
    4. <column name="TITLE" value="row with Title"/>
    5. <column name="THISISGREATCONTENT" valueClobFile="data/somefile.txt"  type="CLOB" />
    6. <column name="TEMPLATE_TYPE" value="HTML_EMAIL"/>
    7. </insert>

    Exception:
    1. liquibase.exception.MigrationFailedException: Migration failed for change set liquibase/content/file.JL.001.xml::file.JL.001::JL:
    2.      Reason: liquibase.exception.DatabaseException: java.sql.SQLIntegrityConstraintViolationException: ORA-01400: cannot insert NULL into ("SOMEDATABASE"."TABLE_NAME"."ID")

    3. at liquibase.changelog.ChangeSet.execute(ChangeSet.java:637)
    4. at liquibase.changelog.visitor.UpdateVisitor.visit(UpdateVisitor.java:53)
    5. at liquibase.changelog.ChangeLogIterator.run(ChangeLogIterator.java:78)
    6. at liquibase.Liquibase.update(Liquibase.java:202)
    7. at liquibase.Liquibase.update(Liquibase.java:179)
    8. at liquibase.Liquibase.update(Liquibase.java:175)
    9. at com.banktrade.clienttrade.service.database.DatabaseCodeUpdate.applyContentUpdates(DatabaseCodeUpdate.java:244)
    10. at com.banktrade.clienttrade.service.EbeanFactoryBean.applyContentUpdates(EbeanFactoryBean.java:245)
    11. at com.banktrade.clienttrade.setup.service.SetupService.applyContentUpdates(SetupService.java:63)
    12. at com.banktrade.clienttrade.frontend.controller.system.SystemStatusController.applyContentUpdates(SystemStatusController.java:110)
    13. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    14. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    15. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    16. at java.lang.reflect.Method.invoke(Method.java:498)
    17. at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
    18. at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
    19. at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
    20. at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
    21. at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
    22. at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
    23. at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
    24. at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
    25. at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
    26. at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
    27. at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)
    28. at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
    29. at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
    30. at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
    31. at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    32. at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    33. at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    34. at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    35. at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:208)
    36. at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
    37. at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
    38. at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
    39. at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    40. at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    41. at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
    42. at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
    43. at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493)
    44. at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
    45. at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
    46. at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:650)
    47. at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
    48. at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
    49. at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800)
    50. at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
    51. at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806)
    52. at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
    53. at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    54. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    55. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    56. at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    57. at java.lang.Thread.run(Thread.java:748)

    Issues with Hadoop upgrade from hadoop 1.1.2 to hadoop 2.9.0

    $
    0
    0

    I am trying to upgrade Hadoop from 1.1.2 to 2.9.0

    I had executed the command start-dfs.sh -upgrade on hadoop2.9.0 cluster.

    It shows that upgrade is in process and not yet finalized. I have been waiting for long but still showing the same status.

    How to track the status of upgrade process?

    Viewing all 2993 articles
    Browse latest View live


    <script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>