How Liquibase is revolutionizing database change management in modern software development

Liquibase is a database change management tool that enables developers to track, manage, and apply changes to databases as code. It provides a platform-independent way to describe, version, and apply database schema changes. Liquibase is an open-source tool, written in Java, that supports a wide range of database platforms, including Oracle, MySQL, PostgreSQL, and many more.

Introduction

The primary purpose of Liquibase is to make it easy for developers to manage changes to their database schema and data over time, without the need for manual interventions. With Liquibase, developers can define their database schema changes in XML, YAML, or JSON files, which can be versioned and managed like any other codebase. This enables developers to track changes, collaborate on database changes, and automate database deployments with confidence, reducing the risk of database errors and downtime.

For example, let’s say you have an application that uses a MySQL database for data storage. Over time, as your application evolves, you need to make changes to the database schema to add new tables, modify existing ones, or update data. Without a tool like Liquibase, these changes would need to be manually executed, which can be error-prone and time-consuming. With Liquibase, you can define your changes in a structured way and apply them automatically, making it easy to manage database changes at scale.

In summary, Liquibase helps developers to keep their database schema and data under control, and provides a repeatable and automated way to apply changes to databases as code. It is a powerful tool that enables developers to improve the quality, reliability, and speed of their database development and deployment processes.

Getting Started

Before you can start using Liquibase, you need to install and set it up in your project. In this section, we’ll cover how to install and set up Liquibase, as well as how to create a new Liquibase project.

Installation

Liquibase can be installed as a standalone command-line tool, or as a library in your Java project. To install the command-line tool, simply download the latest version from the Liquibase website, and extract it to a directory on your machine. You can then add the Liquibase binary to your system’s PATH environment variable to make it globally accessible.

To use Liquibase as a library in your Java project, you can add the following dependency to your Maven pom.xml or Gradle build.gradle file:

Maven:

<dependency>
  <groupId>org.liquibase</groupId>
  <artifactId>liquibase-core</artifactId>
  <version>4.5.0</version>
</dependency>

Gradle:

implementation 'org.liquibase:liquibase-core:4.5.0'

Creating a new Liquibase project

To create a new Liquibase project, you first need to create a directory to hold your project files. In this directory, you should create a subdirectory called db to hold your database schema files.

Next, you can create a new Liquibase project using the init command, passing in the path to your project directory:

liquibase init /path/to/my/project

This will create a basic Liquibase project structure in your project directory, with the following files:

• liquibase.properties: Configuration file for Liquibase
• db.changelog-master.yaml: Master changelog file that includes all other changelogs
• db.changelog-1.0.yaml: Example changelog file that contains a sample changeset

Here’s an example db.changelog-1.0.yaml file that adds a new person table to the database:

databaseChangeLog:
  - changeSet:
      id: create-person-table
      author: me
      changes:
        - createTable:
            tableName: person
            columns:
              - column:
                  name: id
                  type: bigint
                  autoIncrement: true
                  constraints:
                    primaryKey: true
              - column:
                  name: name
                  type: varchar(255)
              - column:
                  name: email
                  type: varchar(255)
                  constraints:
                    nullable: false

This file defines a single changeset that creates a new person table with three columns: id, name, and email. The id column is an auto-incrementing primary key, while the email column is marked as nullable: false.

With Liquibase set up and your project created, you’re ready to start writing changesets to manage your database schema!

Creating and Applying Changesets

Now that you have a Liquibase project set up, you can start creating and applying changesets to your database. In this section, we’ll cover how to create changesets, version them, and apply them to your database using Liquibase.

Creating Changesets

Changesets in Liquibase are XML, YAML, or JSON files that define changes to be applied to your database schema. A changeset typically contains a single change, such as creating a table, adding a column, or inserting data into a table.

Here’s an example changeset that adds a phone column to the person table created in the previous section:

databaseChangeLog:
  - changeSet:
      id: add-phone-column
      author: me
      changes:
        - addColumn:
            tableName: person
            columns:
              - column:
                  name: phone
                  type: varchar(20)

This changeset defines a new column phone with the data type varchar(20) to be added to the person table.

Versioning Changesets

Changesets in Liquibase are versioned using a combination of an id and an author. The id is a unique identifier for the changeset, and the author is the name of the person or team that created it. Liquibase uses the version number to track which changesets have been applied to the database and which ones need to be applied.

For example, the changeset we defined above has an id of add-phone-column and an author of me. We can version this changeset by adding a version attribute to the changeSet element, like this:

databaseChangeLog:
  - changeSet:
      id: add-phone-column
      author: me
      version: 1.1
      changes:
        - addColumn:
            tableName: person
            columns:
              - column:
                  name: phone
                  type: varchar(20)

Here, we’ve added a version attribute with the value 1.1. This means that this changeset is the second version of the me author’s changesets. You can use any version numbering scheme you like, as long as the numbers are unique and monotonically increasing.

Applying Changesets

To apply changesets to your database, you can use the Liquibase command-line tool or the Liquibase API in your Java code. Here’s an example of how to apply changesets using the command-line tool:

liquibase --changeLogFile=db.changelog-master.yaml 
--url=jdbc:mysql://localhost:3306/mydatabase 
--username=myuser --password=mypassword update

This command tells Liquibase to apply all changesets defined in the db.changelog-master.yaml file to the mydatabase MySQL database using the myuser username and mypassword password.

You can also use the Liquibase API in your Java code to apply changesets. Here’s an example of how to do this:

import liquibase.Liquibase;
import liquibase.database.DatabaseFactory;
import liquibase.database.jvm.JdbcConnection;
import liquibase.exception.LiquibaseException;
import liquibase.resource.ClassLoaderResourceAccessor;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;

public class MyApp {

  public static void main(String[] args) throws SQLException, LiquibaseException {
    String url = "jdbc:mysql://localhost:3306/mydatabase";
    String username = "myuser";
    String password = "mypassword";

    Connection connection = DriverManager.getConnection(url, username, password);
    JdbcConnection jdbcConnection = new JdbcConnection(connection);
   
    Liquibase liquibase = new Liquibase("db.changelog-master.yaml", new ClassLoaderResourceAccessor(), jdbcConnection);
    liquibase.update("");
  }
}

This code connects to the mydatabase MySQL database using the myuser username and mypassword password, creates a JdbcConnection object, and passes it to a new Liquibase object along with the name of the changeset file (db.changelog-master.yaml). The liquibase.update("") method applies all changesets defined in the file.

Understanding Changesets and Writing Basic Changesets

Liquibase organizes database changes into changesets. A changeset is a single unit of change to your database schema. Each changeset has a unique ID and author, and may have a description. Changesets can be defined in XML, YAML, or JSON format.

Here’s an example of a basic XML changeset that adds a new table to a database:

<changeSet author="john" id="1">
  <createTable tableName="customer">
    <column name="id" type="int">
      <constraints primaryKey="true" nullable="false" />
    </column>
    <column name="first_name" type="varchar(50)">
      <constraints nullable="false" />
    </column>
    <column name="last_name" type="varchar(50)">
      <constraints nullable="false" />
    </column>
  </createTable>
</changeSet>

This changeset is authored by “john” and has an ID of “1”. It creates a new table called “customer” with three columns: “id”, “first_name”, and “last_name”. The “id” column is defined as an integer with a primary key constraint, and the “first_name” and “last_name” columns are defined as strings with a not-null constraint.

Liquibase supports many types of changes, including creating and modifying tables, adding and dropping columns, creating and modifying indexes, and executing SQL statements. Here are some examples of Liquibase-supported database refactoring operations:

Creating a table:

<changeSet author="jane" id="2">
  <createTable tableName="order">
    <column name="id" type="int">
      <constraints primaryKey="true" nullable="false" />
    </column>
    <column name="customer_id" type="int">
      <constraints nullable="false" />
    </column>
    <column name="order_date" type="datetime">
      <constraints nullable="false" />
    </column>
  </createTable>
</changeSet>

This changeset adds a new table called “order” with three columns: “id”, “customer_id”, and “order_date”.

Adding a column to an existing table:

<changeSet author="john" id="3">
  <addColumn tableName="customer">
    <column name="email" type="varchar(255)">
      <constraints nullable="false" />
    </column>
  </addColumn>
</changeSet>

This changeset adds a new column called “email” to the “customer” table.

Modifying an existing column:

<changeSet author="jane" id="4">
  <modifyDataType tableName="customer" columnName="first_name" newDataType="varchar(100)" />
</changeSet>

This changeset modifies the “first_name” column in the “customer” table to have a new data type of “varchar(100)”.

Dropping a table:

<changeSet author="john" id="5">
  <dropTable tableName="order" />
</changeSet>

This changeset drops the “order” table from the database.

Liquibase’s extensive support for database refactoring operations makes it easy to manage database schema changes as your application evolves.

In the next section, we’ll explore how to use Liquibase with Maven and Gradle to integrate database changes into your build process.

Using Liquibase with Maven and Gradle

Liquibase can be integrated with Maven and Gradle to make managing database changes even easier. Here’s how to set up Liquibase with each of these build tools:

1. Using Liquibase with Maven

To use Liquibase with Maven, you’ll need to add the Liquibase plugin to your pom.xml file:

<plugin>
  <groupId>org.liquibase</groupId>
  <artifactId>liquibase-maven-plugin</artifactId>
  <version>4.4.3</version>
  <configuration>
    <propertyFile>liquibase.properties</propertyFile>
  </configuration>
  <dependencies>
    <dependency>
      <groupId>mysql</groupId>
      <artifactId>mysql-connector-java</artifactId>
      <version>8.0.27</version>
    </dependency>
  </dependencies>
</plugin>

This plugin configuration specifies the version of the Liquibase plugin to use (4.4.3), and the location of the Liquibase properties file (liquibase.properties). The dependencies section includes the MySQL connector JAR file.

With this plugin set up, you can now use Liquibase commands directly in Maven:

mvn liquibase:update

This command tells Maven to apply all changesets to your database.

2. Using Liquibase with Gradle

To use Liquibase with Gradle, you’ll need to add the Liquibase plugin to your build.gradle file:

plugins {
  id "org.liquibase.gradle" version "2.0.3"
}

liquibase {
  activities {
    main {
      changeLogFile 'db.changelog-master.yaml'
      url 'jdbc:mysql://localhost:3306/mydatabase'
      username 'myuser'
      password 'mypassword'
    }
  }
}

This plugin configuration specifies the version of the Liquibase plugin to use (2.0.3), and the database connection information. With this plugin set up, you can now use Liquibase commands directly in Gradle:

gradle update

This command tells Gradle to apply all changesets to your database.

Tracking Changes, Rolling Back Changes, and Applying Changes to Multiple Environments

One of the key benefits of using Liquibase is the ability to track changes to your database schema over time. Liquibase maintains a change log that records every change made to your database, including the author, timestamp, and description of each change.

Here’s an example of a change log in YAML format:

databaseChangeLog:
  - changeSet:
      id: 1
      author: john
      changes:
        - createTable:
            tableName: customer
            columns:
              - column:
                  name: id
                  type: int
                  constraints:
                    primaryKey: true
                    nullable: false
              - column:
                  name: first_name
                  type: varchar(50)
                  constraints:
                    nullable: false
              - column:
                  name: last_name
                  type: varchar(50)
                  constraints:
                    nullable: false
  - changeSet:
      id: 2
      author: jane
      changes:
        - createTable:
            tableName: order
            columns:
              - column:
                  name: id
                  type: int
                  constraints:
                    primaryKey: true
                    nullable: false
              - column:
                  name: customer_id
                  type: int
                  constraints:
                    nullable: false
              - column:
                  name: order_date
                  type: datetime
                  constraints:
                    nullable: false

This change log has two changesets, one that creates the “customer” table and one that creates the “order” table.

Liquibase also provides a mechanism for rolling back changes. You can roll back a specific changeset, or roll back all changes up to a certain point in time. Here’s an example of rolling back the “order” table creation changeset:

<changeSet author="jane" id="2">
  <rollback>
    <dropTable tableName="order" />
  </rollback>
</changeSet>

This changeset rolls back the “order” table creation changeset by dropping the “order” table.

Finally, Liquibase makes it easy to apply database schema changes to multiple environments, such as development, staging, and production. You can create separate change logs for each environment, or use the same change log and apply only certain changes to each environment. Here’s an example of a change log that includes separate change sets for development and production environments:

<databaseChangeLog>
  <include file="changelog-dev.xml" context="dev" />
  <include file="changelog-prod.xml" context="prod" />
</databaseChangeLog>

In this example, the change log includes two separate change logs, one for the “dev” context and one for the “prod” context. When Liquibase is run in the “dev” context, it will only apply changes from the “changelog-dev.xml” file, and when run in the “prod” context, it will only apply changes from the “changelog-prod.xml” file.

By using Liquibase’s change log and rollback functionality, you can manage database schema changes with confidence and ease.

Advanced Features

Liquibase provides several advanced features that allow you to manage complex database schema changes and work effectively in a team environment.

Managing Preconditions

Preconditions are conditions that must be met before a change set can be executed. For example, you may need to ensure that a certain table exists before creating a new column in that table. Liquibase provides several built-in preconditions, such as “tableExists” and “columnExists”, and allows you to define custom preconditions.

Here’s an example of a change set that uses a precondition to ensure that the “customer” table exists before creating a new column:

<changeSet author="john" id="3">
  <preConditions>
    <tableExists tableName="customer" />
  </preConditions>
  <addColumn tableName="customer">
    <column name="address" type="varchar(100)" />
  </addColumn>
</changeSet>

This change set will only be executed if the “customer” table already exists.

Using Liquibase with Spring Boot

Liquibase can be easily integrated with Spring Boot, a popular Java web framework. Spring Boot provides an easy way to configure and run Liquibase during application startup, and includes support for both XML and YAML change logs.

Here’s an example of configuring Liquibase with Spring Boot in the application.properties file:

spring.liquibase.change-log=classpath:db/changelog.xml
spring.liquibase.contexts=dev

This configuration specifies the change log file location and the execution context (in this case, “dev”).

Using Liquibase in a Team Environment

When working in a team environment, it’s important to manage database schema changes collaboratively and avoid conflicts. Liquibase provides several features that support team collaboration, such as:

• The ability to split a large change log into smaller, more manageable files
• Support for change log branching and merging
• The ability to generate SQL scripts from a change log for review and approval before execution

By using Liquibase’s team collaboration features, you can ensure that changes are applied consistently and avoid conflicts that can cause downtime or data loss.

In summary, Liquibase provides advanced features that allow you to manage complex database schema changes, work effectively in a team environment, and ensure that changes are applied consistently and safely.

Best Practices

In order to get the most out of Liquibase, it’s important to follow best practices for organizing changesets, using naming conventions, and version control.

1. Organizing Changesets

Organizing changesets into logical groups can make it easier to understand and manage your database schema changes. For example, you might organize your changesets by feature, module, or release. Liquibase supports the use of “contexts” to group related changesets together. You can also use “labels” to group changesets across multiple contexts.

2. Naming Conventions

Using consistent and descriptive names for your changesets can make it easier to understand and manage them. It’s a good practice to include the author, date, and a brief description of the change in the name of the changeset. For example, “john-20220501-create-customer-table.xml”.

You can also use naming conventions for your SQL objects, such as tables, columns, and indexes. A common convention is to use lowercase letters and underscores to separate words, such as “customer_table” or “order_item_table”.

3. Version Control with Liquibase

Liquibase is designed to work with version control systems, such as Git or SVN, to manage changes to your database schema over time. When using Liquibase with version control, it’s important to:

• Keep your change logs and SQL scripts in the version control.
• Use a consistent and meaningful commit message for each change
• Use branches and merge strategies to manage parallel development and avoid conflicts
• Use Liquibase’s “diff” functionality to generate change logs from an existing database schema

By following these best practices, you can ensure that your database schema changes are well-organized, well-documented, and well-managed over time.

In summary, Liquibase provides several best practices for organizing changesets, using naming conventions, and version control, which can make it easier to manage your database schema changes and work collaboratively with your team.

Conclusion

Liquibase is a powerful tool for managing database schema changes in Java applications. With Liquibase, you can define changes to your database schema as XML, YAML, or JSON files, version them, and apply them to your database using a variety of tools and APIs. By using Liquibase with Maven or Gradle, you can easily integrate database changes into your build process and keep your database schema in sync with your code.

---

My articles on medium