Wednesday, April 07, 2021

Maven Multi-module Gotchas . . .

Typically, in a maven multi-module project you have a parent-module/main-project and multiple sub-modules with each sub-module producing it's own artifact: jar, war etc.. The main module must be of packaging type pom (<packaging>pom</packaging>) with sub-modules listed. 

For instance, the following is main module/project's pom.xml in project's root directory with it's sub-modules listed:

... <groupId>com.my</groupId> <artifactId>my-service</artifactId> <version>1.0.0-SNAPSHOT</version> <packaging>pom</packaging> ... <modules> <module>my-domain</module> <module>my-core</module> <module>my-service-api</module> </modules>

Gotcha-1

Always use {project.version} for module dependencies.

It is also typical that a sub-module depends on another. For instance core sub-module could depend on domain sub-module. In this case the core sub-module needs to add a dependency on the domain sub-module. When such module dependencies are specified in respective module's pom.xml, never ever specify the dependency version. Instead reference {project.version}.

... <parent> <artifactId>my-service</artifactId> <groupId>com.my</groupId> <version>1.0.0-SNAPSHOT</version> </parent> <artifactId>my-core</artifactId> <packaging>jar</packaging> <name>My Core Name</name> <description>My Core Description</description> <dependencies> <dependency> <groupId>${project.groupId}</groupId> <artifactId>my-domain</artifactId> <version>${project.version}</version> </dependency> ... </dependencies>

Instead of {project.version}, if you use parent module/project version. e.g. in this case, 1.0.0-SNAPSHOT, then you might run into issues when compiling core module. For instance, when a new domain class is added to domain module and is used in core module, you might run into core module compilation issues of not finding new class added.

This is because if you have an artifact repository in which your previous my-domain-1.0.0.SNAPSHOT-*.jar versions are available maven downloads the most recent one into it's cache which results into the newly added domain class not found. This would cause nasty compilation issues giving no clues why.

When you build domain module with mvn clean install, it builds domain module and produces it's new artifact (my-domain-1.0.0.SNAPSHOT.jar) and installs it in local cache. But when you clean up cache and build core module or even main project/module (mvn clean install), the domain module doesn't get built and installed from it's sources, instead it gets downloaded form your artifact repo, which results into older version not having newly added domain class.

If you use {project.version}, and build core module or main project module with mvn clean install, it always builds and installs domain module producing new my-domain-1.0.0.SNAPSHOT.jar in local cache.

Gotcha-2

Multiple application modules sharing same test-case source code.

I recently ran into this situation. The task was to migrate a spring-boot Java micro-service api application from MySQL to PostgreSQL. But let the MySQL api app continue be in place for sometime in parallel along with new api-pg app module added which interfaces with PostgreSQL database. With the addition of new api-pg module, two api artifacts would come out of the maven build: existing api app, and new api-pg app.

First I modularized the existing spring-boot application backed by typical old-fashioned JDBC, DAO layer with inline SQL statements for MySQL database into multiple modules like domain, core, api etc in order to facilitate code reuse between two api applications with minimal specific code in each application. Then added a new api-pg module which is a spring-boot application by itself with minimal Java source code like, main spring-boot application, additional Java configurations, data access layer (DAO Impl), and bootstrap configuration files etc.

This posed a challenge to make the test-cases runnable during maven build for both the applications, keeping the source code in one module. Maven by default runs all test-cases during it's test phase unless otherwise told to skip by passing additional flag like: -DskipTests. As test-cases source code was chosen to be left in the api app module (MySQL based), maven finds it in test-phase and runs. Where as, for the new api-pg module, as test-cases source code is not there in that module, it wouldn't bother to run.

So, it requires bit of a hack getting test-case source code in one api module but making it available in both api modules during maven test-phase of each module. The module that has source code is obviously compiled and ran during it's test phase. It doesn't make sense to somehow make test-case source code available for the other module. The new api-pg module at least needs the compiled classes to be available in Maven's target directory to make them running during this apps test-phase.

That idea of having test-cases source code in one module, getting compiled and run for that module, but make compiled test-case classes available for the other module requires bit of hack. This is is where the following two plugins come in handy to deal with this kind of situation:


The maven-jar-plugin can be leveraged in api module to create a jar file of all the compiled test-case classes from the api module.

The maven-dependency-plugin can be leveraged in api-pg module to unpack the jar file to it's target/test-classes dir so that they get executed as part of it's build. This technique works pretty neat.

By leveraging maven-jar-plugin, the maven build file of api application module: pom.xml  needs the following addition:

... <parent> <artifactId>my-service-api</artifactId> <groupId>com.giri</groupId> <version>1.0.0-SNAPSHOT</version> </parent> ... <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-jar-plugin</artifactId> <version>3.2.0</version> <executions> <execution> <goals> <goal>test-jar</goal> </goals> </execution> </executions> </plugin> ... </plugins>

With the above change, when the api app is built, it produces an additional jar file like: <artifactId>-<version>-tests.jar. For instance, if the artifactId of api module is my-api-service and version is 1.0.0-SNAPSHOT, then the build produces my-api-service-1.0.0-SNAPSHOT-tests.jar file.

Now, by leveraging maven-dependency-plugin, the maven build file of api-pg application module: pom.xml  needs the following addition:
 
... <parent> <artifactId>my-service-api-pg</artifactId> <groupId>com.giri</groupId> <version>1.0.0-SNAPSHOT</version> </parent> ... <dependencies> <dependency> <groupId>com.giri</groupId> <artifactId>my-service-api</artifactId> <version>${project.version}</version> <classifier>tests</classifier> <type>test-jar</type> <scope>test</scope> </dependency> </dependencies> ... <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-dependency-plugin</artifactId> <executions> <execution> <id>unpack</id> <phase>process-test-classes</phase> <goals> <goal>unpack</goal> </goals> <configuration> <artifactItems> <artifactItem> <groupId>com.giri</groupId> <artifactId>my-service-api</artifactId> <version>${project.version}</version> <type>test-jar</type> <outputDirectory>${project.build.directory}/test-classes</outputDirectory> </artifactItem> </artifactItems> </configuration> </execution> </executions> </plugin> ... </plugins> ...

Make sure that you also add test scope dependency for the artifact (tests jar) that gets produced.

That's it. When you run the project build, when maven builds api module, it compiles all test, runs and builds. Then, when it goes for api-pg module build, it unpacks the compiled test classes into it's target and runs all tests for this module as well. This way test-cases source resides in one module, but gets run in both the modules.

NOTE
You may need to make sure that in the main project pom.xml file you list modules in the order that you want them to be built. For instance:

... <modules> <module>my-service-domain</module> <module>my-service-core</module> <module>my-service-api</module> <module>my-service-api-pg</module> </modules> ...

TIP

  • In a multi-module project, whenever there are issues with dependencies, always go to local maven cache for your groupId: ~/.m2/repository/com/my and blow out all sub-directories, files and then run maven build from root project. If  a specific sub-module runs into issues, blow out all that sub-module's module dependencies in maven cache and just build that sub-module and see.