Showing posts with label Java15. Show all posts
Showing posts with label Java15. Show all posts

Wednesday, April 07, 2021

Maven Multi-module Gotchas . . .

Typically, in a maven multi-module project you have a parent-module/main-project and multiple sub-modules with each sub-module producing it's own artifact: jar, war etc.. The main module must be of packaging type pom (<packaging>pom</packaging>) with sub-modules listed. 

For instance, the following is main module/project's pom.xml in project's root directory with it's sub-modules listed:

... <groupId>com.my</groupId> <artifactId>my-service</artifactId> <version>1.0.0-SNAPSHOT</version> <packaging>pom</packaging> ... <modules> <module>my-domain</module> <module>my-core</module> <module>my-service-api</module> </modules>

Gotcha-1

Always use {project.version} for module dependencies.

It is also typical that a sub-module depends on another. For instance core sub-module could depend on domain sub-module. In this case the core sub-module needs to add a dependency on the domain sub-module. When such module dependencies are specified in respective module's pom.xml, never ever specify the dependency version. Instead reference {project.version}.

... <parent> <artifactId>my-service</artifactId> <groupId>com.my</groupId> <version>1.0.0-SNAPSHOT</version> </parent> <artifactId>my-core</artifactId> <packaging>jar</packaging> <name>My Core Name</name> <description>My Core Description</description> <dependencies> <dependency> <groupId>${project.groupId}</groupId> <artifactId>my-domain</artifactId> <version>${project.version}</version> </dependency> ... </dependencies>

Instead of {project.version}, if you use parent module/project version. e.g. in this case, 1.0.0-SNAPSHOT, then you might run into issues when compiling core module. For instance, when a new domain class is added to domain module and is used in core module, you might run into core module compilation issues of not finding new class added.

This is because if you have an artifact repository in which your previous my-domain-1.0.0.SNAPSHOT-*.jar versions are available maven downloads the most recent one into it's cache which results into the newly added domain class not found. This would cause nasty compilation issues giving no clues why.

When you build domain module with mvn clean install, it builds domain module and produces it's new artifact (my-domain-1.0.0.SNAPSHOT.jar) and installs it in local cache. But when you clean up cache and build core module or even main project/module (mvn clean install), the domain module doesn't get built and installed from it's sources, instead it gets downloaded form your artifact repo, which results into older version not having newly added domain class.

If you use {project.version}, and build core module or main project module with mvn clean install, it always builds and installs domain module producing new my-domain-1.0.0.SNAPSHOT.jar in local cache.

Gotcha-2

Multiple application modules sharing same test-case source code.

I recently ran into this situation. The task was to migrate a spring-boot Java micro-service api application from MySQL to PostgreSQL. But let the MySQL api app continue be in place for sometime in parallel along with new api-pg app module added which interfaces with PostgreSQL database. With the addition of new api-pg module, two api artifacts would come out of the maven build: existing api app, and new api-pg app.

First I modularized the existing spring-boot application backed by typical old-fashioned JDBC, DAO layer with inline SQL statements for MySQL database into multiple modules like domain, core, api etc in order to facilitate code reuse between two api applications with minimal specific code in each application. Then added a new api-pg module which is a spring-boot application by itself with minimal Java source code like, main spring-boot application, additional Java configurations, data access layer (DAO Impl), and bootstrap configuration files etc.

This posed a challenge to make the test-cases runnable during maven build for both the applications, keeping the source code in one module. Maven by default runs all test-cases during it's test phase unless otherwise told to skip by passing additional flag like: -DskipTests. As test-cases source code was chosen to be left in the api app module (MySQL based), maven finds it in test-phase and runs. Where as, for the new api-pg module, as test-cases source code is not there in that module, it wouldn't bother to run.

So, it requires bit of a hack getting test-case source code in one api module but making it available in both api modules during maven test-phase of each module. The module that has source code is obviously compiled and ran during it's test phase. It doesn't make sense to somehow make test-case source code available for the other module. The new api-pg module at least needs the compiled classes to be available in Maven's target directory to make them running during this apps test-phase.

That idea of having test-cases source code in one module, getting compiled and run for that module, but make compiled test-case classes available for the other module requires bit of hack. This is is where the following two plugins come in handy to deal with this kind of situation:


The maven-jar-plugin can be leveraged in api module to create a jar file of all the compiled test-case classes from the api module.

The maven-dependency-plugin can be leveraged in api-pg module to unpack the jar file to it's target/test-classes dir so that they get executed as part of it's build. This technique works pretty neat.

By leveraging maven-jar-plugin, the maven build file of api application module: pom.xml  needs the following addition:

... <parent> <artifactId>my-service-api</artifactId> <groupId>com.giri</groupId> <version>1.0.0-SNAPSHOT</version> </parent> ... <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-jar-plugin</artifactId> <version>3.2.0</version> <executions> <execution> <goals> <goal>test-jar</goal> </goals> </execution> </executions> </plugin> ... </plugins>

With the above change, when the api app is built, it produces an additional jar file like: <artifactId>-<version>-tests.jar. For instance, if the artifactId of api module is my-api-service and version is 1.0.0-SNAPSHOT, then the build produces my-api-service-1.0.0-SNAPSHOT-tests.jar file.

Now, by leveraging maven-dependency-plugin, the maven build file of api-pg application module: pom.xml  needs the following addition:
 
... <parent> <artifactId>my-service-api-pg</artifactId> <groupId>com.giri</groupId> <version>1.0.0-SNAPSHOT</version> </parent> ... <dependencies> <dependency> <groupId>com.giri</groupId> <artifactId>my-service-api</artifactId> <version>${project.version}</version> <classifier>tests</classifier> <type>test-jar</type> <scope>test</scope> </dependency> </dependencies> ... <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-dependency-plugin</artifactId> <executions> <execution> <id>unpack</id> <phase>process-test-classes</phase> <goals> <goal>unpack</goal> </goals> <configuration> <artifactItems> <artifactItem> <groupId>com.giri</groupId> <artifactId>my-service-api</artifactId> <version>${project.version}</version> <type>test-jar</type> <outputDirectory>${project.build.directory}/test-classes</outputDirectory> </artifactItem> </artifactItems> </configuration> </execution> </executions> </plugin> ... </plugins> ...

Make sure that you also add test scope dependency for the artifact (tests jar) that gets produced.

That's it. When you run the project build, when maven builds api module, it compiles all test, runs and builds. Then, when it goes for api-pg module build, it unpacks the compiled test classes into it's target and runs all tests for this module as well. This way test-cases source resides in one module, but gets run in both the modules.

NOTE
You may need to make sure that in the main project pom.xml file you list modules in the order that you want them to be built. For instance:

... <modules> <module>my-service-domain</module> <module>my-service-core</module> <module>my-service-api</module> <module>my-service-api-pg</module> </modules> ...

TIP

  • In a multi-module project, whenever there are issues with dependencies, always go to local maven cache for your groupId: ~/.m2/repository/com/my and blow out all sub-directories, files and then run maven build from root project. If  a specific sub-module runs into issues, blow out all that sub-module's module dependencies in maven cache and just build that sub-module and see.

Friday, March 05, 2021

Spring Cloud Config - Gotchas . . .

Spring Cloud Config takes externalization of application property-resources or configuration-files (name-value pair property files: *.properties, and YML files: *.yml) one step further away from the application into an external and central location like a git repo or a file-system.

It comes with a server-side application that accesses property resources from the central location and makes them available for client applications through HTTP resource-based API end-points. This Server application can easily be embeddable in a simple Spring Boot application. Once embedded in a Spring Boot application, this will be your Cloud config server application to be integrated with all client applications. Client applications integrate with the server by setting spring.cloud.config properties like uri, user, password etc.

Environment: Java 15, Spring Boot 2.3.8.RELEASE on macOS Catalina 10.15.7

Gotchas

  • Client applications are identified by Cloud config server by their application names (spring.application.name property value of the respective client application)
  • With git repository hosting property resource files (configuration files like *.properties or *.yml), if you use application.yml or application.properties names for configuration files, these files become common configuration to all client applications that depend on Cloud config server for externalized config files.
  • One config server can serve multiple clients' (multiple applications) externalized configurations. To best leverage this feature, organize externalized config files by the application's name. For instance if there are two Spring Boot client applications (named: my-app-1, and my-app-2) and both are configured to use one Spring Cloud config server for their externalized configurations hosted in a git repo, better to have two separate config files like my-app-1.yml and my-app-2.yml for the respective application configurations in the git repo. One yml file can have configurations for all environments like local, dev, int, cert, prod etc with each environment mapping to respective spring profile and each profile configuration separated by yml directive ---. 
  • The HTTP service of Cloud Config Server exposes end-points in different forms with client-application-name, client-application-profile, and label as part of the end-point forms. Label is nothing but the git branch and is optional with master as implicit value.
  • With git repository set as the host for clients' externalized configuration files, if any specific changes to those files are made and committed in a branch other than master, the cloud config server does not serve that branch unless it is explicitly asked for by specifying the optional label part in the end-point URL. This is the case even if that specific branch is currently checked out.
  • When label (branch name) is not specified, it always uses the implicit label and checks out master branch and responds with the configuration taken from property resources available in the master branch.
  • If explicitly asked for a different branch by  specifying the branch-name as the label in the end-point URL, it checks out that specific branch and servers property resources available in that branch for the given application and it's profile.

    Local Testing

  • For local testing, if your git repo (.git) for clients' external configuration files is in a directory under your home e.g. ~/dev/my-cloud-config, then set spring.cloud.config.server.git.uri to file://Users/<yourMacUserId>/dev/my-cloud-config in the cloud config server spring boot application bootstrap.yml/application.properties for the profile, let's say: int
  • Assuming your cloud config server is running locally on port:8888, the URLs: 1) http://localhost:8888/my-app-1/int 2) http://localhost:8888/my-app-1-int.yml 3) http://localhost:8888/my-app-1-int.properties to check my-app-1 application's int environment/profile configuration results in checking out master branch to serve my-app-1 application's configuration. If you are on a branch other than master, you will notice that you will be switched to master once you make a request to the above URLs to verify the configuration. This is due to the implicit master branch checkout that the cloud config server does under the hood.

  • In your local git repo where client applications' config files are hosted, if your are on a branch other than master and have local changes pending (not committed), the implicit master checkout fails resulting into an exception like:
    org.springframework.cloud.config.server.environment.NoSuchRepositoryException: Cannot clone or checkout repository: file:///Users/<myMacUserId>/dev/my-cloud-config
  • If your configuration files for client applications are on a specific git branch other than master, and if you want to test the cloud config server against that specific branch, then the endpoint /{application}/{profile}[/{label}] URL (e.g. http://localhost:8888/my-app-1/int/my-branch may work.

    Encrypted Content

  • If clients' configuration files have password property like spring.datasource.password and the values are plain text, then the response shows the property with the original value. If the password uses an encrypted value starting with {cipher} followed by encrypted password (e.g. `{cipher}my3ncryp!ed^assword`) then the response doesn't contain spring.datasource.password property or it's value. Instead, you will see a special property "invalid.spring.datasource.password" with a special value "<n/a>"   "invalid.spring.datasource.password": "<n/a>". That means you gotta use a special way to see those encrypted password properties. There is a separate section in the doc explaining this with two special end-points (/encrypt, /decrypt) provided.

TIP

  • In this case, your client application's property resource file changes are on a specific branch, you are unable to get that working, you will be better off changing the Cloud Server Spring Boot application's default implicit master branch to your specific branch by setting the property spring.cloud.config.server.git.default-label to your branch name. This works even if your branch name has / in it (e.g. feature/my-branch).

References