Showing posts with label Tips. Show all posts
Showing posts with label Tips. Show all posts

Friday, September 20, 2024

Java - Gotcha - Sealed interface and mocking in unit tests . . .

Seal(noun)
dictionary meaning - a device or substance that is used to join two things together so as to prevent them from coming apart or to prevent anything from passing between them. 

Prevent anything from passing between them. That's exactly what sometimes you want to put in place. When you have an interface and want to restrict other interfaces to extend or classes to implement to have a control on, you ned to seal your interface by specifying all those are permitted to extend or implement.

Java sealed interfaces is a feature introduced in Java 15 as a preview feature and became a standard feature in Java 17. Sealed interface restricts which classes or interfaces can implement or extend it. Classes that implement a sealed interface must be declared as final, sealed, or non-sealed. This provides more control over the inheritance hierarchy and helps to enforce certain design constraints.
 
To declare a sealed interface, use the sealed keyword followed by the permits clause, which lists the permitted subtypes.

E.g.
public sealed interface Shape permits Circle, Rectangle, Triangle { double area(); }

Each permitted subtype must be declared as one of the following:
  • Final: Cannot be extended further.
  • Sealed: Can specify its own permitted subtypes.
  • Non-Sealed: Removes the sealing restriction, allowing any class to extend it.
// Final class public final class Circle implements Shape { ... } // Sealed class public sealed class Rectangle implements Shape permits Square { ... } // Non-sealed class, additional permitted sub-type public final Square extends Rectangle { ... }

Benefits of Sealed Interfaces

Enhanced Control: Provide more control over the inheritance hierarchy, ensuring that only specific classes can implement the interface.
Improved Maintainability: By restricting the set of permitted subtypes, you can make your codebase easier to understand and maintain.
Better Exhaustiveness Checking: Sealed interfaces improve exhaustiveness checking in switch statements, especially when used with pattern matching (introduced in later Java versions).

The exhaustive checking in switch statement itself is very useful feature to have that makes your code not to miss handling a case of interface type in switch which otherwise is prone to bugs. The compiler would not let your code compile until all possible cases are handled in a switch statement making your code robust.

Shape aShape; ... switch(aShape) { case Circle circle -> circle.radius(); case Rectangle rectangle -> // do something // handle all remaining cases or provide default case, otherwise you code fails compilation }

Gotcha - Mockito, mocking sealed interface

Mocking is common in unit testing. If you are writing unit test for an object A that depends on object B, you will not be interested in B and can simply mock it's behavior. If Mockito is your mocking framework, and B happens to be a sealed interface with some permitted implementations, then you will not be able to mock like usually you do as follows:

class ATest { ... @Mock private B objB; ... }

Your test fails with the following error when it is run:
org.mockito.exceptions.base.MockitoException: Mockito cannot mock this class: interface B. If you're not sure why you're getting this error, please open an issue on GitHub. Java : 22 JVM vendor name : Amazon.com Inc. JVM vendor version : 22.0.1+8-FR JVM name : OpenJDK 64-Bit Server VM JVM version : 22.0.1+8-FR JVM info : mixed mode, sharing OS name : Mac OS X OS version : 13.6.6 You are seeing this disclaimer because Mockito is configured to create inlined mocks. You can learn about inline mocks and their limitations under item #39 of the Mockito class javadoc. Underlying exception : org.mockito.exceptions.base.MockitoException: Unsupported settings with this type 'B'

Solution
Change mock to a specific implementation of the interface.
private final B objB = mock(BImpl.class); // sealed interface, specify specific implementation class to be mocked

Sunday, September 15, 2024

Spring Code TIP-1: Get code coverage for the main method in Spring Boot application . . .

Setting up a new Spring boot project is made trivial by the Spring Initializer. IDEs like IntelliJ has integrated support for this as well. The application generated by this initializer contains three files under src.
    1) The application file (e.g. DemoAppliction.java) is Spring Boot main class that bootstraps the project.
    2) A properties file (application.properties) is the application configuration file.
    3) A test-case file (DemoApplictionTests.java). 

The main application file looks like:
package com.example.demo; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; @SpringBootApplication public class DemoApplication { public static void main(String[] args) { SpringApplication.run(DemoApplication.class, args); } }
The SpringApplication.run(DemoApplication.class, args) performs bootstrapping, creates application context, and runs the application.

The configuration file looks like:
spring.application.name=demo

The test-case looks like:
package com.example.demo; import org.junit.jupiter.api.Test; import org.springframework.boot.test.context.SpringBootTest; @SpringBootTest class DemoApplicationTests { @Test void contextLoads() { } }
This is in fact a simple yet very useful test class and I would keep it around. The is an integration test-case that ensures that the application context loads successfully. If there are any issues this test fails. So, this can be treated like a integration smoke-test for your application. I would rename that test method name to smokeTest_contextLoads().

Spring Code TIP - Code coverage

However, once you progress on with the application and have code coverage check applied using build system like maven and its plugins Surefire, Failsafe and JaCoCo plugins, the main method is left uncovered by this test-case. Little strange!

Having a main method in the application supported by integrated test-case, one expects that the main method is invoked by the test-case. The annotation @SpringBootTest by default has this turned off. Hence @SpringBootTest calls SpringApplication.run(DemoApplication.class, args) and not the main method. So, in order to get the main method called instead when the test-case is run, we need to explicitly set a property. Once this is set, the test-case invokes the main method and the main method gets the code coverage.

The property is shown below to get the test coverage for the main method.
@SpringBootTest(useMainMethod = SpringBootTest.UseMainMethod.ALWAYS)

Not just for the code coverage, you might also want to use the useMainMethod property in scenarios where your application's main method performs additional setup or configuration that is necessary for your tests as well.

Summary

The useMainMethod property of the @SpringBootTest annotation allows you to control whether the main method of your application is invoked to start the Spring Boot application context during testing. By default, useMainMethod is set to UseMainMethod.NEVER, but you can set it to UseMainMethod.ALWAYS or UseMainMethod.WHEN_AVAILABLE to ensure that the main method is called during testing.

Monday, August 28, 2023

IntelliJ - the Community Edition rescued the broken Ultimate Edition . . . ;)

It sound funny to say that - IntelliJ Community Edition (free edition) rescued Ultimate Edition (paid edition). But that's what really put me back on Ultimate Edition after many months of switching to Community Edition for my day-to-day development.

Environment: IntelliJ Community Edition 2023.2.1IntelliJ Ultimate Edition 2023.2.1, maven 3.9.3 on macOS Catalina 10.15.7

The Issue

I stopped using IntelliJ Ultimate Edition as for some weird reason it got stuck with maven dependencies broken paths issue. Specifically, SLF4J and javax libraries were shown in maven dependency broken paths in red. It happened few months ago. One day one of my maven projects that uses Lombok and had classes annotated with @Slf4J suddenly couldn't recognize the log (logger object) statements. When I checked project module dependencies there were broken maven dependency paths.

I tried all recommended and possible ways to recover from this issue like: Invalidate Caches and restart the IDE(A), blowing away specific libraries maven cache ~/.m2/libraries/.../*.* and letting it build again, blowing away entire maven local cache ~/.m2/*.* and letting it build again, downloading dependency jars and explicitly setting dependency paths in the project/module settings, reinstalling the Ultimate Edition, even reinstalling different version of the Ultimate Edition etc. Nothing worked. I spent lot of time few times since then and couldn't get it back to working. Finally gave up and moved to Community Edition. Same projects that were having broken dependency issues in Ultimate Edition, when opened in Community edition, had no issues with those dependencies, same maven local cache paths are used by both. The Ultimate Edition complains, the Community Edition doesn't. Alas!

The Fix

Here is what I did to fix it.

Started IntelliJ Ultimate Edition. At the startup there is a Customize link, click that and click Import Settings... link as shown below:

Selected the Community Edition settings directory (~/Library/Application Support/JetBrains/IdeaIC2023.2) of Community Edition in which my projects were fine with dependencies. It prompted to take a backup of current settings. I did that and imported ommunity Edition settings. It opened projects and the issue of broken dependency paths was gone.

It seemed like, there was some broken dependency path setting saved in the Ultimate Edition settings that was stuck and not getting fixed by any means.

Some Internals

IntelliJ saves all it's settings under the specific version's area. On Mac, by default, this area is under ~/Library/Application Support/JetBrains dir. The community edition directories start with IdeaIC<version> (e.g IdeaIC2023.2) and ultimate edition directories start with IntelliJIdea<version> (e.g. IntelliJIdea2023.2). The options sub-directory is where plugins settings get saved. 

Plugin Settings

Plugin settings are pat of IntelliJ settings and get stored in xml files under options sub-directory of specific IntelliJ version's settings directory. For instance, awesome editor plugin is a simple and pretty neat plugin which lets add image backgrounds. I did setup plugin awesome editor to display different kinds of images for different types of files. To get all the settings from one IntelliJ version to another, just copy the plugin settings file (in this case it is: awesome-editor-3.xml). 

Here is an example to copy plugin settings set in Ultimate edition to Community Edition.
NOTE: Once copied restart IntelliJ Community Edition.

$ cd ~/Library/"Application Support"/JetBrains $ ls -al |grep IdeaIC drwxr-xr-x 17 pottepalemg 163264107 544 Jul 26 2022 IdeaIC2022.1 drwxr-xr-x 20 pottepalemg 163264107 640 Nov 30 2022 IdeaIC2022.2 drwxr-xr-x 20 pottepalemg 163264107 640 Mar 30 15:23 IdeaIC2022.3 drwxr-xr-x 20 pottepalemg 163264107 640 Jul 7 11:31 IdeaIC2023.1 drwxr-xr-x 20 pottepalemg 163264107 640 Aug 28 10:13 IdeaIC2023.2 $ ls -al |grep IntelliJIdea drwxr-xr-x 22 pottepalemg 163264107 704 Aug 24 15:43 IntelliJIdea2020.1 drwxr-xr-x 19 pottepalemg 163264107 608 Nov 19 2020 IntelliJIdea2020.2 drwxr-xr-x 21 pottepalemg 163264107 672 Mar 2 2022 IntelliJIdea2020.3 drwxr-xr-x 20 pottepalemg 163264107 640 Jul 9 2021 IntelliJIdea2021.1 drwxr-xr-x 24 pottepalemg 163264107 768 Feb 9 2022 IntelliJIdea2021.2 drwxr-xr-x 25 pottepalemg 163264107 800 Apr 12 2022 IntelliJIdea2021.3 drwxr-xr-x 26 pottepalemg 163264107 832 Sep 23 2022 IntelliJIdea2022.1 drwxr-xr-x 25 pottepalemg 163264107 800 Nov 30 2022 IntelliJIdea2022.2 drwxr-xr-x 25 pottepalemg 163264107 800 Jun 28 13:26 IntelliJIdea2022.3 drwxr-xr-x 24 pottepalemg 163264107 768 Aug 16 15:40 IntelliJIdea2023.1 drwxr-xr-x 22 pottepalemg 163264107 704 Aug 28 10:36 IntelliJIdea2023.2 drwxr-xr-x 11 pottepalemg 163264107 352 Aug 25 15:35 IntelliJIdea2023.2-backup $ find . -name awesome* ./IntelliJIdea2023.2/options/awesome-editor-3.xml $ ls -ltr ./IdeaIC2023.2/options | grep awesome* $ cp ./IntelliJIdea2023.2/options/awesome-editor-3.xml ./IdeaIC2023.2/options

TIPS

Getting back lost database connection settings into Ultimate Edition

IntelliJ Ultimate Edition comes bundled with Database plugin that supports all features that are available in DataGrip (an SQL IDE which is a product of JetBrains). I had database connections set to connect to various PostgreSQL databases (local, int, cert, prod etc.) which I lost by importing Community Edition settings. But I had a settings backup prompted and done for the Ultimate Edition settings that my broken Ultimate Edition was setup with when I imported Community Edition Settings. The backup directory is also listed in the above list of Ultimate Editions setting directories. To get those settings back onto my new settings imported from Community Edition, I had to repeat the Customize > Import Settings... step two more times. First time pointing it to the backup directory and selecting and copying all database connections settings to the clipboard. Second time pointing it to the Community Edition Directory and pasting those connection settings from the clipboard. This was the only way I could get those database connections copied. I got all connection settings except passwords fro every connection. I had to set password for every single connection individually. This was not possible by simply copying xml files like I did for awesome editor plugin.

Summary

No software application or tool is bug-free. Applications do crash, tools do get corrupted. There is no one solution that works or fixes a similar issue for everybody. Some stupid, nasty, unknown, not very well documented internals of tools do take up lot of time to discover a fix that works for your situation and may help some others who get into similar situation.

Hope this blog post on my discovery saves someone's time sometime when that someone bumps into it.

Friday, August 11, 2023

Spring boot - your own banner, actuator, version details etc . . .

Art is good for eyes. Spring boot out of the box comes with a nice text banner of it's name : Spring Boot and displays the version right below the banner. Out of the box, the banner mode is set on and it shows up when the application gets started. There are several articles available on customizing this banner. Also, spring boot documentation has a brief section on thus as well (link in the resources).

Environment: Java 20, Spring Boot 2.7.14, maven 3.9.3 on macOS Catalina 10.15.7

It's always good to customize the banner and see your application name whenever it comes up. Of course, you can display more good-to-have version details like: Spring Boot version, Java version, Your application version, and other available properties along with the banner.

Following is the quick list of steps to have a custom banner in your application.
  • Generate a text banner for your application name. There are several sites for doing this. The one such is: https://devops.datenkollektiv.de/banner.txt/index.html. Generate a text banner and download the text file. I usually go with the standard banner font.
  • Edit the file and add the following properties for versions at the bottom:
____ _ ____ ___ __ | __ ) ___ ___ | |_ / ___|_ __ __ _ __ _| \ \ / / __ ___ | _ \ / _ \ / _ \| __| | | _| '__/ _` |/ _` | |\ \ / / '_ ` _ \ | |_) | (_) | (_) | |_ | |_| | | | (_| | (_| | | \ V /| | | | | | |____/ \___/ \___/ \__| \____|_| \__,_|\__,_|_| \_/ |_| |_| |_| :: Spring Boot :: ${spring-boot.version} :: Running on Java :: ${java.version} :: Application :: ${project.version}
  • Place the text file with file-name: banner.txt under src/java/resources folder.
  • In your maven build file (pom.xml), make sure that you turn on maven filtering and add the resource directory for the extra version properties added to be resolved during the build.
  • Also, make sure you have maven-resources-plugin configured to filter resources.
<build> <resources> <resource> <filtering>true</filtering> <directory>${project.basedir}/src/main/resources/</directory> </resource> </resources> <testResources> <testResource> <filtering>true</filtering> <directory>${project.basedir}/src/test/resources/</directory> </testResource> </testResources> ... <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-resources-plugin</artifactId> <version>3.3.1</version> <executions> <execution> <id>filter-resources</id> <phase>process-resources</phase> <goals> <goal>copy-resources</goal> </goals> <configuration> <resources> <!-- # Filtered Resources --> <resource> <directory>${project.basedir}/src/main/resources/</directory> <filtering>true</filtering> </resource> </resources> <outputDirectory>${project.build.directory}</outputDirectory> </configuration> </execution> </executions> </plugin> ... </build>

Gotcha-1

When you run maven build goal(s) that also runs test-cases like for e.g. ./mvnw clean install, the banner shows up with all properties resolved for every integration test-case. However, for the custom properties used in the banner to be filtered and shown you need to add the same custom property in application.yml if you happen to separate out test configurations under test/resources. Also, make sure that you have enabled <testResources> filtering as well as shown above.

Gotcha-2 (Spring Boot 3.x)

The above ${project.version} doesn't work in Spring boot 3.x. In this case a custom application version property (e.g. app.version) can be defined in application.yml or application.properties and that can be used in the banner.txt file.
E.g. application.yml
spring: application: name: @project.name@ # custom property for banner app: version: @project.version@

____ _ ____ ___ __ | __ ) ___ ___ | |_ / ___|_ __ __ _ __ _| \ \ / / __ ___ | _ \ / _ \ / _ \| __| | | _| '__/ _` |/ _` | |\ \ / / '_ ` _ \ | |_) | (_) | (_) | |_ | |_| | | | (_| | (_| | | \ V /| | | | | | |____/ \___/ \___/ \__| \____|_| \__,_|\__,_|_| \_/ |_| |_| |_| :: Spring Boot :: ${spring-boot.version} :: Running on Java :: ${java.version} :: Application :: ${app.version}

Actuator

Actuator provides production-ready endpoints for monitoring the application. Just adding the following dependency in pom.xml will do. Once the application is up, check http://localhost:8080/actuator

<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency>

However, by default not all end-points are enabled. Only /actuator, and /actuator/health end-points are enabled. By setting the property management.endpoints.web.exposure.include=* in application.properties or corresponding application.yml will activate all end-points. 

Also, multiple end-points can be listed separated by comma.
e.g. management.endpoints.web.exposure.include=health,info,beans,env

Selectively certain end-points can be disabled, for instance to disable refresh end-point, set management.endpoints.web.exposure.exclude=refresh 

NOTE: The refresh end-point requires an empty POST request. e.g. curl -X POST http://localhost:8081/actuator/refresh 

Application information

The /actuator/info endpoint displays application information. By default there is no information. So, http://localhost:8080/actuator/info displays empty JSON:
{}

Maven plugin - spring-boot-maven-plugin

This plugin comes with build execution goal: build-info which is run by default and creates build information file: build-info.properties under target/classes/META_INF directory. Properties listed in this file are available through the endpoint: http://localhost:8080/actuator/info.
The following is an example of build-info goal execution configuration:

<plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> <executions> <execution> <!-- Useful info on /actuator/info --> <id>build-info</id> <goals> <goal>build-info</goal> </goals> </execution> </executions> </plugin>

The above configuration generates build-info.properties file with pre-defined build info properties like:
build.artifact=boot-graalvm build.group=com.example build.name=boot-graalvm build.time=2023-09-22T16\:04\:27.970Z build.version=0.0.1-SNAPSHOT

With the above file generated, the /actuator/info endpoint response would look like:
{ "build": { "artifact": "boot-graalvm", "name": "boot-graalvm", "time": "2023-09-22T16:04:27.970Z", "version": "0.0.1-SNAPSHOT", "group": "com.example" } }
 
Additional custom properties can be added by configuring the build-info goal. For instance to add Java version and the Spring Boot version that the application is running with, the following additional configuration can be added:
... <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>3.1.2</version> <relativePath/> <!-- lookup parent from repository --> </parent> <groupId>com.example</groupId> <artifactId>boot-graalvm</artifactId> <version>0.0.1-SNAPSHOT</version> <name>boot-graalvm</name> <description>GraalVm project for Spring Boot</description> <properties> <java.version>20</java.version> <spring.boot.version>${parent.version}</spring.boot.version> </properties> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> <executions> <execution> <!-- Useful info on /actuator/info --> <id>build-info</id> <goals> <goal>build-info</goal> </goals> <configuration> <additionalProperties> <java.version>${java.version}</java.version> <spring.boot.version>${spring.boot.version}</spring.boot.version> </additionalProperties> </configuration> </execution> </executions> ...

Which would result with additional properties in the build-info.properties file like:
build.java.version=20 build.spring.boot.version=3.1.2

The /actuator/info endpoint response looks like:
{ "build": { "java": { "version": "20" }, "spring": { "boot": { "version": "3.1.2" } }, "version": "0.0.1-SNAPSHOT", "artifact": "boot-graalvm", "name": "boot-graalvm", "time": "2023-09-22T18:52:39.193Z", "group": "com.example" } }

Java information

Info endpoint has several info contributors like build, env, java, git etc. By default they are disabled. Enabling these would show related information collected by Spring.

To enable java info, add management.info.java.enabled=true in application.properties or management.info.java.enabled:true in application.yml.

This will show the following java info details collected like:
{ "build": { "java": { "version": "20" }, "spring": { "boot": { "version": "3.1.2" } }, "version": "0.0.1-SNAPSHOT", "artifact": "boot-graalvm", "name": "boot-graalvm", "time": "2023-09-22T18:52:39.193Z", "group": "com.example" }, "java": { "version": "20", "vendor": { "name": "Amazon.com Inc.", "version": "Corretto-20.0.0.36.1" }, "runtime": { "name": "OpenJDK Runtime Environment", "version": "20+36-FR" }, "jvm": { "name": "OpenJDK 64-Bit Server VM", "vendor": "Amazon.com Inc.", "version": "20+36-FR" } } }

TIP

These build properties generated and available through /actuator/info are also are available through BuildProperties object which can be auto-wired into any of the Spring managed beans and be accessed. For instance these properties can be outputted on swagger-ui page.


References

Saturday, May 07, 2022

Keep your Maven builds DRY - leverage placeholder feature in multi-module project for version . . .

Another Maven blog post in a row, makes me feel like digging into Maven never ends ;). It's an XML world anyway, and requires considerable effort in making any small feature change to work.

I created a maven multi-module Spring Boot micro-service application a couple of years ago which is a key service for the business. Every time when there is a feature change, or a new feature addition, I always look for opportunities to upgrade tech-stack. Of course, Maven cannot be left behind. I keep upgrading maven wrapper, the tech-stack, and make build scripts better following the DRY programming principle.

Problem Context

The Spring Boot micro-service is a maven multi-module build project with about 4 sub modules (lib, domain, etl, and api). The root module and all sub-modules have semantic <version> tag specified. Due to some limitations that I ran into earlier with an older maven version, the semantic <version> tag value in all modules was repeated. So, every time when there is a version change, we had to update value in all modules. Our CI environment tags every pull request that gets merged into master/main Git branch by appending timestamp, and git-commit-id to the semantic version tag specified in the build scripts like: <semantic-version>-<timestamp in YYYYMMddHHmm format>.<short-commitId> (e.g. 2.0.1-202205070824.174cd82), thus making every commit a release candidate. However, the semantic version that we specify in maven builds is what we decide to change based on the nature of the feature. 

Environment: Java 17, Spring Boot 2.5.6, maven 3.8.5 on macOS Catalina 10.15.7

Starting from 3.5.0 maven started to allow placeholders for versions. A property (e.g. my-version) can be defined in root module with a value and the property can be used as a placeholder for <version> tags in all modules including the root module like: <version>${my-version}</version>. This feature alone might work for a single module project, but doesn't work when you have a multi-module maven build project. An extra plugin is needed for it to work. Otherwise your placeholders in sub-module dependencies are not resolved and replaced with its value and causes errors on instances like when you run any maven goal for a specific sub-module that has root module specified in <parent> block. Several blogposts and Stackoverflow question-answer references only talk about this feature with example XML snippets. 

Maven Flatten Plugin

The missing important piece is the Maven Flatten Plugin. This plugin makes the feature complete. Maven documentation about this feature does talk about this. But due to the nature of today's fast-paced development and not that great Maven's documentation, developers rely more on Stackoverflow and other direct Google hits. I also went through this, but finally ended up reading maven documentation, and test trials to make it work.

Following are examples pom.xml snippets of this feature.

Root module's pom.xml
<project ...> <groupId>com.giri.services</groupId> <artifactId>my-service</artifactId> <version>${my-service.version}</version> <modules> <module>my-lib</module> <module>my-service-domain</module> <module>my-service-etl</module> <module>my-service-api</module> </modules> <properties> <my-service.version>2.1.0-SNAPSHOT</my-service.version> ... </properties> <build> ... <plugins> ... <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>flatten-maven-plugin</artifactId> <version>1.2.7</version> <configuration> <updatePomFile>true</updatePomFile> <flattenMode>resolveCiFriendliesOnly</flattenMode> </configuration> <executions> <execution> <id>flatten</id> <phase>process-resources</phase> <goals> <goal>flatten</goal> </goals> </execution> <execution> <id>flatten.clean</id> <phase>clean</phase> <goals> <goal>clean</goal> </goals> </execution> </executions> </plugin> </plugins> </build> </project>

Sub-module's pom.xml
<project ...> ... <parent> <groupId>com.giri.services</groupId> <artifactId>my-service</artifactId> <version>${my-service.version}</version> </parent> <artifactId>my-service-domain</artifactId> <packaging>jar</packaging> ... </project>

With this, next time when I want to bump up revision numbers, I only need to change the value in one pom.xml file from the root, unlike 5 earlier. That makes it DRY.

TIPS

IntelliJ inline Error in sub-module's pom.xml file
IntelliJ IDEA still complains with an error saying Properties in parent definition are prohibited for the inline placeholder in sub-module's <parent> block though you set it to use maven wrapper of your app or maven installed on your system which is higher than 3.5.0, or whatever through IDEA preferences for Maven Build.

Simply ignore this. Your build works both inside IDEA or outside from command line.

Extra . files generated
Also, notice that there are extra .flattened-pom.xml files generated in root and every sub-module folders. Just let them hang around there.

Avoid conflicting placeholder names
I wouldn't use ${revision} as  as the placeholder name for this feature as specified in the maven document when I also have the release candidate plugin. Release candidate plugin has this placeholder name reserved for git-commit-id.

References

Wednesday, December 08, 2021

Maven Dependencies Fix - Need to support both Log4j (ver 1) & Log4j2 (ver 2) but exclude Log4j (ver 1) . . .

Software development never gets simple. A simple logging in modern Java applications is not simple enough. There are several logging frameworks: java.util.logging, log4j, log4j2, SLF4J, Logback etc. A typical Java application these days brings in one, or more, or even all of these dependencies. It's often a confusion which one is better, which one to use, which one is in action, which one to configure, and how to configure etc. It becomes worse if you need to exclude any.

I was recently working on a task to fix Security Vulnerabilities detected and reported by a commercial SaaS tool that scans Java project codebase and it's dependent libraries. The tool scans and generates a report listing vulnerabilities detected by categorizing each into Critical/High/Medium/Low. Obviously, when there is an unpatched dependent library marked Critical, that draws superior-attention and brings in a worry with the word: "hacking"- a scary or not-scary word in Software Engineering, it depends ;).

This post is NOT about the messy path that Java logging has been going on right from the beginning. It's about dealing with a need to support both log4j (ver 1) and log4j2 (ver 2) in a Java project but want to exclude log4j from the dependency list to fix the security vulnerability reported in reference to the specific vulnerability reported on log4j in the National Vulnerability Database

To deal with this issue, all you ned to do is exclude log4j from the dependency library that depends on it, and add an explicit dependency of log4j2 and the log4j1-2 bridge.

The following is a snipper of maven pom.xml.
... <properties> <log4j.version>2.15.0</log4j.version> </properties> <dependencies> ... <dependency> <groupId>com.some.lib</groupId> <artifactId>somelib-need-log4j</artifactId> <exclusions> <!-- Fix Security Vulnerability reported with log4j-1.2.17, the last discontinued version of log4j 1. Exclude log4j 1 and depend on the log4j 1 to 2 bridge and bring in log4j 2 explicitly --> <exclusion> <groupId>log4j</groupId> <artifactId>log4j</artifactId> </exclusion> </exclusions> </dependency> <!-- Fix Security Vulnerabily reported with log4j-1.2.17 Log4j 1 to 2 bridge to forward log4j 1 calls to 2 --> <!-- log4j 1 to 2 bridge --> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-1.2-api</artifactId> <version>${log4j.version}</version> </dependency> <!-- log4j 2 --> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-api</artifactId> <version>${log4j.version}</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-core</artifactId> <version>${log4j.version}</version> </dependency> ... </dependencies> ...

TIP

Leverage maven dependency:tree goal to see which dependency's transitive dependencies bring in the library that needs an attention.

mvn dependency:tree


References

Wednesday, April 07, 2021

Maven Multi-module Gotchas . . .

Typically, in a maven multi-module project you have a parent-module/main-project and multiple sub-modules with each sub-module producing it's own artifact: jar, war etc.. The main module must be of packaging type pom (<packaging>pom</packaging>) with sub-modules listed. 

For instance, the following is main module/project's pom.xml in project's root directory with it's sub-modules listed:

... <groupId>com.my</groupId> <artifactId>my-service</artifactId> <version>1.0.0-SNAPSHOT</version> <packaging>pom</packaging> ... <modules> <module>my-domain</module> <module>my-core</module> <module>my-service-api</module> </modules>

Gotcha-1

Always use {project.version} for module dependencies.

It is also typical that a sub-module depends on another. For instance core sub-module could depend on domain sub-module. In this case the core sub-module needs to add a dependency on the domain sub-module. When such module dependencies are specified in respective module's pom.xml, never ever specify the dependency version. Instead reference {project.version}.

... <parent> <artifactId>my-service</artifactId> <groupId>com.my</groupId> <version>1.0.0-SNAPSHOT</version> </parent> <artifactId>my-core</artifactId> <packaging>jar</packaging> <name>My Core Name</name> <description>My Core Description</description> <dependencies> <dependency> <groupId>${project.groupId}</groupId> <artifactId>my-domain</artifactId> <version>${project.version}</version> </dependency> ... </dependencies>

Instead of {project.version}, if you use parent module/project version. e.g. in this case, 1.0.0-SNAPSHOT, then you might run into issues when compiling core module. For instance, when a new domain class is added to domain module and is used in core module, you might run into core module compilation issues of not finding new class added.

This is because if you have an artifact repository in which your previous my-domain-1.0.0.SNAPSHOT-*.jar versions are available maven downloads the most recent one into it's cache which results into the newly added domain class not found. This would cause nasty compilation issues giving no clues why.

When you build domain module with mvn clean install, it builds domain module and produces it's new artifact (my-domain-1.0.0.SNAPSHOT.jar) and installs it in local cache. But when you clean up cache and build core module or even main project/module (mvn clean install), the domain module doesn't get built and installed from it's sources, instead it gets downloaded form your artifact repo, which results into older version not having newly added domain class.

If you use {project.version}, and build core module or main project module with mvn clean install, it always builds and installs domain module producing new my-domain-1.0.0.SNAPSHOT.jar in local cache.

Gotcha-2

Multiple application modules sharing same test-case source code.

I recently ran into this situation. The task was to migrate a spring-boot Java micro-service api application from MySQL to PostgreSQL. But let the MySQL api app continue be in place for sometime in parallel along with new api-pg app module added which interfaces with PostgreSQL database. With the addition of new api-pg module, two api artifacts would come out of the maven build: existing api app, and new api-pg app.

First I modularized the existing spring-boot application backed by typical old-fashioned JDBC, DAO layer with inline SQL statements for MySQL database into multiple modules like domain, core, api etc in order to facilitate code reuse between two api applications with minimal specific code in each application. Then added a new api-pg module which is a spring-boot application by itself with minimal Java source code like, main spring-boot application, additional Java configurations, data access layer (DAO Impl), and bootstrap configuration files etc.

This posed a challenge to make the test-cases runnable during maven build for both the applications, keeping the source code in one module. Maven by default runs all test-cases during it's test phase unless otherwise told to skip by passing additional flag like: -DskipTests. As test-cases source code was chosen to be left in the api app module (MySQL based), maven finds it in test-phase and runs. Where as, for the new api-pg module, as test-cases source code is not there in that module, it wouldn't bother to run.

So, it requires bit of a hack getting test-case source code in one api module but making it available in both api modules during maven test-phase of each module. The module that has source code is obviously compiled and ran during it's test phase. It doesn't make sense to somehow make test-case source code available for the other module. The new api-pg module at least needs the compiled classes to be available in Maven's target directory to make them running during this apps test-phase.

That idea of having test-cases source code in one module, getting compiled and run for that module, but make compiled test-case classes available for the other module requires bit of hack. This is is where the following two plugins come in handy to deal with this kind of situation:


The maven-jar-plugin can be leveraged in api module to create a jar file of all the compiled test-case classes from the api module.

The maven-dependency-plugin can be leveraged in api-pg module to unpack the jar file to it's target/test-classes dir so that they get executed as part of it's build. This technique works pretty neat.

By leveraging maven-jar-plugin, the maven build file of api application module: pom.xml  needs the following addition:

... <parent> <artifactId>my-service-api</artifactId> <groupId>com.giri</groupId> <version>1.0.0-SNAPSHOT</version> </parent> ... <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-jar-plugin</artifactId> <version>3.2.0</version> <executions> <execution> <goals> <goal>test-jar</goal> </goals> </execution> </executions> </plugin> ... </plugins>

With the above change, when the api app is built, it produces an additional jar file like: <artifactId>-<version>-tests.jar. For instance, if the artifactId of api module is my-api-service and version is 1.0.0-SNAPSHOT, then the build produces my-api-service-1.0.0-SNAPSHOT-tests.jar file.

Now, by leveraging maven-dependency-plugin, the maven build file of api-pg application module: pom.xml  needs the following addition:
 
... <parent> <artifactId>my-service-api-pg</artifactId> <groupId>com.giri</groupId> <version>1.0.0-SNAPSHOT</version> </parent> ... <dependencies> <dependency> <groupId>com.giri</groupId> <artifactId>my-service-api</artifactId> <version>${project.version}</version> <classifier>tests</classifier> <type>test-jar</type> <scope>test</scope> </dependency> </dependencies> ... <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-dependency-plugin</artifactId> <executions> <execution> <id>unpack</id> <phase>process-test-classes</phase> <goals> <goal>unpack</goal> </goals> <configuration> <artifactItems> <artifactItem> <groupId>com.giri</groupId> <artifactId>my-service-api</artifactId> <version>${project.version}</version> <type>test-jar</type> <outputDirectory>${project.build.directory}/test-classes</outputDirectory> </artifactItem> </artifactItems> </configuration> </execution> </executions> </plugin> ... </plugins> ...

Make sure that you also add test scope dependency for the artifact (tests jar) that gets produced.

That's it. When you run the project build, when maven builds api module, it compiles all test, runs and builds. Then, when it goes for api-pg module build, it unpacks the compiled test classes into it's target and runs all tests for this module as well. This way test-cases source resides in one module, but gets run in both the modules.

NOTE
You may need to make sure that in the main project pom.xml file you list modules in the order that you want them to be built. For instance:

... <modules> <module>my-service-domain</module> <module>my-service-core</module> <module>my-service-api</module> <module>my-service-api-pg</module> </modules> ...

TIP

  • In a multi-module project, whenever there are issues with dependencies, always go to local maven cache for your groupId: ~/.m2/repository/com/my and blow out all sub-directories, files and then run maven build from root project. If  a specific sub-module runs into issues, blow out all that sub-module's module dependencies in maven cache and just build that sub-module and see.

Friday, March 05, 2021

Spring Cloud Config - Gotchas . . .

Spring Cloud Config takes externalization of application property-resources or configuration-files (name-value pair property files: *.properties, and YML files: *.yml) one step further away from the application into an external and central location like a git repo or a file-system.

It comes with a server-side application that accesses property resources from the central location and makes them available for client applications through HTTP resource-based API end-points. This Server application can easily be embeddable in a simple Spring Boot application. Once embedded in a Spring Boot application, this will be your Cloud config server application to be integrated with all client applications. Client applications integrate with the server by setting spring.cloud.config properties like uriuser, password etc.

Environment: Java 15, Spring Boot 2.3.8.RELEASE on macOS Catalina 10.15.7

Gotchas

  • Client applications are identified by Cloud config server by their application names (spring.application.name property value of the respective client application)
  • With git repository hosting property resource files (configuration files like *.properties or *.yml), if you use application.yml or application.properties names for configuration files, these files become common configuration to all client applications that depend on Cloud config server for externalized config files.
  • One config server can serve multiple clients' (multiple applications) externalized configurations. To best leverage this feature, organize externalized config files by the application's name. For instance if there are two Spring Boot client applications (named: my-app-1, and my-app-2) and both are configured to use one Spring Cloud config server for their externalized configurations hosted in a git repo, better to have two separate config files like my-app-1.yml and my-app-2.yml for the respective application configurations in the git repo. One yml file can have configurations for all environments like local, dev, int, cert, prod etc with each environment mapping to respective spring profile and each profile configuration separated by yml directive ---
  • The HTTP service of Cloud Config Server exposes end-points in different forms with client-application-name, client-application-profile, and label as part of the end-point forms. Label is nothing but the git branch and is optional with master as implicit value.
  • With git repository set as the host for clients' externalized configuration files, if any specific changes to those files are made and committed in a branch other than master, the cloud config server does not serve that branch unless it is explicitly asked for by specifying the optional label part in the end-point URL. This is the case even if that specific branch is currently checked out.
  • When label (branch name) is not specified, it always uses the implicit label and checks out master branch and responds with the configuration taken from property resources available in the master branch.
  • If explicitly asked for a different branch by  specifying the branch-name as the label in the end-point URL, it checks out that specific branch and servers property resources available in that branch for the given application and it's profile.

    Local Testing

  • For local testing, if your git repo (.git) for clients' external configuration files is in a directory under your home e.g. ~/dev/my-cloud-config, then set spring.cloud.config.server.git.uri to file://Users/<yourMacUserId>/dev/my-cloud-config in the cloud config server spring boot application bootstrap.yml/application.properties for the profile, let's say: int
  • Assuming your cloud config server is running locally on port:8888, the URLs: 1) http://localhost:8888/my-app-1/int 2) http://localhost:8888/my-app-1-int.yml 3) http://localhost:8888/my-app-1-int.properties to check my-app-1 application's int environment/profile configuration results in checking out master branch to serve my-app-1 application's configuration. If you are on a branch other than master, you will notice that you will be switched to master once you make a request to the above URLs to verify the configuration. This is due to the implicit master branch checkout that the cloud config server does under the hood.

  • In your local git repo where client applications' config files are hosted, if your are on a branch other than master and have local changes pending (not committed), the implicit master checkout fails resulting into an exception like:
    org.springframework.cloud.config.server.environment.NoSuchRepositoryException: Cannot clone or checkout repository: file:///Users/<myMacUserId>/dev/my-cloud-config
  • If your configuration files for client applications are on a specific git branch other than master, and if you want to test the cloud config server against that specific branch, then the endpoint /{application}/{profile}[/{label}] URL (e.g. http://localhost:8888/my-app-1/int/my-branch may work.

    Encrypted Content

  • If clients' configuration files have password property like spring.datasource.password and the values are plain text, then the response shows the property with the original value. If the password uses an encrypted value starting with {cipher} followed by encrypted password (e.g. `{cipher}my3ncryp!ed^assword`) then the response doesn't contain spring.datasource.password property or it's value. Instead, you will see a special property "invalid.spring.datasource.password" with a special value "<n/a>"   "invalid.spring.datasource.password": "<n/a>". That means you gotta use a special way to see those encrypted password properties. There is a separate section in the doc explaining this with two special end-points (/encrypt, /decrypt) provided.

TIP

  • In this case, your client application's property resource file changes are on a specific branch, you are unable to get that working, you will be better off changing the Cloud Server Spring Boot application's default implicit master branch to your specific branch by setting the property spring.cloud.config.server.git.default-label to your branch name. This works even if your branch name has / in it (e.g. feature/my-branch).

References


Thursday, October 08, 2020

Make your Spring Boot application's API documentation a complete specification with enhanced Swagger-UI annotations . . .

In a RESTful application, documenting end-point specification/schema is very important. There are various frameworks with different approaches available in Java space addressing this problem. It is obvious the best way is: to generate API specification from the source code so that it stays on up-to-date and accurate with your source code.

Spring RESTDocs offers a very good solution. It generates API doc from hand-written Asciidoctor templates merged with auto-generated snippets that are generated from unit/integration tests by promoting end-point testing to great levels. (Refer to my earlier post on this in a Grails application.)

Swagger UI is another solution which generates visual documentation from the source code. This also generates a testable Swagger UI page for all end-points along with Open API specification for each end-point. To get this right and complete, it requires adding additional details for documenting API specification/schema either in an yml file or by annotating source code, basically end-point action methods and objects involved in request/response handling.

Swagger UI is very useful and convenient to not only to know the specification details, but also to test REST APIs both from the same page. Spring boot comes with good support for this. I am not going to go into details of how to add Swagger UI support with Open API specification for a Spring Boot application. There are numerous posts on this.

This post is more on leveraging Swagger (OpenAPI 3 implementation) annotations in order to get better API specification/schema generated. Also, it goes into details on customizing the example end-point request/response JSON that shows sample request with meaningful data, rather than default data. Without adding any specific annotations for API specification, you will get a decent Swagger UI page. However, it is good to add little more details and make the specification much cleaner and clear.

Environment: Java 13, Spring Boot 2.2.4.RELEASE, PostgreSQL 12.1, Maven 3.6.2 on macOS Catalina 10.15.6

Without any additional Swagger annotations

For instance, in a Spring Boot application, a POST operation end-point method to create a Person, and the request objects with no additional swagger annotations like shown below:

@RestController @Slf4j public class PersonController { ... @PostMapping(value = "/person", produces = { MediaType.APPLICATION_JSON_VALUE, MediaTypes.HAL_JSON_VALUE }) public ResponseEntity create(@Valid @RequestBody Person person) { ... return new ResponseEntity<>(newPerson, HttpStatus.OK); } } @Data public class Person { private String firstName; private String lastName; private Gender gender private int age; private String email; private Address address; } @Data public class Address { private String address1; private String address2; private String city; private String state; private String zip; } public enum Gender { FEMALE, MALE }

would result into Swagger UI as shown below:


and request schema details look like:


Note that the example request JSON is not good with respect to data for fields. When you click on Try it out button to test the API, you will have to edit the values of all fields with good data. To have a  good example request with good sample data generated in Swagger UI page requires additional Swagger annotations.

With additional Swagger annotations

Enhancing code by adding annotations as shown below:

@RestController @Slf4j public class PersonController { ... @Operation(summary = "Creates a new Person.", tags = { "Person" }) @ApiResponses(value = { @ApiResponse(responseCode = "200", description = "Returns newly created Person."), @ApiResponse(responseCode = "403", description = "Authorization key is missing or invalid."), @ApiResponse(responseCode = "400", description = "Invalid request.") }) @PostMapping(value = "/person", produces = { MediaType.APPLICATION_JSON_VALUE, MediaTypes.HAL_JSON_VALUE }) public ResponseEntity create(@Valid @RequestBody Person person) { ... return new ResponseEntity<>(newPerson, HttpStatus.OK); } } @Data @Schema( description = "A JSON request object to create Person" ) public class Person { @NotNull @Size(min = 4, max = 128) @Schema(example = "John") private String firstName; @NotNull @Size(min = 4, max = 128) @Schema(example = "Smith") private String lastName; @NotNull @Schema(type = "enum", example = "MALE") private Gender gender; @NotNull @Min(1) @Max(100) @Schema(type = "integer", example = "25") private int age; @NotEmpty @Email @Schema(example = "john.smith@smith.com") private String email; @NotNull @Valid private Address address; } @Data @Schema( example = """ { "address1" : "1240 E Diehl Rd.", "address2" : "#560", "city" : "Naperville", "state" : "IL", "zip" : "60563" } """ ) public class Address { @NotNull @Size(min = 4, max = 128) @Schema(example = "1 N Main St.") private String address1; @Schema(example = "Apt. 100") private String address2; @NotNull @Size(min = 4, max = 128) @Schema(example = "Sharon") private String city; @NotEmpty @Size(min = 2, max = 2) @Schema(example = "MA") private String state; @NotEmpty @Size(min = 5, max = 5) @Schema(example = "02067") private String zip; } public enum Gender { FEMALE, MALE }


would result into Swagger UI as shown below:


and request schema details look like:


Note that the specification and example is much cleaner with good data for all elements.

@NotNull, @NotEmpty, etc. - javax Validation Annotations

Also, javax field constraint annotations used for validation are very well considered. For instance all required fields (annotated with @NotNull or @NotEmpty) are marked as required elements with suffix * added to the element name.

Also, any invalid request results with more meaningful error response. In the example shown below, required field gender is missing and age has invalid value 0 sent in the request: 




@Operation - swagger ui annotation

Annotate resource operations (controller methods) to add more details. The summary element of this annotation can be leveraged to add a meaningful description about the operation. The default will not add any description. The tags element can be leveraged to logically group operations so that they all show up under that tag on the page. If not specified, the default tag value is hyphenated class-name, in the above code example (without annotations), the default tag value is: person-controller.

@Operation(summary = "Returns a list of MyDomain", tags = { "MyDomain" })

@ApiResponsse - swagger ui annotation

Further enhance Response descriptions by annotating controller method with @ApiResponses and describing every possible response code as shown below:

@ApiResponses(value = { @ApiResponse(responseCode = "200", description = "Returns list of MyDomains."), @ApiResponse(responseCode = "403", description = "Authorization key is missing or invalid."), @ApiResponse(responseCode = "400", description = "Invalid request.") })

Also, the class can be annotated with @ApiResponse annotation for describing all common response codes like 400, 401, 404, 500 etc. to keep annotations DRY. The controller methods can just describe 200 and any additional specific response codes. Also, can override class level annotated common response code descriptions. The following is an example annotation at the class level common for all controller methods:
 
@ApiResponses(value = { @ApiResponse( responseCode = "400", description = "Bad Request.", content = { @Content(mediaType = "application/json", schema = @Schema(implementation = Errors.class)) } ), @ApiResponse( responseCode = "401", description = "Unauthorized. Authorization key is missing or invalid.", content = { @Content(schema = @Schema(implementation = Void.class)) } ), @ApiResponse( responseCode = "404", description = "Not Found.", content = { @Content(schema = @Schema(implementation = Errors.class)) } ), @ApiResponse( responseCode = "500", description = "Internal Server Error.", content = { @Content(schema = @Schema(implementation = Errors.class)) } ) }) public class PersonController { ... @ApiResponses(value = { @ApiResponse(responseCode = "200", description = "Returns newly created Person.") }) @PostMapping(value = "/person", produces = { MediaType.APPLICATION_JSON_VALUE, MediaTypes.HAL_JSON_VALUE }) public ResponseEntity create(@Valid @RequestBody Person person) { ... return new ResponseEntity<>(newPerson, HttpStatus.OK); } }

For error responses with codes like 400, 404, 500 etc., that return spring Errors object for any kind of failures like validation, exceptions etc. the implementation class can be specified as shown above. If there no-content for any response like 401, then Void.class is suitable which results with no details/schema for the response.

@Schema - swagger ui annotation

Annotate request and response objects with this annotation to describe it. Also, annotate object properties to add data type and example data in order to enhance sample request with more meaningful data.

The type element of this annotation can be used to specify type data type and example element to specify example value. Otherwise, the value defaults to Java default. For enums it picks the first one in the list of enumerations.

For String data types, if there is a specific set of values expected the list can be specified as an array of Strings for allowedValues element. This shows up in the schema for that element as an enumeration of values allowed.

TIPS

  • @Schema annotation can be used at class level to specify a JSON representation of the object with meaningful data for all the object fields as an example as shown for the Address object in the code snippet above. When specified at this level it takes precedence over field level example data. I used Java 13 preview feature of multiline string in there.
  • http://localhost:8080/swagger-ui.html shows Swagger UI page of your application. It basically redirects to http://localhost:8080/swagger-ui/index.html?configUrl=/v3/api-docs/swagger-config.
  • http://localhost:8080/swagger-ui/index.html gives the Swagger UI page for pet-store based on https://petstore.swagger.io/v2/swagger.json. This is enabled by default. I have not found a way to disable this :(
  • If there is a collection property like List in the object, for instances a List<Address> address; then you need to annotate it as shown below:
@ArraySchema(schema = @Schema(implementation = Address.class)) List<Address> addresses;
  • Operations can be logically grouped by tags. Each tag can have a name and description properties. If annotations are used, then @Operation annotation can only take tag names, but no description. This is a limitation. The @Tag annotation supports both name, and description properties. So, if there are couple of operations that need to be grouped into one by one tag name, but also want to have a description, then one operation/method can use @Tag annotation with name and description, the other operation/method can use @Operation with tags property. This works and both operations get grouped under same tag name, and tag description is also shown along with tag name. So, @Tag and @Operation can mix and match across various operations/methods for the same tag group.
  • By default all response messages are generated for response codes: 200, 400, 403, 404, 405, 406, 500, 503 in the responses section of the page, though the method is annotated with @ApiResponses annotation for only response codes 200, 400, and 403. In order to fix this you need to add the following property in application.yml or application.properties appropriately (springdoc properties).
springdoc: override-with-generic-response: false