tag:blogger.com,1999:blog-136082492024-03-17T17:18:44.909-04:00My Te(a)ch NotesMy Technology Notes - to share my ideas, findings and learnings of new Software Technologies, primarily focusing on JVM languages and frameworks.Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.comBlogger82125tag:blogger.com,1999:blog-13608249.post-90702630663669848942024-03-16T19:40:00.006-04:002024-03-17T17:18:12.188-04:00Spring Data JPA - Join Fetch, Entity Graphs - to fetch nested levels of OneToMany related data . . .<span style="font-size: x-large;">O</span>ne-to-many relationship in Databases is quite common. It is also quite cumbersome in terms of how many aspects that need to be considered for getting it correctly implemented. Just to list few aspects - related JPA annotations, relationship keys specified in annotations, fetch modes, fetch types, joins, query types, performance, N+1 data problem, cartesian product, DISTINCT to eliminate duplicates, indexes etc. With Spring Data, JPA and Hibernate as the default implementation provider there are few JPA annotations, Hibernate specific annotations, JPQL queries, Java collection types, all these will get added to the mix.<div><br /></div><div><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 21, </span><span style="font-family: "courier new", courier, monospace;">Spring Boot 3.2.3,</span><span style="font-family: "courier new" , "courier" , monospace;"> PostgreSQL 15.3, maven 3.9.6 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div><span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div><div>If you have JPA entities related through OneToMany relationships two multiple levels, then some special care is required to fetch data in a performant manner by avoiding the classic N+1 query issue, or even multiple queries. Eash query is a roundtrip to Database and it adds up its own baggage.</div><div><br /></div><div>Let's take a simple example of Country has many States, State has many Cities. We want to represent this relationship in JPA Entities and query using Spring Data JPA Repositories.</div><div><br /></div><div>The entities with relationships look something like:</div><div><br /></div>
<div class="code">
public abstract class BaseEntity implements Serializable {
private static final long serialVersionUID = 1L;
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(nullable = false, updatable = false)
@ToString.Exclude
protected Long id;
/** For optimistic locking */
@Version
protected Long version;
@CreationTimestamp
@Column(nullable = false, updatable = false, columnDefinition = "TIMESTAMP WITH TIME ZONE")
protected OffsetDateTime dateCreated = OffsetDateTime.now();
@UpdateTimestamp
@Column(nullable = false, columnDefinition = "TIMESTAMP WITH TIME ZONE")
protected OffsetDateTime lastUpdated = OffsetDateTime.now();
}
@Entity
public class Country extends BaseEntity {
private String name;
@OneToMany(mappedBy = "country", cascade = CascadeType.ALL, orphanRemoval = true)
@Builder.Default
@ToString.Exclude
private Set<State> states = new LinkedHashSet<>();
}
@Entity
public class State extends BaseEntity {
private String name;
@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "country_id")
@ToString.Exclude
private Country country; // owning side of the relationship
@OneToMany(mappedBy = "state", cascade = CascadeType.ALL, orphanRemoval = true)
@Builder.Default
@ToString.Exclude
private Set<City> cities = new LinkedHashSet<>();
}
@Entity
public class City extends BaseEntity {
private String name;
@ManyToOne
@JoinColumn(name = "state_id")
@ToString.Exclude
private State state;
}
</div><div><br /></div>
And a repository interface like:
<div class="code">
@Repository
public interface CountryRepository extends JpaRepository<Country, Long> {
Optional<State> findByName(String name);
}
</div><div><br /></div>
One way to fetch all related data in a single query is by writing JPQL query with <span style="font-family: courier;">JOIN FETCH</span>. This involves making sure to use all <span style="font-family: courier;">@OneToMany</span> annotated properties to use <span style="font-family: courier;">Set</span> and not <span style="font-family: courier;">List</span>, and not using <span style="font-family: courier;">FetchType.EAGER</span> and <span style="font-family: courier;">FetchMode.JOIN</span>., and by writing a JPQL query with <span style="font-family: courier;">@Query</span> annotation as shown below. Make a note of both DISTINCT and JOIN FETCH.
This will result into one query which fetches for a Country all States, for each State all Cities data.
If it is huge set of records, your best bet is to use <span style="font-family: courier;">@EntityGraph</span> recommended. Lets say that our data is not huge and we want to use JPQL. In this case, the repository method annotatted with JPQL Query looks like:<div><br />
<div class="code">@Repository
public interface CountryRepository extends JpaRepository<country long=""> {
@Query("""
SELECT DISTINCT country from Country country
JOIN FETCH country.states state
JOIN FETCH state.cities city
WHERE country.name = :name
""")
Optional<Country> findByName(String name);
}
</country></div>
<div><br /></div><h3 style="text-align: left;">Gotcha</h3><div><ul style="text-align: left;"><li>If you use <span style="font-family: courier;">List</span> instead of <span style="font-family: courier;">Set</span>, you might bump into infamous HIBERNATE concept called <span style="font-family: courier;">bag</span> and an exception like - <span style="font-family: courier;">MultipleBagFetchException: cannot simultaneously fetch multiple bags</span> which will force you to read a whole lot of text in finding the information you need, digging through googled links and StackOverflow without much luck, and eventually breaking your head ;)</li></ul><ul style="text-align: left;"><li>There are also other ways to tackle this problem. Writing <b>native query</b> in the <span style="font-family: courier;">@Query</span> annotation is another way. I wouldn't go that route as I don't want to get sucked by databases. I am sure if you take that route, you will have someone around you ready to argue in favor of Sub Selects, Stored Procedures etc. ;). My personal preference is to stay away from diving deep into database, avoid <b>abyss</b> ;)</li></ul></div><h3 style="text-align: left;">Sample Spring Boot Application - <a href="https://github.com/gpottepalem/country-state-city" target="_blank">Country-State-City</a></h3><div><a href="https://github.com/gpottepalem/country-state-city" target="_blank">Here is the link</a> to a sample Spring Boot 3.2.3 GraphQL application which has both <span style="font-family: courier;">@Query</span> JPQL way and <span style="font-family: courier;">@EntityGraph</span> way of getting the single generated query that is performant in fetching all related data in one roundtrip.</div><div><br /></div><h3 style="text-align: left;">References</h3><div><ul style="text-align: left;"><li><a href="https://docs.spring.io/spring-data/jpa/docs/current-SNAPSHOT/reference/html/#jpa.entity-graph" target="_blank">Spring data JPA - Fetch and Load Graphs</a></li></ul></div></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-25104262321767582912024-03-09T12:26:00.006-05:002024-03-10T17:56:26.564-04:00Spring Boot - Java GraphQL - extended scalar types . . .<div style="text-align: left;">This is my first te(a)ch note on <a href="https://spec.graphql.org/" target="_blank">GraphQL</a>. I had hit a couple road blocks in a few of days of my hands on journey with it. Unlike good-old-days when books were the primary source of learning that had everything documented, there is no single place to find all details these days.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 21, </span><span style="font-family: "courier new", courier, monospace;">Spring Boot 3.2.3,</span><span style="font-family: "courier new" , "courier" , monospace;"> PostgreSQL 15.3, maven 3.9.6 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Extended </b>or<b> Custom Scalar types</b></div><div style="text-align: left;"><br /></div><div style="text-align: left;">GraphQL specifies very limited set of well-defined <a href="https://spec.graphql.org/October2021/#sec-Scalars.Built-in-Scalars" target="_blank">built-in scalar data types</a> (primitive data types): <span style="font-family: courier;">Int</span>, <span style="font-family: courier;">Float</span>, <span style="font-family: courier;">String</span>, <span style="font-family: courier;">Boolean</span> and <span style="font-family: courier;">ID</span>. GraphQL systems <b>must</b> support these as described in the specification. Everything else is an extended or custom scalar data type.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">That, obviously is a <b>very limited</b> set supported. All other data types need custom scalar implementations which basically require coercing values at run-time and converting those to Java run-time representation. Luckily, the Java ecosystem is so huge that you almost don't need to break the ground in doing so. You will always find few open-source libraries that have tackled it already for you. <a href="https://github.com/graphql-java/graphql-java-extended-scalars" target="_blank">GraphQL Java Scalars</a> is one such in this Java GraphQL for extended scalar data types.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The primitive data type set supported is just not enough. You at least need support for few other data types used in any Java application like: <span style="font-family: courier;">Long</span>, <span style="font-family: courier;">UUID</span>, <span style="font-family: courier;">DateTime</span> etc. They all need special considerations in your application's GraphQL schema. The <span style="font-family: courier;">DateTime</span> takes a very special seat. In fact, anything around Dates in Java always scares me. To humans, Date and Time are the most obvious types in day-to-day life, but non in Software systems. Date is the most abused type than any other data type. Just recollect how many billions of dollars of money was wasted on this one data type in 1998 and 1999 around the globe. After 23 years of learning the mistake, the Date is still not dealt easily; it is still a complex data type to deal with ;).</div><div style="text-align: left;"><br /></div><div style="text-align: left;">To use custom scalar types other than that limited primitive set, you have to look for writing code that handles serialization, parsing and literal parsing for each additional data type. The <a href="https://github.com/graphql-java/graphql-java-extended-scalars" target="_blank">graphql-java-extended-scalars</a> library provides implementation for many other data types.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">With a maven dependency added for this library, all you need to do is to register a scalar data type with <span style="font-family: courier;">RuntimeWiringConfigurer</span> as described in the <a href="https://github.com/graphql-java/graphql-java-extended-scalars/blob/master/README.md" target="_blank">README</a>. If you need to register multiple types, it's a builder, so you can just chain those like:</div><div style="text-align: left;"><br /></div>
<div class="code">@Configuration
@Slf4j
public class GraphQLConfiguration {
/**
* Custom scalar support for UUID, Long, and DateTime. Registers extended scalar types used GraphQL query schema.
*/
@Bean
public RuntimeWiringConfigurer runtimeWiringConfigurer() {
log.info("Registering extended GraphQL scalar types for UUID, Long, DateTime...");
return wiringBuilder -> wiringBuilder.scalar(ExtendedScalars.<span style="background-color: #fcff01;">UUID</span>)
.scalar(ExtendedScalars.<span style="background-color: #fcff01;">GraphQLLong</span>)
.scalar(ExtendedScalars.<span style="background-color: #fcff01;">DateTime</span>);
}
}
</div><div><br /></div>
In addition to this, specify these scalar types in your application's <span style="font-family: courier;">schema.graphqls</span> schema specification like:
<div class="code">"Extended scalar types"
scalar UUID
@specifiedBy(url: "https://tools.ietf.org/html/rfc4122")
scalar Long
@specifiedBy(url: "https://ibm.github.io/graphql-specs/custom-scalars/long.html")
scalar DateTime
@specifiedBy(url: "https://scalars.graphql.org/andimarek/date-time.html")
...
</div><div><br /></div>
You are good to go.<div><b><br /></b></div><div><b>Note</b> that the extended scalar type for <span style="font-family: courier;">Long</span> is named as <span style="font-family: courier;">GraphQLLong</span> by this library. But, you should use <span style="font-family: courier;">Long</span> in your your schema when you specify it as shown above. The directive <span style="font-family: courier;">@sprifiedBy</span> is recommended to be used by GraphQL specification and is also a good practice to follow. Never ignore good practices ;)<div><br /></div><h3 style="text-align: left;">Gotcha</h3><div><div style="text-align: left;"><b>Java JPA - <span style="font-family: courier;">Instant</span> vs. <span style="font-family: courier;">OffsetDateTime</span></b></div><div style="text-align: left;"><span style="font-family: courier;"><br /></span></div><div>If you are dealing with DateTime, make sure that whatever the Java type used in your code, it complies with GraphQL specification that requires date time offset.</div><div><br /></div><div>I initially used <span style="font-family: courier;">Instant</span> type in my JPA <span style="font-family: courier;">BaseEntity</span> class for two properties: <span style="font-family: courier;">createdOn</span> and <span style="font-family: courier;">updatedOn</span> that are mapped by <a href="https://hibernate.org/" target="_blank">Hibernate</a> provided <span style="font-family: courier;"><a href="https://docs.jboss.org/hibernate/orm/6.4/javadocs/org/hibernate/annotations/CreationTimestamp.html" target="_blank">@CreationTimestamp</a></span> and <span style="font-family: courier;"><a href="https://docs.jboss.org/hibernate/orm/6.4/javadocs/org/hibernate/annotations/UpdateTimestamp.html" target="_blank">@UpdateTimestamp</a></span> mapped to <a href="https://www.postgresql.org/" target="_blank">PostgreSQL</a> column type <span style="font-family: courier;"><a href="https://www.postgresql.org/docs/current/datatype-datetime.html" target="_blank">TIMESTAMP WITH TIME ZONE</a></span>. I switched to <span style="font-family: courier;">OffsetDateTime</span> type because <span style="font-family: courier;">Instant</span> is not supported and will never be by this library due to it not complying with the specification for DateTime. Java's <span style="font-family: courier;">Instant</span>, <span style="font-family: courier;">Date</span> and <span style="font-family: courier;">LocalDateTime</span> do not include offset.</div><div><br /></div><div><a href="https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/time/OffsetDateTime.html" target="_blank">OffsetDateTime</a> is an immutable representation of a date-time with an offset. This class stores all date and time fields, to a precision of nanoseconds, as well as the offset from UTC/Greenwich.</div><div><br /></div><h3 style="text-align: left;">TIP</h3><div>PostgreSQL offers two date time types: <span style="font-family: courier;">timestamp</span>, <span style="font-family: courier;">timestamptz</span> (is abbreviation of <span style="font-family: courier;">timestamp with time zone</span>).</div><div><br /></div><div>The following query results tell the date time story on this <b>day light savings day</b> of this year (Sun Mar 10, 2024). I ran it on my local <a href="https://www.postgresql.org/docs/release/15.3/" target="_blank">PostgreSQL 15.3</a> running in <a href="https://hub.docker.com/_/postgres" target="_blank">Docker container</a>.</div><div><br /></div>
<div class="code">-- Ran the query on Mar 10, 2024 day light savings day at EST 5:13:13 PM, EDT: 17:13:13
select version(); -- PostgreSQL 15.3 (Debian 15.3-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
show time zone; -- UTC
SELECT now(), -- 2024-03-10 21:13:13.956877 +00:00 (timestamp with time zone UTC)
now() AT TIME ZONE 'EST' AS est, -- 2024-03-10 16:13:13.956877 (??)
now() AT TIME ZONE 'EDT' AS edt, -- 2024-03-10 17:13:13.956877 (right)
now() AT TIME ZONE 'CST' AS cst, -- 2024-03-10 15:13:13.956877 (??)
now() AT TIME ZONE 'CDT' AS cdt, -- 2024-03-10 16:13:13.956877 (right)
now()::timestamp AT TIME ZONE 'EDT' AS timestamp_without_tz, -- 2024-03-11 01:13:13.956877 +00:00 (wrong)
now()::timestamptz AT TIME ZONE 'EDT' AS timestamptz; -- 2024-03-10 17:13:13.956877 (right)
</div>
<div><br /></div><div>Here is the <a href="https://dbfiddle.uk/MkB8XDl-" target="_blank">DbFiddle playground</a> to play with the above query.</div><div><br /></div><div>That's it in this te(a)ch note, more might come in as I walk forward along this GraphQL path.</div><div><br /><h3 style="text-align: left;">Reference</h3><div style="text-align: left;"><ul style="text-align: left;"><li><a href="https://spec.graphql.org/draft/#sec-Scalars" target="_blank">GraphQL Scalars - specification</a></li><li><a href="https://spec.graphql.org/draft/#sec-Scalars.Custom-Scalars" target="_blank">GraphQL Custom Scalar Specification</a></li><li><a href="https://graphql.org/blog/2023-01-14-graphql-scalars/" target="_blank">GraphQL DateTime Scalar type</a></li><li><a href="https://ibm.github.io/graphql-specs/custom-scalars/" target="_blank">IBM on Custom Scalars</a></li></ul><ul style="text-align: left;"><li><a href="https://www.graphql-java.com/documentation/master/scalars/" target="_blank">Scalars - GraphQL Java</a></li><li><a href="https://github.com/graphql-java/graphql-java-extended-scalars" target="_blank">GraphQL Java Extended Scalars</a></li><li><a href="https://github.com/graphql-java/graphql-java-extended-scalars/pull/26#issuecomment-1313154175" target="_blank">Why Instant is not included in extended scalar types</a></li></ul><div><ul style="text-align: left;"><li><a href="https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/time/OffsetDateTime.html" target="_blank">Java OffsetDateTime</a></li><li><a href="https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/time/package-summary.html" target="_blank">Java time package</a></li></ul><ul style="text-align: left;"><li><a href="https://vinodhinic.medium.com/handling-dst-switch-in-java-application-using-postgres-db-c434e3859272" target="_blank">Handling DST switch</a></li></ul></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div></div></div></div></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-37385452114654541932024-03-03T20:18:00.000-05:002024-03-03T20:18:19.899-05:00Enums - all the way to persistence (revisited and revised for today's tech stack) . . .<div style="text-align: left;">About two years ago I blogged on this combination: <a href="https://giri-tech.blogspot.com/2022/09/postgresql-enum-db-fiddle.html" target="_blank">Enums - all the way to persistence</a>. Technology is moving at faster pace than ever before. <a href="https://www.oracle.com/java/technologies/java-se-support-roadmap.html" target="_blank">Java's release cadence</a> is moving at rapid 6 months cycle, every March and September. Spring boot catches Java and other technologies and moves along at the same pace as Java, <a href="https://github.com/spring-projects/spring-boot/wiki/Supported-Versions" target="_blank">every 6 months in May and November</a>. Of course, the PostgreSQL database, Hibernate and even Maven build system keep moving as well, at their own pace.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The challenge for Java developer is to keep up with all the moving technologies. As software development requires talented developers with years of experience and knowledge, debates go on Artificial Intelligence (AI). Some who have moved away from coding strongly feel that the generative AI which currently is capable of generating even code will replace software developers. I don't believe in that, at least at this time. The add-on '<i>at least at this time</i>' is only a cautious extension to that non-generative human statement). I have tried <a href="https://www.codegpt.co/" target="_blank">CodeGPT</a> lately at work, a couple of times when I was stuck with things not working together as described in documentations and blog posts, asking it's generative trained intelligence to see if it would be able to be my copilot in those development situations. It couldn't really stand up the hype in anyway, and I had to go and figure out myself all those situations.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Enums persistence - is one such problem again I did hit roadblocks lately after two years. The only change is newer versions of all of these technologies. It required additional exploration of few things before arriving at a solution that worked eventually.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 21, </span><span style="font-family: "courier new", courier, monospace;">Spring Boot 3.2.3,</span><span style="font-family: "courier new" , "courier" , monospace;"> PostgreSQL 15.3, maven 3.9.6 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div><div style="text-align: left;">Spring boot 3.2.3 data jpa brings in Hibernate 6.4.4 dependency.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The same persistent model described in my earlier blog post: <a href="https://giri-tech.blogspot.com/2022/09/postgresql-enum-db-fiddle.html" target="_blank">Enums - all the way to persistence</a> would need the following changes for enums to work.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The <b>DDL script</b> requires an extra PostgreSQL casting as shown below for enums:</div><div style="text-align: left;"><br /></div>
<div class="code">-- create enum type genders
CREATE TYPE genders AS ENUM(
'MALE', 'FEMALE'
);
<span style="background-color: #fcff01;">CREATE CAST (varchar AS genders) WITH INOUT AS IMPLICIT;
</span></div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">In <b>maven</b> <span style="font-family: courier;">pom.xml</span>, the spring boot version is 3.2.3 and the <span style="font-family: courier;">hibernate-types-55</span> dependency is not needed.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Changes to <b>domain object</b> <span style="font-family: courier;">Person.java</span> are shown below (the <span style="font-family: courier;">@TypeDef</span> annotation is not required):</div><div style="text-align: left;"><br /></div>
<div class="code">
...
import jakarta.persistence.EnumType;
import jakarta.persistence.Enumerated;
import org.hibernate.annotations.JdbcTypeCode;
import org.hibernate.type.SqlTypes;
...
@NotNull
<span style="background-color: #fcff01;">@Enumerated(EnumType.STRING)</span>
<span style="background-color: #fcff01;">@JdbcTypeCode(SqlTypes.NAMED_ENUM)</span>
Gender gender;
...
}
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">Changes look simple after figuring out and making things to work, but finding things that work required a bit of exploration ;)</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">References</h3><div style="text-align: left;"><ul style="text-align: left;"><li><a href="https://www.postgresql.org/docs/current/datatype-enum.html" target="_blank">PostgreSQL Enumerated Types</a></li><li><a href="https://www.postgresql.org/docs/current/sql-createcast.html" target="_blank">PostgreSQL CREATE CAST</a></li><li>Spring Boot 3, Hibernate 6 enum mapping issue - <a href="https://stackoverflow.com/questions/75595407/spring-boot-3-hibernate-6-postgressql-enum-mapping-issue" target="_blank">stackoverflow</a></li><li><a href="https://github.com/vladmihalcea/hypersistence-utils" target="_blank">The hypersistence-utils library</a></li><li><a href="https://pgpedia.info/p/pg_cast.html" target="_blank">PostgreSQL pg_cast</a></li><li><a href="https://www.postgresql.org/docs/current/catalog-pg-cast.html" target="_blank">PostgreSQL pg_cast catalog</a></li><li><a href="https://www.baeldung.com/jpa-persisting-enums-in-jpa" target="_blank">Baeldung - persisting enums in Java</a></li></ul></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-10803545187316324862024-02-08T17:19:00.003-05:002024-02-08T17:19:29.725-05:00Spring Boot - log database query and binding parameters . . .<div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 21, </span><span style="font-family: "courier new", courier, monospace;">Spring Boot 3.2.2,</span><span style="font-family: "courier new" , "courier" , monospace;"> PostgreSQL 16, maven 3.9.6 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div><div style="text-align: left;">In a Spring Boot, Spring Data JPA application with Hibernate as the default JPA implementation, to log generated SQLs along with the values bound to parameters configure the following properties in relevant env related application.properties or application.yml file.</div><div style="text-align: left;">E.g. <span style="font-family: courier;">application-test.yml</span></div><div style="text-align: left;"><br /></div>
<div class="code">logging:
level:
org.hibernate.SQL: debug
org.hibernate.orm.jdbc.bind: trace
</div><div>With the above Hibernate log levels, generated SQL query gets logged followed by binding parameter values.</div><div><br /></div><div>To have the generated SQL query formatted, add the following JPA property:</div>
<div class="code">spring:
jpa:
properties:
hibernate:
format_sql: true
</div>
<div><br /></div><h3 style="text-align: left;">Gotcha</h3><div>Setting the property <span style="font-family: courier;">spring.jpa.show-sql</span> to <span style="font-family: courier;">true</span> is another way to see generated SQL, but it only gets outputted to the standard out, not to logs.</div><div><br /></div><div><br /></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-7793740612616926702024-01-15T19:07:00.001-05:002024-01-15T19:07:12.468-05:00Spring Boot - Docker Compose - Run init script . . .<div style="text-align: left;"><a href="https://spring.io/blog/2023/06/21/docker-compose-support-in-spring-boot-3-1/" target="_blank">Spring Boot 3.1 enhanced docker-compose support</a>, made it lot simpler and better suited for local development. With that we don't need to worry about installing services like database locally and managing them manually, letting Docker do that for us and <a href="https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#howto.docker-compose" target="_blank">Spring Boot</a> do the rest of starting and stopping docker container.</div><div style="text-align: left;"> </div><div style="text-align: left;">This post is about details explored on - how to run additional init db script with <a href="https://www.postgresql.org/docs/16/index.html" target="_blank">PostgreSQL</a> service defined in <a href="https://docs.docker.com/compose/" target="_blank">Docker compose</a> file in a Spring Boot application.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 21, </span><span style="font-family: "courier new", courier, monospace;">Spring Boot 3.2.1,</span><span style="font-family: "courier new" , "courier" , monospace;"> PostgreSQL 16, maven 3.9.6 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">The Scenario</h3><div style="text-align: left;">My <a href="https://docs.spring.io/spring-boot/docs/3.2.x/reference/htmlsingle/" target="_blank">Spring Boot 3.2.x</a> application uses PostgreSQL database, a <a href="https://www.postgresql.org/docs/16/index.html" target="_blank">specific version</a> of it. By leveraging Spring Boot support for <span style="font-family: courier;">docker-compose</span> in development, I would like to have a new schema and user created, granting the user required privileges on the schema.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">A typical PostgreSQL Service configuration in docker compose file looks like:</div><div style="text-align: left;"><span style="font-family: courier;">docker-compose.yml</span></div>
<div class="code">version: '3'
services:
<span style="background-color: #fcff01;">PostgreSQL16</span>:
image: 'postgres:<span style="background-color: #fcff01;">16.1</span>'
ports:
- '<span style="background-color: #fcff01;">54321</span>:<span style="background-color: #fcff01;">5432</span>'
environment:
- 'POSTGRES_DB=<span style="background-color: #fcff01;">my_app</span>'
- 'POSTGRES_USER=<span style="background-color: #fcff01;">postgres</span>'
- 'POSTGRES_PASSWORD=<span style="background-color: #fcff01;">s3cr3t</span>'
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">In the above docker compose configuration, we have specified database name, user, and password through environment variables in the container, and mapped host port (local port) to container port (default postgres port). With this when the docker-compose command: <span style="font-family: courier;">docker-compose up</span> is run to create and start the container, the <span style="font-family: courier;">my_app</span> database gets created, and PostgreSQL will be up and running in the container. The <span style="font-family: courier;">postgres</span> user created is the <b>superuser</b> with access and ownership to all database objects including the <span style="font-family: courier;">public</span> schema. When the docker container is created for PostgreSQL16 database service, the value of POSTGRES_USER environment variable is used to create the superuser, and <span style="font-family: courier;">public</span> is the default schema created.</div><div style="text-align: left;"><b><br /></b></div><div style="text-align: left;"><b>PostgreSQL 15 changes to <span style="font-family: courier;">public</span> schema</b></div><div style="text-align: left;"><b><br /></b></div><div style="text-align: left;">From version 15 onwards, privileges on <span style="font-family: courier;">public</span> schema are restricted and the schema is accessible to <span style="font-family: courier;">superuser</span> only. So, it is good to create an application specific schema, and application specific database user with all needed privileges granted on the application schema. This requires a way to run a one-time initial databases script for creating application schema and user. The following shell script is an example to do so:</div>
<div style="text-align: left;"><span style="font-family: courier;">init-database.sh</span></div>
<div class="code">#!/bin/sh
set -e
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
/* Create schema, user and grant permissions */
<span style="background-color: #fcff01;">CREATE SCHEMA my_app_schema;</span>
<span style="background-color: #fcff01;">CREATE USER my_app_user_local WITH PASSWORD 'password';</span>
<span style="background-color: #fcff01;">GRANT ALL PRIVILEGES ON SCHEMA my_app_schema TO my_app_user_local;</span>
EOSQL
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">In order to run the above db init script when the container is created, reference the shell script file under <span style="font-family: courier;">volumes:</span> to attach the init file directory (<span style="font-family: courier;">./</span>) to the container directory (<span style="font-family: courier;">/docker-entrypoint-initdb.d/</span>) as shown below:</div><div style="text-align: left;"><span style="font-family: courier;">docker-compose.yml</span></div>
<div class="code">version: '3'
services:
PostgreSQL16:
image: 'postgres:16.1'
ports:
- '54321:5432'
environment:
- 'POSTGRES_DB=my_app'
- 'POSTGRES_USER=postgres'
- 'POSTGRES_PASSWORD=s3cr3t'
volumes:
- <span style="background-color: #fcff01;">./init-database.sh</span>:<span style="background-color: #fcff01;">/docker-entrypoint-initdb.d/init-database.sh</span></div>
<div style="text-align: left;"> </div><div style="text-align: left;">With this, when the docker container is created for PostgreSQL service, the init db script gets executed which results with new schema <span style="font-family: courier;">my_app_schema</span> and user <span style="font-family: courier;">my_app_user_local</span>. with privileges granted.</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">Gotchas</h3><div><b>Auto configured <span style="font-family: courier;">datasource</span> properties</b></div><div>When the application is run, the PostgreSQL container is created and run by Spring Boot. It also auto configures <span style="font-family: courier;">dataSource</span> bean with properties: <span style="font-family: courier;">url</span>, <span style="font-family: courier;">username</span>, and <span style="font-family: courier;">password</span> taking them from docker compose file. The user is superuser created from the <span style="font-family: courier;">POSTGRES_USER</span> container environment variable. If the application has any initialization database scripts within the application under <span style="font-family: courier;">main/resources</span> dir like <span style="font-family: courier;">schema.sql</span> for initial schema or even flyway scripts in flyway enabled application under <span style="font-family: courier;">main/resources/db.migration</span> dir, all the database tables and other objects created are owned by the superuser as the datasource uses superuser for connecting to the database.</div><div><br /></div><div>If you want the data objects like tables, indices created in the application schema instead of public, you may need to specify it as the prefix in your schema.sql or for an app with flyway support, add the property <span style="font-family: courier;">spring.flyway.schemas</span> appropriately, e.g in <span style="font-family: courier;">application.yml</span>.</div><div><br /></div><h3 style="text-align: left;">TIPS</h3><div style="text-align: left;"><span style="font-size: large;">1.</span> With docker-compose managing the database service, if you need to use <span style="font-family: courier;">psql</span>, the terminal based frontend to PostgreSQL to connect to db and run commands, invoke <span style="font-family: courier;">psql</span> like:</div><div style="text-align: left;"><br /></div>
<div class="code">$ <span style="background-color: #fcff01;"># list docker containers running</span>
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7caf031c31a4 postgres:16.0 "docker-entrypoint.s…" 56 minutes ago Up 56 minutes 0.0.0.0:5222->5432/tcp docker-PostgreSQL16-1
45fa1a477ac3 postgres:15.3 "docker-entrypoint.s…" 4 days ago Up 3 days 0.0.0.0:54321->5432/tcp docker-compose-postgres15-1
$ <span style="background-color: #fcff01;"># run psql command to coonect to PostgreSQL16 db and list users</span>
$ docker exec -it docker-PostgreSQL16-1 psql -U postgres
psql (16.0 (Debian 16.0-1.pgdg120+1))
Type "help" for help.
postgres=#
postgres=# <span style="background-color: #fcff01;">\?</span>
General
\bind [PARAM]... set query parameters
\copyright show PostgreSQL usage and distribution terms
\crosstabview [COLUMNS] execute query and display result in crosstab
\errverbose show most recent error message at maximum verbosity
\g [(OPTIONS)] [FILE] execute query (and send result to file or |pipe);
\g with no arguments is equivalent to a semicolon
\gdesc describe result of query, without executing it
\gexec execute query, then execute each value in its result
\gset [PREFIX] execute query and store result in psql variables
\gx [(OPTIONS)] [FILE] as \g, but forces expanded output mode
\q quit psql
--More--
postgres=# <span style="background-color: #fcff01;">\dn</span>
List of schemas
Name | Owner
--------+-------------------
public | pg_database_owner
(1 row)
postgres=# <span style="background-color: #fcff01;">\du</span>
List of roles
Role name | Attributes
-------------------+------------------------------------------------------------
my_app_user_local |
postgres | Superuser, Create role, Create DB, Replication, Bypass RLS
postgres=# <span style="background-color: #fcff01;">SELECT version();</span>
version
---------------------------------------------------------------------------------------------------------------------
PostgreSQL 16.0 (Debian 16.0-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
(1 row)
postgres=# <span style="background-color: #fcff01;">\conninfo</span>
You are connected to database "postgres" as user "postgres" via socket in "/var/run/postgresql" at port "5432".
postgres=# select current_date;
current_date
--------------
2024-01-15
(1 row)
postgres=# <span style="background-color: #fcff01;">SHOW search_path;</span>
search_path
-----------------
"$user", public
(1 row)
postgres=# <span style="background-color: #fcff01;">\l</span>
List of databases
Name | Owner | Encoding | Locale Provider | Collate | Ctype | ICU Locale | ICU Rules | Access privileges
--------------+----------+----------+-----------------+------------+------------+------------+-----------+-----------------------
boot-graalvm | postgres | UTF8 | libc | en_US.utf8 | en_US.utf8 | | |
postgres | postgres | UTF8 | libc | en_US.utf8 | en_US.utf8 | | |
template0 | postgres | UTF8 | libc | en_US.utf8 | en_US.utf8 | | | =c/postgres +
| | | | | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | libc | en_US.utf8 | en_US.utf8 | | | =c/postgres +
| | | | | | | | postgres=CTc/postgres
(4 rows)
postgres=# <span style="background-color: #fcff01;">\c boot-graalvm</span>
You are now connected to database "boot-graalvm" as user "postgres".
boot-graalvm=# <span style="background-color: #fcff01;">\dt</span>
List of relations
Schema | Name | Type | Owner
--------+-----------------------+-------+----------
public | account_holder | table | postgres
public | accounts | table | postgres
public | addresses | table | postgres
public | flyway_schema_history | table | postgres
(4 rows)
boot-graalvm=#
boot-graalvm=# <span style="background-color: #fcff01;">\c postgres</span>
You are now connected to database "postgres" as user "postgres".
postgres=#
postgres=# <span style="background-color: #fcff01;">\q
</span></div>
<div style="text-align: left;"><br /></div><h3 style="text-align: left;">References</h3><div style="text-align: left;"><ul style="text-align: left;"><li><a href="https://docs.spring.io/spring-boot/docs/3.2.1/reference/htmlsingle/#features.docker-compose" target="_blank">Spring Boot Docker Support</a></li><li><a href="https://docs.docker.com/compose/" target="_blank">Docker Compose</a></li><li><a href="https://spring.io/blog/2023/06/21/docker-compose-support-in-spring-boot-3-1/" target="_blank">Spring Boot Docker Support on top of ConnectionDetails</a></li><li><a href="https://www.enterprisedb.com/blog/new-public-schema-permissions-postgresql-15" target="_blank">New Public schema permissions in PostgreSQL 15</a></li></ul></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-11024534426004030662024-01-08T18:17:00.015-05:002024-01-09T11:01:47.126-05:00Spring Boot - Check database connectivity after the application starts up . . .Database Integration is much simpler with Spring Boot's non-invasive <a href="https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#using.auto-configuration" target="_blank">Auto Configuration</a> feature. A typical Spring Boot application is configured to run in multiple environments, a.k.a <a href="https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.profiles" target="_blank">profiles</a>. However, there are multiple options available when it comes to configuring Database, like <a href="https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.docker-compose" target="_blank">Docker Compose</a>, <a href="https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.testcontainers" target="_blank">Testcontainers</a>, explicit <a href="https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#howto.data-access.configure-custom-datasource" target="_blank">DataSource</a> profile based properties/yaml, externalized DataSource properties through Vault etc. In any case, it is good to have a database connection check in place to make sure that the database connection looks good once the application boots up and starts to run.<div><br /></div><div><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 21, </span><span style="font-family: "courier new", courier, monospace;">Spring Boot 3.2.1,</span><span style="font-family: "courier new" , "courier" , monospace;"> PostgreSQL 16, maven 3.9.6 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div><br /></div><h3 style="text-align: left;">The Scenario</h3><div>The Database is PostgreSQL and we want to run a simple query to make sure that the database connection looks good once the application starts up.</div><div><br /></div><h3 style="text-align: left;">One way to achieve this</h3><div>One way to achieve this is to execute a simple query after the application starts up. Spring Boot's <a href="https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.spring-application.command-line-runner" target="_blank">CommandLineRunner</a> or <a href="https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.spring-application.command-line-runner" target="_blank">ApplicationRunner</a> can be leveraged to do this. This is a good place to run specific code after the application has started.</div><div><div><br /></div><div>Here is a code snippet for this:</div>
<div class="code">
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.jdbc.core.simple.JdbcClient;
@SpringBootApplication
@Slf4j
public class MyApplication {
public static void main(String[] args) {
SpringApplication.run(MyApplication.class, args);
}<span style="background-color: #fcff01;">
@Autowired(required = false)
JdbcClient jdbcClient;
@Bean
public CommandLineRunner commandLineRunner() {
return args -> {
if (jdbcClient != null) {
log.info("Database check: {}", jdbcClient.sql("SELECT version()").query(String.class).single());
}
};
}</span>
}
</div>
<div><br /></div><div>The above <span style="background-color: #fcff01;">highlighted</span> is the code snippet that gets executed after the application gets started. It just logs the executed query result, nothing but the database version. </div><div><br /></div><div>An integration test case can also be put in place as shown below, which makes sure that the database connection and version look good. This kind of testcase is good to have to make sure that the code is tested against the same db version as the production.</div><div><br /></div>
<div class="code">
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.jdbc.AutoConfigureTestDatabase;
import org.springframework.boot.test.autoconfigure.orm.jpa.DataJpaTest;
import org.springframework.context.annotation.Import;
import org.springframework.jdbc.core.simple.JdbcClient;
import org.springframework.test.context.ActiveProfiles;
import static org.assertj.core.api.Assertions.*;
/**
* An integration test to check Database connectivity.
*/
@ActiveProfiles("test")
// We don't want the H2 in-memory database.
// We will provide a custom 'test container' as DataSource, so don't replace it.
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
@DataJpaTest
<span style="background-color: #fcff01;">@Import(TestContainersConfiguration.class)</span>
public class DatabaseCheckIT {
@Autowired
JdbcClient jdbcClient;
@Test
void database_connection_works_and_version_looks_good() {
assertThat(jdbcClient.sql("SELECT version()").query(String.class).single())
.contains("<span style="background-color: #fcff01;">16.0</span>");
}
}
</div>
<div><br /></div><div>The above test case uses <a href="https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.testing.testcontainers" target="_blank">Testcontainers</a> and a test configuration as shown below for unit/integration tests:</div><div><br /></div>
<div class="code">
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.test.context.TestConfiguration;
import org.springframework.boot.testcontainers.service.connection.ServiceConnection;
import org.springframework.context.annotation.Bean;
import org.testcontainers.containers.PostgreSQLContainer;
/**
* Test Configuration for testcontainers.
*/
@TestConfiguration(proxyBeanMethods = false)
@Slf4j
public class TestContainersConfiguration {
private static final String POSTGRES_IMAGE_TAG = "postgres:<span style="background-color: #fcff01;">16.0</span>";
@Bean
@ServiceConnection
PostgreSQLContainer postgreSQLContainer() {
return new PostgreSQLContainer<>(POSTGRES_IMAGE_TAG)
.withDatabaseName("my-application")
.withUsername("my-application")
.withPassword("s3cr3t")
.withReuse(true);
}
}</div><div><br /></div>
<h3 style="text-align: left;"><span>⚠ </span>Gotcha</h3>
</div><div>Note that in the main application class, for <span style="font-family: courier;">JdbcClient</span> <span style="font-family: courier;">@Autowired</span> annotation, the optional property <span style="font-family: courier;">required</span> is explicitly set to <span style="font-family: courier;">false</span>. The reason for this is if there are any integration test cases to test specific layers (<a href="https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#appendix.test-auto-configuration" target="_blank">test slices</a> like <span style="font-family: courier;">@GraphQlTest</span>) that do not auto configure datasource, when the application class is run as part of starting Spring Boot run, the testcase runs into exception as <span style="font-family: courier;">JdbcClient</span> bean is not available for auto wiring. So, in those cases <span style="font-family: courier;">jdbcClient</span> property would be null. So, a non null check is required to safely run the SQL statement.</div><div><br /></div>
<h3 style="text-align: left;"><span>💡 </span>TIPS</h3>
<div>The <span style="font-family: courier;">CommandLineRunner</span> bean in the application class can conditionally be defined on some DataSource related bean/class by annotation it with <span style="font-family: courier;">@ConditionalOnBean</span> or <span style="font-family: courier;">@ConditionalOnClass</span>. I couldn't find a way to get it conditionally defined and be working for all scenarios.</div><div><br /></div><h3 style="text-align: left;">Resources</h3><div><ul style="text-align: left;"><li><a href="https://docs.spring.io/spring-boot/docs/3.2.1/reference/htmlsingle/#features.spring-application.command-line-runner" target="_blank">CommadLineRunner</a></li><li><a href="https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.testing.testcontainers" target="_blank">Spring Boot support for Testcontainers</a></li><li><a href="https://medium.com/@AlexanderObregon/working-with-springs-conditional-annotation-for-conditional-bean-registration-f65bc6a486ea" target="_blank">Conditional Beans</a></li></ul></div><div><br /></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-79967252774799398972024-01-01T12:19:00.003-05:002024-01-01T12:19:26.591-05:00Polyglot makes you a think better and do better - my musing . . .<div style="text-align: left;">Fluency in multiple spoken languages (Polyglot) always makes you think better and communicate even better. In Software Development, Polyglot programming makes you a better Software Developer. Being able to code in more than one language makes you think different and write better code.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">No language is superior or best for all use-cases. Polyglot experience is very beneficial. It makes you think better when approaching a problem for a solution. In software programming world, it matters more than in the normal world.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><a href="https://en.wikipedia.org/wiki/Java_(programming_language)" target="_blank">Java</a> is undoubtedly the programming language that has been dominant in Software world, longer than any other, and probably will continue to remain dominant for many more years. I worked in <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> for a decade before I moved to <a href="https://en.wikipedia.org/wiki/Apache_Groovy" target="_blank">Groovy</a>. For several years I enjoyed coding in Groovy and did not want to go back to <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a>. Life doesn't go your way. And now, I am back to <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a>. I'd rather say, I am back to Java with <a href="https://groovy-lang.org/" target="_blank">Groovy</a> eyes and coding experience ;)</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><a href="https://groovy-lang.org/" target="_blank">Groovy</a> taught me many things in programming, which otherwise, I wouldn't have learnt or changed my object-oriented mindset to think different, if I had just stick myself to <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a>. I do notice a lot that <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> developers who have been coding in just <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> for awhile still write <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> 1.2 code. <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> is evolving faster now for good. But <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> developers are not evolving at the same pace with it. Coming back to <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> from <a href="https://groovy-lang.org/" target="_blank">Groovy</a>, I am not hesitant to use any of the new features that Java is adding version after <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> at a fast pace. I did write production <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> 13 code with multi-line text blocks which was only a preview feature in <a href="https://docs.oracle.com/en/java/javase/13/">Java 13</a> with <span style="font-family: courier;">--enable-preview</span> flag for compilation and execution. Having experienced even superior multi-line text blocks in <a href="https://groovy-lang.org/" target="_blank">Groovy</a> on JVM, I just couldn't write code with several <span style="font-family: courier;">"s</span>, and <span style="font-family: courier;">+s. </span>Some developer wouldn't even use spaces in between concatenating strings. My eyes get blurry and mind goes blank when I see such code. Polyglot helped me embrace that multi-line text blocks even as an experimental feature in <a href="https://docs.oracle.com/en/java/javase/13/">Java 13</a>.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Once in recent years, I had to get my hands dirty with a super-rigid, early twenty-first-century-way written <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> family of simple applications with <span style="font-family: courier;">main</span> methods, and tightly coupled code with inheritance, only static member variables in the class hierarchy, no sensible differences between a class and an object, the worst of all- quite a bit of blindly followed manual code changes to be done and checked in after every single run of the code, and a lot of manual copying of both input files before the run and result files after the run. Bringing in a new <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> application member to this family of applications require copying one of the applications and start making changes to meet the new application's needs with much of code inherited from the hierarchy.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">When I had to add in a new member application to that family of applications, I couldn't follow that family legacy of copy-and-paste tradition. <b>DRY</b> - <b>D</b>on't <b>R</b>epeat <b>Y</b>ourself, is the principle that I believe should be taught before even teaching programming. I added a new member to that family following all the messy inheritance as the family was super adamant upfront not to refactor anything. OK, that tells the how bad the code smells. At least I wanted to change the manual procedures and automate them, wanted to change the practice of changing code for every run. <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> application's <span style="font-family: courier;">main</span> method takes arguments for this reason. I worked for a financial company (very rigid domain in Software field) in the past and rewrote their bread and butter Oracle stored procedures that computed earnings at the end of each month with its 10,000 lines of code with not event a single line of documentation and the person who wrote it left the company. Nobody was dared to touch the code. People only knew how to calculate earnings, but had no clue how it was implemented in Stored Procedures. I rewrote the whole app in <a href="https://groovy-lang.org/" target="_blank">Groovy</a> as a simple runnable <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> app with superior command line support with all possible flexibilities to run. The whole app rewritten in <a href="https://groovy-lang.org/" target="_blank">Groovy</a> with just few hundred lines of code, made it multi-threaded by bringing down the month end run-time from hours to minutes. That was about a decade ago. If I had to this in <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> at that time, it would have made the number of lines of code at least 5 times that <a href="https://groovy-lang.org/" target="_blank">Groovy</a> with noise and boilerplate code in dealing with database.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">In my current day-to-day development, <a href="https://groovy-lang.org/" target="_blank">Groovy</a> is not a choice for production code; only Java. But, we catch up fast using latest versions of <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> in production code, few months after a newer version gets released. That makes me leverage, most recent syntax improvements, language constructs, and feature enhancements and additions being added in every version. In some cases, now, <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> code looks little closer to <a href="https://groovy-lang.org/" target="_blank">Groovy</a> like code when newer language features are used in support with frameworks.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The very first step I took in adding a new application member to the legacy family was to find good CLI <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> framework. I found <a href="https://picocli.info/" target="_blank">Picocli</a>, which is super simple to use with no coding, just annotating code. There you go, I used it and brought in a change to the family and paved path for newly joining members to follow the path. This eliminated the need to change code for every run by changing hard-coded constants and check the modified code into version control. By leveraging Picocli, and main method arguments, I externalized few hardcoded values as coming from arguments. That eliminated the need to touch code for every single run. Then automated some more tasks like renaming the generated file manually to meet certain expected naming convention, copying that to another source repo, and checking in that file etc.</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">Groovy's CliBuilder</h3><div style="text-align: left;">In my <a href="https://groovy-lang.org/" target="_blank">Groovy</a> development days, I had used <a href="https://docs.groovy-lang.org/next/html/gapi/groovy/cli/picocli/CliBuilder.html" target="_blank">Groovy's CliBuilder</a> that comes with <a href="https://groovy-lang.org/" target="_blank">Groovy</a>. Only few lines of code makes the application super flexible for driving the inside implementation, processing, or any such logic that depends on values that get passes as arguments to run the application. My <a href="https://groovy-lang.org/" target="_blank">Groovy</a> experience helped me a lot to think better, and make the newly added <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> application member a very flexible super-kid in the family by leveraging <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a>'s modern features and frameworks like Picocli. </div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">Java - Picocli</h3><div style="text-align: left;">Annotate class and fields, and add either the dependent <a href="https://picocli.info/" target="_blank">Picocli</a> class or maven/gradle dependency. With a quick couple of hours of exploration and reading the docs, in few minutes you can add the powerful CLI feature to your <a href="https://docs.oracle.com/en/java/javase/index.html" target="_blank">Java</a> Application. It makes it runnable for various scenarios by passing values through different arguments that can drive its functionality in specific ways.</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">Conclusion</h3><div style="text-align: left;">Writing code should be more for developers to read than it is for machines to execute. After all, machine can execute any type of syntactically correct code. There is more than just syntax and semantics in programming, which is READABILITY for humans. <b>Code must first be readable before it is executable.</b></div><div style="text-align: left;"><b><br /></b></div><div style="text-align: left;">Change is a constant and there is always scope for improvement, ONLY if you are willing to learn, change, and not afraid to improve ;)</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">References</h3><div><ul style="text-align: left;"><li><a href="https://www.bbc.com/future/article/20160811-the-amazing-benefits-of-being-bilingual" target="_blank">The amazing benefits of being bilingual</a></li><li><a href="https://picocli.info/" target="_blank">Picocli</a></li><li><a href="https://docs.groovy-lang.org/next/html/gapi/groovy/cli/picocli/CliBuilder.html" target="_blank">Groovy's cli builder</a></li></ul></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-57967538613846025992023-12-29T07:01:00.005-05:002024-01-09T11:02:01.772-05:00Testable Spring Boot Application main method with Docker Compose added - passing required arguments . . .<div style="text-align: left;"><span style="font-size: x-large;">J</span>ava application's entry point is its <span style="font-family: courier;">main</span> method which takes <span style="font-family: courier;">String</span> array as argument. The command line arguments are passed by Java runtime system to the <span style="font-family: courier;">main</span> method as an array of <span style="font-family: courier;">String</span>s.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Spring Boot application is a command line runnable Java application with a <span style="font-family: courier;">main</span> method. The <a href="https://docs.spring.io/spring-boot/docs/current/api/org/springframework/boot/SpringApplication.html" target="_blank">SpringApplication</a> class is used to bootstrap and launch Spring app from Java's <span style="font-family: courier;">main</span> method. Spring framework promotes POJO development and hence makes it easy to write testable code. A simple <a href="https://junit.org/junit5/docs/current/user-guide/" target="_blank">JUnit</a> test can be written to test the main method.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">This post is about writing a simple <a href="https://junit.org/junit5/docs/current/user-guide/" target="_blank">JUnit</a> test case for Spring Boot application's <span style="font-family: courier;">main</span> method in the context of Docker Compose dependency, and passing required arguments.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 21, </span><span style="font-family: "courier new", courier, monospace;">Spring Boot 3.2.1,</span><span style="font-family: "courier new" , "courier" , monospace;"> maven 3.9.6 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><br /></div><div style="text-align: left;">A typical, simple Spring Boot application class with a <span style="font-family: courier;">main</span> method looks like:</div>
<div class="code">import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
<span style="background-color: #fcff01;">@SpringBootApplication</span>
public class GraalVmApplication {
public static void main(String[] args) {
<span style="background-color: #fcff01;">SpringApplication.run(GraalVmApplication.class, args);</span>
}
}
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">A simple JUnit test case to test the application looks like:</div>
<div class="code">import org.junit.jupiter.api.Test;
public class ApplicationStartsTest {
@Test
void application_starts() {
<span style="background-color: #fcff01;">GraalVmApplication.main(new String[]{});</span>
}
}
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">The test case is simple. It just invokes application's <span style="font-family: courier;">main</span> method which in turn launches the Spring boot application and exits. Note that there are no arguments passed, so it's launched with default profile, which is ok to make sure the application runs. </div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">The Scenario</h3><div style="text-align: left;">Now, let's says that we want to add JPA and Postgres DB support to our application. Also want to try Spring support for <a href="https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.docker-compose" target="_blank">Docker Compose</a> by adding Postgres database service docker container for database. Spring docker support takes care of starting up the Postgres docker container and bringing it down when the app exists. All it requires is to to add <span style="font-family: courier;">spring-boot-docker-compose</span> dependency and <span style="font-family: courier;">compose.yaml</span> or <span style="font-family: courier;">docker-compose.yaml</span> under the root project. Refer to this <a href="https://github.com/gpottepalem/boot-graalvm" target="_blank">Sample Git project</a>.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">With the JPA, <span style="font-family: courier;">spring-boot-docker-compose</span> dependencies and <span style="font-family: courier;">docker-compose.yaml</span> added, the above JUnit test case fails to run with the following exception:</div>
<div class="code">
***************************
APPLICATION FAILED TO START
***************************
Description:
Failed to configure a DataSource: 'url' attribute is not specified and no embedded datasource could be configured.
Reason: <span style="background-color: #fcff01;">Failed to determine a suitable driver class</span>
Action:
Consider the following:
If you want an embedded database (H2, HSQL or Derby), please put it on the classpath.
<span style="background-color: #ffa400;">If you have database settings to be loaded from a particular profile you may need to activate it (no profiles are currently active).</span>
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.731 s <<< FAILURE! -- in com.giri.boot.graalvm.ApplicationStartsTest
[ERROR] com.giri.boot.graalvm.ApplicationStartsTest.applicationStarts -- Time elapsed: 3.705 s <<< ERROR!
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'dataSourceScriptDatabaseInitializer' defined in class path resource [org/springframework/boot/autoconfigure/sql/init/DataSourceInitializationConfiguration.class]: Unsatisfied dependency expressed through method 'dataSourceScriptDatabaseInitializer' parameter 0: Error creating bean with name 'dataSource' defined in class path resource [org/springframework/boot/autoconfigure/jdbc/DataSourceConfiguration$Hikari.class]: Failed to instantiate [com.zaxxer.hikari.HikariDataSource]: Factory method 'dataSource' threw exception with message: <span style="background-color: #fcff01;">Failed to determine a suitable driver class</span>
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:802)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:546)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1334)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1164)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:561)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:521)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:325)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:312)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1232)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:950)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:625)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:762)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:464)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:334)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1358)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1347)
at com.giri.boot.graalvm.GraalVmApplication.main(GraalVmApplication.java:12)
at com.giri.boot.graalvm.ApplicationStartsTest.applicationStarts(ApplicationStartsTest.java:17)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [org/springframework/boot/autoconfigure/jdbc/DataSourceConfiguration$Hikari.class]: Failed to instantiate [com.zaxxer.hikari.HikariDataSource]: Factory method 'dataSource' threw exception with message: <span style="background-color: #fcff01;">Failed to determine a suitable driver class</span>
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:655)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:643)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1334)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1164)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:561)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:521)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:325)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:254)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1443)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1353)
at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:911)
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:789)
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">The reason for the failure is - by default docker compose support is disabled when running tests.</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">The Fix</h3><div style="text-align: left;">The fix is to pass additional arguments like - enable docker compose support for this test, profile (<span style="font-family: courier;">local</span>) for which datasource properties are configured etc. With these, when the unit test is run, docker compose is up, which brings up Postgres container and the application connects to the database through datasource configuration properties specified in <span style="font-family: courier;">application-local.yml</span> file for <span style="font-family: courier;">local</span> profile. With the fix, the test case looks like:</div><div style="text-align: left;"><br /></div>
<div class="code">
public class ApplicationStartsTest {
@Test
void applicationStarts() {
GraalVmApplication.main(
new String[]{
<span style="background-color: #fcff01;">"--spring.profiles.active=local",</span>
<span style="background-color: #fcff01;">"--spring.docker.compose.skip.in-tests=false",</span>
<span style="background-color: #fcff01;">"--spring.docker.compose.file=docker/docker-compose.yaml"</span>
}
);
}
}
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">Now, the test case passes. Below is the output with docker compose highlighted:</div>
<div class="code">
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.giri.boot.graalvm.ApplicationStartsTest
a<span style="background-color: #fcff01;">rg: --spring.profiles.active=local</span>
<span style="background-color: #fcff01;">arg: --spring.docker.compose.skip.in-tests=false</span>
<span style="background-color: #fcff01;">arg: --spring.docker.compose.file=docker/docker-compose.yaml</span>
____ _ ____ ___ __
| __ ) ___ ___ | |_ / ___|_ __ __ _ __ _| \ \ / / __ ___
| _ \ / _ \ / _ \| __| | | _| '__/ _` |/ _` | |\ \ / / '_ ` _ \
| |_) | (_) | (_) | |_ | |_| | | | (_| | (_| | | \ V /| | | | | |
|____/ \___/ \___/ \__| \____|_| \__,_|\__,_|_| \_/ |_| |_| |_|
:: Spring Boot :: 3.2.1
:: Running on Java :: 21
:: Application :: 0.0.1-SNAPSHOT
2023-12-28T20:09:11.059-05:00 INFO 65561 --- [boot-graalvm] [ main] c.giri.boot.graalvm.GraalVmApplication : Starting GraalVmApplication using Java 21 with PID 65561 (started by pottepalemg in /Users/pottepalemg/dev/boot-graalvm)
2023-12-28T20:09:11.065-05:00 INFO 65561 --- [boot-graalvm] [ main] c.giri.boot.graalvm.GraalVmApplication : The following 1 <span style="background-color: #fcff01;">profile is active: "local"</span>
2023-12-28T20:09:11.523-05:00 INFO 65561 --- [boot-graalvm] [ main] .s.b.d.c.l.DockerComposeLifecycleManager : Using Docker Compose file '/Users/pottepalemg/dev/boot-graalvm/docker/docker-compose.yaml'
2023-12-28T20:09:12.547-05:00 INFO 65561 --- [boot-graalvm] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli : <span style="background-color: #fcff01;">Container docker-postgres-1 Creating</span>
2023-12-28T20:09:12.614-05:00 INFO 65561 --- [boot-graalvm] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli : Container docker-postgres-1 Created
2023-12-28T20:09:12.618-05:00 INFO 65561 --- [boot-graalvm] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli : Container docker-postgres-1 Starting
2023-12-28T20:09:13.169-05:00 INFO 65561 --- [boot-graalvm] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli : Container docker-postgres-1 Started
2023-12-28T20:09:13.174-05:00 INFO 65561 --- [boot-graalvm] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli : Container docker-postgres-1 Waiting
2023-12-28T20:09:13.682-05:00 INFO 65561 --- [boot-graalvm] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli : Container docker-postgres-1 Healthy
2023-12-28T20:09:16.211-05:00 INFO 65561 --- [boot-graalvm] [ main] .s.d.r.c.RepositoryConfigurationDelegate : <span style="background-color: #fcff01;">Bootstrapping Spring Data JPA repositories in DEFAULT mode</span>.
2023-12-28T20:09:16.237-05:00 INFO 65561 --- [boot-graalvm] [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 17 ms. Found 0 JPA repository interfaces.
2023-12-28T20:09:17.133-05:00 INFO 65561 --- [boot-graalvm] [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port 8080 (http)
2023-12-28T20:09:17.157-05:00 INFO 65561 --- [boot-graalvm] [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2023-12-28T20:09:17.157-05:00 INFO 65561 --- [boot-graalvm] [ main] o.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/10.1.17]
2023-12-28T20:09:17.233-05:00 INFO 65561 --- [boot-graalvm] [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2023-12-28T20:09:17.234-05:00 INFO 65561 --- [boot-graalvm] [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 2082 ms
2023-12-28T20:09:17.648-05:00 INFO 65561 --- [boot-graalvm] [ main] com.zaxxer.hikari.HikariDataSource : <span style="background-color: #fcff01;">HikariPool-1 - Starting...</span>
2023-12-28T20:09:17.895-05:00 INFO 65561 --- [boot-graalvm] [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@24a99b1c
2023-12-28T20:09:17.897-05:00 INFO 65561 --- [boot-graalvm] [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2023-12-28T20:09:17.970-05:00 INFO 65561 --- [boot-graalvm] [ main] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default]
2023-12-28T20:09:18.102-05:00 INFO 65561 --- [boot-graalvm] [ main] org.hibernate.Version : HHH000412: Hibernate ORM core version 6.4.1.Final
2023-12-28T20:09:18.163-05:00 INFO 65561 --- [boot-graalvm] [ main] o.h.c.internal.RegionFactoryInitiator : HHH000026: Second-level cache disabled
2023-12-28T20:09:18.557-05:00 INFO 65561 --- [boot-graalvm] [ main] o.s.o.j.p.SpringPersistenceUnitInfo : No LoadTimeWeaver setup: ignoring JPA class transformer
2023-12-28T20:09:19.123-05:00 INFO 65561 --- [boot-graalvm] [ main] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000489: No JTA platform available (set 'hibernate.transaction.jta.platform' to enable JTA platform integration)
2023-12-28T20:09:19.129-05:00 INFO 65561 --- [boot-graalvm] [ main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2023-12-28T20:09:19.398-05:00 WARN 65561 --- [boot-graalvm] [ main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
2023-12-28T20:09:20.474-05:00 INFO 65561 --- [boot-graalvm] [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 13 endpoint(s) beneath base path '/actuator'
2023-12-28T20:09:20.559-05:00 INFO 65561 --- [boot-graalvm] [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port 8080 (http) with context path ''
2023-12-28T20:09:20.576-05:00 INFO 65561 --- [boot-graalvm] [ main] c.giri.boot.graalvm.GraalVmApplication : <span style="background-color: #fcff01;">Started GraalVmApplication in 10.156 seconds</span> (process running for 11.266)
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.57 s -- in com.giri.boot.graalvm.ApplicationStartsTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] <span style="background-color: #fcff01;">BUILD SUCCESS</span>
[INFO] ------------------------------------------------------------------------
</div>
<div style="text-align: left;"><br /></div>
<h3 style="text-align: left;"><span>💡 </span>TIPS</h3>
<div style="text-align: left;">It is also helpful to print arguments in the Spring boot application:</div>
<div class="code">
@SpringBootApplication
public class GraalVmApplication {
public static void main(String[] args) {
<span style="background-color: #fcff01;">for (String arg: args) { // let's examine arguments passed</span>
<span style="background-color: #fcff01;">System.out.println("arg: " + arg);</span>
<span style="background-color: #fcff01;">}</span>
SpringApplication.run(GraalVmApplication.class, args);
}
}
</div>
<div style="text-align: left;"><br /></div><h3 style="text-align: left;">Conclusion</h3><div style="text-align: left;">Much of the learning comes, not from working code, but from failing code. If everything works on the first go, there isn't much learned ;)</div><div style="text-align: left;">Keep exploring!</div><div style="text-align: left;">Happy Holidays!!</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">References</h3><div style="text-align: left;"><ul style="text-align: left;"><li><a href="https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.docker-compose" target="_blank">Spring Boot Docker Compose Support</a></li></ul></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-2456844505544987142023-12-28T13:10:00.015-05:002024-01-18T10:45:43.443-05:00Spring Boot, Docker compose on Mac OS Catalina - Gotcha . . .<div style="text-align: left;">Spring Boot 3.0 <a href="https://spring.io/blog/2023/06/21/docker-compose-support-in-spring-boot-3-1/" target="_blank">enhanced docker compose support</a> with which container startup and destroy is taken care by spring boot and is quite convenient.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Anything I try first time, I hit road blocks. It's both good and bad. The good part is - it makes me explore things little deeper, which otherwise is not possible. The bad part is - it is frustrating and needs patience to get it to work.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 21, </span><span style="font-family: "courier new", courier, monospace;">Spring Boot 3.2.0,</span><span style="font-family: "courier new" , "courier" , monospace;"> maven 3.9.6 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div><h3 style="text-align: left;">Scenario</h3><div style="text-align: left;">I wanted to add Postgres db service through docker compose. So, just added docker compose support to an existing simple <a href="https://github.com/gpottepalem/boot-graalvm" target="_blank">Spring Boot 3.2.0 application</a>. All that I had to do was simple. Add the dependency in pom.xml and a docker-compose.yml file in the root directory of the project. This is customizable through the property: <span style="font-family: courier;">spring.docker.compose.file</span>.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><span style="font-family: courier;">pom.xml</span></div>
<div class="code"> ...
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-docker-compose</artifactId>
<scope>runtime</scope>
<optional>true</optional>
</dependency>
...
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;"><span style="font-family: courier;">docker-compose.yaml</span></div>
<div class="code">services:
postgres:
image: 'postgres16:16.0'
environment:
- 'POSTGRES_DB=boot-graalvm'
- 'POSTGRES_USER=postgres'
- 'POSTGRES_PASSWORD=s3cr3t'
ports:
- '5222:5432'
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">Application fails to run with the following error:</div>
<div class="code">
2023-12-27T08:58:48.534-05:00 ERROR 229 --- [ main] o.s.boot.SpringApplication : Application run failed
org.springframework.boot.docker.compose.core.ProcessExitException: 'docker-compose version --format json' failed with exit code 1.
Stdout:
Stderr:
Show version information and quit.
Usage: version [--short]
Options:
--short Shows only Compose's version number.
at org.springframework.boot.docker.compose.core.ProcessRunner.run(ProcessRunner.java:96) ~[spring-boot-docker-compose-3.2.0.jar:3.2.0]
at org.springframework.boot.docker.compose.core.ProcessRunner.run(ProcessRunner.java:74) ~[spring-boot-docker-compose-3.2.0.jar:3.2.0]
at org.springframework.boot.docker.compose.core.DockerCli$DockerCommands.getDockerComposeCommand(DockerCli.java:165) ~[spring-boot-docker-compose-3.2.0.jar:3.2.0]
at org.springframework.boot.docker.compose.core.DockerCli$DockerCommands.<init>(DockerCli.java:130) ~[spring-boot-docker-compose-3.2.0.jar:3.2.0]
at org.springframework.boot.docker.compose.core.DockerCli.lambda$new$0(DockerCli.java:65) ~[spring-boot-docker-compose-3.2.0.jar:3.2.0]
at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1228) ~[na:na]
at org.springframework.boot.docker.compose.core.DockerCli.<init>(DockerCli.java:64) ~[spring-boot-docker-compose-3.2.0.jar:3.2.0]
at org.springframework.boot.docker.compose.core.DockerCompose.get(DockerCompose.java:92) ~[spring-boot-docker-compose-3.2.0.jar:3.2.0]
at org.springframework.boot.docker.compose.lifecycle.DockerComposeLifecycleManager.getDockerCompose(DockerComposeLifecycleManager.java:149) ~[spring-boot-docker-compose-3.2.0.jar:3.2.0]
at org.springframework.boot.docker.compose.lifecycle.DockerComposeLifecycleManager.start(DockerComposeLifecycleManager.java:110) ~[spring-boot-docker-compose-3.2.0.jar:3.2.0]
at org.springframework.boot.docker.compose.lifecycle.DockerComposeListener.onApplicationEvent(DockerComposeListener.java:53) ~[spring-boot-docker-compose-3.2.0.jar:3.2.0]
at org.springframework.boot.docker.compose.lifecycle.DockerComposeListener.onApplicationEvent(DockerComposeListener.java:35) ~[spring-boot-docker-compose-3.2.0.jar:3.2.0]
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:178) ~[spring-context-6.1.1.jar:6.1.1]
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:171) ~[spring-context-6.1.1.jar:6.1.1]
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:149) ~[spring-context-6.1.1.jar:6.1.1]
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:137) ~[spring-context-6.1.1.jar:6.1.1]
at org.springframework.boot.context.event.EventPublishingRunListener.multicastInitialEvent(EventPublishingRunListener.java:136) ~[spring-boot-3.2.0.jar:3.2.0]
at org.springframework.boot.context.event.EventPublishingRunListener.contextLoaded(EventPublishingRunListener.java:98) ~[spring-boot-3.2.0.jar:3.2.0]
at org.springframework.boot.SpringApplicationRunListeners.lambda$contextLoaded$4(SpringApplicationRunListeners.java:72) ~[spring-boot-3.2.0.jar:3.2.0]
at java.base/java.lang.Iterable.forEach(Iterable.java:75) ~[na:na]
at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118) ~[spring-boot-3.2.0.jar:3.2.0]
at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112) ~[spring-boot-3.2.0.jar:3.2.0]
at org.springframework.boot.SpringApplicationRunListeners.contextLoaded(SpringApplicationRunListeners.java:72) ~[spring-boot-3.2.0.jar:3.2.0]
at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:431) ~[spring-boot-3.2.0.jar:3.2.0]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:322) ~[spring-boot-3.2.0.jar:3.2.0]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1342) ~[spring-boot-3.2.0.jar:3.2.0]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1331) ~[spring-boot-3.2.0.jar:3.2.0]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
</init></init></div>
<div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">The Fix</h3><div style="text-align: left;">The issue was the older version of Docker Desktop 2.5.0.1 running on my laptop. The menu item <span style="font-family: courier;">Check for Updates...</span> wouldn't even work. It seemed like I was stuck with very old version of Docker Desktop. The fix was to upgrade to a version that works. Yes, you have to find out the version that works for Mac OS Catalina. I tried and learned the fact that Docker Desktop latest version (4.26.1) doesn't run on Mac OS Catalina and <a href="https://github.com/docker/for-mac/issues/6671" target="_blank">Docker Desktop silently removed the support for Mac OS Catalina in 4.16.0</a>. So the only version to upgrade for Mac OS Catalina is: 4.15.0.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Jeez, what a mess of higher elevated software development. It's just ridiculous!</div><div style="text-align: left;"><br /></div><div style="text-align: left;">There is no easy way even to find the version 4.15.0 download link. Finally found the links for both the versions: 2.5.0.1 that I had and 4.15.0 that I want to upgrade to. </div><div style="text-align: left;"><br /></div><div style="text-align: left;">To downgrade to 2.5.0.1, here is the link: <a href="https://desktop.docker.com/mac/stable/49550/Docker.dmg" target="_blank">https://desktop.docker.com/mac/stable/49550/Docker.dmg</a></div><div style="text-align: left;">To upgrade to 4.15.0, here is the link: <a href="https://desktop.docker.com/mac/main/amd64/93002/Docker.dmg" target="_blank">https://desktop.docker.com/mac/main/amd64/93002/Docker.dmg</a></div><div style="text-align: left;"><br /></div>
<h3 style="text-align: left;"><span> </span>TIPS</h3>
<div style="text-align: left;"><span style="font-size: large;">1.</span> Spring Boot docker compose out-of-the-box looks for docker compose file named <span style="font-family: courier;">compose.yaml</span> or <span style="font-family: courier;">docker-compose.yaml</span> in the project root directory. If you want to place the file anywhere else under the project root folder, for instance like a <span style="font-family: courier;">docker</span> sub-dir under project root, here is how the customization property should look like in the appropriate <span style="font-family: courier;">application.yml</span> or <span style="font-family: courier;">application-<env>.yml</span> related <span style="font-family: courier;">yml</span> file:</div><div style="text-align: left;"><br /></div>
<div class="code">spring:
docker:
compose:
file: docker/docker-compose.yaml
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;"><span style="font-size: large;">2.</span> Docker compose is enabled by default. When the dependent jar is found, it looks for <span style="font-family: courier;">compose.yaml</span>/<span style="font-family: courier;">docker-compose.yaml</span>. It can be disabled by setting explicitly the property <span style="font-family: courier;">spring.docker.enabled</span> to <span style="font-family: courier;">false</span>. Typically, docker compose is good for local development. If there are multiple profiles (environments) that the application is configured to run in, the <span style="font-family: courier;">application.properties</span> or <span style="font-family: courier;">application.yml</span> file should have docker compose disabled and it can be enabled in the appropriate profile that it's used, e.g <span style="font-family: courier;">local</span>.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><span style="font-size: large;">3.</span> In the <span style="font-family: courier;">compose.yaml</span> or <span style="font-family: courier;">docker-compose.yaml</span> file, note that we specify the HOST_POST and CONTAINER_PORT for PostgreSQL in the format HOST_PORT:CONTAINER_PORT. The container port is 5432 (postgres default). The host port also can be 5432. But in case if you want to have multiple postgres databases managed by Docker for multiple applications/versions, you need to use different HOST_PORT for each image for both to be up and running at the same time. <i><b>The HOST_PORT is the port you use to connect to the database.</b></i></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><span style="font-size: large;">4.</span> Give a unique name for each postgres image if you need to have multiple versions of postgres instances managed by Docker. Otherwise, you will end up having just one. You can use version number in the image name to distinguish.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><span style="font-size: large;">5.</span> Check docker and docker-compose versions:</div><div style="text-align: left;"><span style="font-family: courier;">$ docker -v or docker --version</span></div><div style="text-align: left;"><span style="font-family: courier;">$ docker compose version</span></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><span style="font-size: large;">6.</span> If the application fails with the following exception:</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><span style="font-family: courier;">org.springframework.boot.docker.compose.lifecycle.ReadinessTimeoutException: Readiness timeout of PT2M reached while waiting for services</span></div><div style="text-align: left;"><br /></div><div style="text-align: left;">Add the following to <span style="font-family: courier;">docker-compose.yaml</span> file </div>
<div class="code">services:
postgres:
image: 'postgres16:16.0'
environment:
- 'POSTGRES_DB=boot-graalvm'
- 'POSTGRES_USER=postgres'
- 'POSTGRES_PASSWORD=s3cr3t'
ports:
- '5222:5432'
<span style="background-color: #fcff01;">labels:</span>
<span style="background-color: #fcff01;">org.springframework.boot.readiness-check.tcp.disable: true
</span></div>
<div style="text-align: left;"><br /></div><h3 style="text-align: left;">References</h3><div style="text-align: left;"><ul style="text-align: left;"><li><a href="https://spring.io/blog/2023/06/21/docker-compose-support-in-spring-boot-3-1/" target="_blank">Spring Boot Docker Compose Support 3.1</a></li><li><a href="https://howtodoinjava.com/spring-boot/spring-boot-docker-compose/" target="_blank">Using Docker Compose in Spring Boot 3</a></li><li><a href="https://github.com/docker/for-mac/issues/6671" target="_blank">Docker Desktop removed Mac OS Catalina support in 4.16.0</a></li><li><a href="https://blog.carbonfive.com/running-multiple-versions-of-postgres-with-docker-compose-for-local-development/" target="_blank">Running Multiple Versions of Postgres with Docker Compose for Local Development</a></li><li><a href="https://docs.docker.com/compose/networking/" target="_blank">Networking in Compose</a></li></ul></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-26652625418560749332023-12-21T11:41:00.007-05:002024-01-09T11:02:26.253-05:00Looking into Testcontainers . . .<div style="text-align: left;">Everything is easy and stays easy only up until experienced. In software development the "<b>Aha-moment</b><i>"</i> of someone would be a "<b>Huh-moment</b><i>"</i> of some other. We hear the phrase <i>"</i><b>It worked for me</b><i>"</i> almost daily that whispers aloud "<b>It didn't work for someone else</b>".</div><div style="text-align: left;"><br /></div><div style="text-align: left;">This blog post is not about <a href="https://testcontainers.com/" target="_blank">Testcontainers</a> but an issue that took awhile for me to find a workaround for. Testcontainers is a framework that provides lightweight throwaway instance for anything that run in a docker container. It's just appropriate for using in integration tests that require resources like Database server, Redis instance etc.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">With the enhanced support in Spring Boot 3, <a href="https://testcontainers.com/" target="_blank">Testcontainers</a> got much simpler and better. Much of this enhanced support comes in as a new annotation: <span style="font-family: courier;"><a href="https://github.com/spring-projects/spring-boot/blob/a42f549612fc8bacb60461f4f06b57e32f0a9e32/spring-boot-project/spring-boot-testcontainers/src/main/java/org/springframework/boot/testcontainers/service/connection/ServiceConnection.java#L42" target="_blank">@ServieConnection</a></span>. A bean method annotated with this annotation is enough and Spring Boot takes care of the rest - managing the life cycle of the container like staring at the application startup and stopping, establishing connection to the service running in the container etc. No other configuration like datasource connection is needed. Check this detailed <a href="https://spring.io/blog/2023/06/23/improved-testcontainers-support-in-spring-boot-3-1/" target="_blank">Spring team blogpost</a> on this.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The enthusiasm of "<b>Lets give it a try</b>", I got stuck for a couple of hours with a problem that I couldn't find a solution by tirelessly by trying many possible including googling, reading and what not.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 20, </span><span style="font-family: "courier new", courier, monospace;">Spring Boot 3.2.1,</span><span style="font-family: "courier new" , "courier" , monospace;"> maven 3.9.6 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><h3>The Scenario</h3><div>The scenario is simple. I wanted to quickly try the newly <a href="https://spring.io/blog/2023/06/23/improved-testcontainers-support-in-spring-boot-3-1/" target="_blank">enhanced and improved Spring Boot support</a> for <a href="https://java.testcontainers.org/modules/databases/postgres/" target="_blank">Testcontainers with PostgreSQL database</a>.</div><div><br /></div><div>However, I got stuck with getting the container started successfully for PostgreSQL database image. Started with the latest <a href="https://hub.docker.com/_/postgres" target="_blank">Postgres image tag</a>, tried specific older tags including 16, 16.1. Downgraded testcontainer library versions from the latest 1.19.3 to 1.18.3 and few other as I hit for people who faced similar issue said it worked with other versions.</div><div><br /></div><div>For everything that I tried, the issue was at least consistent with the following stacktrace:</div><div><br /></div>
<div class="code">
2023-12-20T12:07:56.443-05:00 INFO 60313 --- [boot-graalvm] [ main] tc.postgres:16 : <span style="background-color: #fcff01;">Container postgres:16 is starting:</span> 995cb534ef6982062dcc98d9841cdf7962c0e1d37c809f3e479d06667cc567f7
2023-12-20T12:08:56.871-05:00 ERROR 60313 --- [boot-graalvm] [ main] tc.postgres:16 : Could not start container
java.lang.IllegalStateException: Failed to load ApplicationContext for [WebMergedContextConfiguration@2df9b4f3 testClass = com.giri.boot.graalvm.GraalVmApplicationIT, locations = [], classes = [com.giri.boot.graalvm.GraalVmApplication], contextInitializerClasses = [], activeProfiles = ["test"], propertySourceDescriptors = [], propertySourceProperties = ["org.springframework.boot.test.context.SpringBootTestContextBootstrapper=true"], contextCustomizers = [[ImportsContextCustomizer@6112390a key = [com.giri.boot.graalvm.config.TestContainersConfiguration]], org.springframework.boot.test.context.filter.ExcludeFilterContextCustomizer@bdc8014, org.springframework.boot.test.json.DuplicateJsonObjectContextCustomizerFactory$DuplicateJsonObjectContextCustomizer@13047d7d, org.springframework.boot.test.mock.mockito.MockitoContextCustomizer@0, org.springframework.boot.test.web.client.TestRestTemplateContextCustomizer@675ffd1d, org.springframework.boot.test.autoconfigure.actuate.observability.ObservabilityContextCustomizerFactory$DisableObservabilityContextCustomizer@1f, org.springframework.boot.test.autoconfigure.properties.PropertyMappingContextCustomizer@0, org.springframework.boot.test.autoconfigure.web.servlet.WebDriverContextCustomizer@3104351d, org.spockframework.spring.mock.SpockContextCustomizer@0, org.springframework.boot.testcontainers.service.connection.ServiceConnectionContextCustomizer@0, org.springframework.boot.test.context.SpringBootTestAnnotation@156baa3c], resourceBasePath = "src/main/webapp", contextLoader = org.springframework.boot.test.context.SpringBootContextLoader, parent = null]
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:180)
at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:130)
at org.springframework.test.context.web.ServletTestExecutionListener.setUpRequestContextIfNecessary(ServletTestExecutionListener.java:191)
at org.springframework.test.context.web.ServletTestExecutionListener.prepareTestInstance(ServletTestExecutionListener.java:130)
at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:247)
at org.springframework.test.context.junit.jupiter.SpringExtension.postProcessTestInstance(SpringExtension.java:163)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:310)
at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:735)
at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734)
at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
at java.base/java.util.Optional.orElseGet(Optional.java:364)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'graalVmApplication': Error creating bean with name 'postgreSQLContainer' defined in class path resource [com/giri/boot/graalvm/config/TestContainersConfiguration.class]: Container startup failed for image postgres:16
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:608)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:523)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:325)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:973)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:946)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:616)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:753)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:455)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:323)
at org.springframework.boot.test.context.SpringBootContextLoader.lambda$loadContext$3(SpringBootContextLoader.java:137)
at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:58)
at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:46)
at org.springframework.boot.SpringApplication.withHook(SpringApplication.java:1442)
at org.springframework.boot.test.context.SpringBootContextLoader$ContextLoaderHook.run(SpringBootContextLoader.java:552)
at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:137)
at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:108)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:225)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:152)
... 17 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'postgreSQLContainer' defined in class path resource [com/giri/boot/graalvm/config/TestContainersConfiguration.class]: Container startup failed for image postgres:16
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:608)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:523)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:325)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.boot.testcontainers.lifecycle.TestcontainersLifecycleBeanPostProcessor.getBeans(TestcontainersLifecycleBeanPostProcessor.java:128)
at org.springframework.boot.testcontainers.lifecycle.TestcontainersLifecycleBeanPostProcessor.initializeContainers(TestcontainersLifecycleBeanPostProcessor.java:119)
at org.springframework.boot.testcontainers.lifecycle.TestcontainersLifecycleBeanPostProcessor.postProcessAfterInitialization(TestcontainersLifecycleBeanPostProcessor.java:79)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsAfterInitialization(AbstractAutowireCapableBeanFactory.java:437)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1778)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:601)
... 37 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Container startup failed for image postgres:16
at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:362)
at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:333)
at java.base/java.lang.Iterable.forEach(Iterable.java:75)
at org.springframework.boot.testcontainers.lifecycle.TestcontainersStartup$1.start(TestcontainersStartup.java:43)
at org.springframework.boot.testcontainers.lifecycle.TestcontainersLifecycleBeanPostProcessor.start(TestcontainersLifecycleBeanPostProcessor.java:114)
at org.springframework.boot.testcontainers.lifecycle.TestcontainersLifecycleBeanPostProcessor.initializeStartables(TestcontainersLifecycleBeanPostProcessor.java:103)
at org.springframework.boot.testcontainers.lifecycle.TestcontainersLifecycleBeanPostProcessor.postProcessAfterInitialization(TestcontainersLifecycleBeanPostProcessor.java:83)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsAfterInitialization(AbstractAutowireCapableBeanFactory.java:437)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1778)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:601)
... 48 more
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:88)
at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:347)
... 57 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:566)
at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:357)
at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
... 58 more
Caused by: java.lang.IllegalStateException: Wait strategy failed. Container exited with code 1
at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:536)
... 60 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Timed out waiting for log output matching '.*database system is ready to accept connections.*\s'
at org.testcontainers.containers.wait.strategy.LogMessageWaitStrategy.waitUntilReady(LogMessageWaitStrategy.java:47)
at org.testcontainers.containers.wait.strategy.AbstractWaitStrategy.waitUntilReady(AbstractWaitStrategy.java:52)
at org.testcontainers.containers.PostgreSQLContainer.waitUntilContainerStarted(PostgreSQLContainer.java:147)
at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:503)
... 60 more
</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Finally the logs from the container Docker dashboard for postgres:16 container that failed with the message was the last resort looked into:</div><p class="jss1839" style="background-color: #253138; box-sizing: inherit; color: #aed9fa; font-family: "Open Sans", "Helvetica Neue", sans-serif; font-size: 12px; margin: 0px;"><code class="jss1838" style="box-sizing: inherit; overflow-wrap: break-word;">popen failure: Cannot allocate memory</code></p><p class="jss1839" style="background-color: #253138; box-sizing: inherit; color: #aed9fa; font-family: "Open Sans", "Helvetica Neue", sans-serif; font-size: 12px; margin: 0px;"><span face=""Open Sans", "Helvetica Neue", sans-serif" style="color: #aed9fa;">initdb: error: program "postgres" is needed by initdb but was not found in the same directory as "/usr/lib/postgresql/16/bin/initdb"</span><span style="background-color: transparent;">, </span></p><div style="text-align: left;"><br /></div><div style="text-align: left;">That wasn't that helpful. By googling for the above error message, I landed <a href="https://codereviewvideos.com/postgres-16-docker-workaround-program-postgres-is-needed-by-initdb/" target="_blank">at a blog post</a>. The solution that was described there worked for me as well.</div>
<div style="text-align: left;"><br /></div><h3 style="text-align: left;">The solution</h3><div style="text-align: left;">Use <a href="https://hub.docker.com/_/postgres" target="_blank">Alpine Postgres</a>. Diving further deeper is not worth my time at this time. I had to move on with what worked for me ;). Couldn't resist that saying, "<b>it (also) worked for me</b>".</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><div>The following is the code snippet:</div>
<div class="code">
@TestConfiguration(proxyBeanMethods = false)
@Slf4j
public class TestContainersConfiguration {
private static final String POSTGRES_IMAGE_TAG = "postgres:16-alpine"; // Issue: postgres:latest or postgres:16
@Bean
@ServiceConnection
PostgreSQLContainer<?> postgreSQLContainer() {
log.info("PostgreSQLContainer bean...");
return new PostgreSQLContainer<>(DockerImageName.parse(POSTGRES_IMAGE_TAG))
.withDatabaseName("boot-graalvm")
.withUsername("postgres")
.withPassword("s3cr3t");
}
}
</div><br />
</div><div style="text-align: left;"><a href="https://github.com/gpottepalem/boot-graalvm" target="_blank">Github Repo</a></div><div style="text-align: left;"><br /></div></div><h3 style="text-align: left;">What is Alpine Linux?</h3><div style="text-align: left;"><a href="https://alpinelinux.org/about/" target="_blank">Alpine Linux</a> is a lightweight Linux distribution with the focus on SSS distribution - Small.Simple.Secure. It is famous for its small size with the fastest boot time, hence heavily used in containers.</div><div style="text-align: left;"><br /></div>
<h3 style="text-align: left;"><span>💡 </span>TIPS</h3>
<div>Check your docker desktop version. If you have old version like what I have: 2.5.0.1, it is time to upgrade to newer version that works. For Mac OS Catalina, the newer version that works is 4.15.0.</div><div>Check <a href="https://giri-tech.blogspot.com/2023/12/spring-boot-docker-compose-on-mac-os.html" target="_blank">this blog post</a> on this.</div><div><br /></div><h3 style="text-align: left;">Conclusion</h3><div style="text-align: left;">The elevated development experience with underpinned layers of languages, frameworks, and systems increase the depth of things to know and understand for a Software Engineer, keeping the self-training capabilities and building knowledge, the real and not artificial, a daily challenge.</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">References</h3><div><ul style="text-align: left;"><li><a href="https://codereviewvideos.com/postgres-16-docker-workaround-program-postgres-is-needed-by-initdb/" target="_blank">Workaround for program “postgres” is needed by initdb</a></li><li><a href="https://java.testcontainers.org/" target="_blank">Testcontainers for Java</a></li><li><a href="https://www.infoq.com/news/2023/07/enhanced-testcontainers-support/" target="_blank">Enhanced Testcontainers support in Spring Boot 3.1</a></li><li><a href="https://hub.docker.com/_/postgres" target="_blank">Postgres docker supported tags</a></li></ul></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-78591367409341271672023-12-12T16:57:00.002-05:002024-01-09T11:01:28.951-05:00Spot-check Spotless . . .<div style="text-align: left;">Sometimes, overwhelming details are underwhelming. I happened to quickly explore <a href="https://github.com/diffplug/spotless" target="_blank">Spotless</a> for code formatting and checks. <a href="https://github.com/diffplug/spotless" target="_blank">Spotless</a> is the plugin to spot check. It's good that the <a href="https://github.com/diffplug/spotless#readme" target="_blank">README of Spotless</a> upfront has separated Gradle and Maven documentation. I was looking to add <a href="https://maven.apache.org/" target="_blank">maven</a> support for a new Java <a href="https://spring.io/projects/spring-boot" target="_blank">Spring Boot</a> 3.0 <a href="https://spring.io/projects/spring-graphql" target="_blank">GraphQL</a> project. But the <a href="https://github.com/diffplug/spotless/blob/main/plugin-maven/README.md" target="_blank">Spotless Maven README</a> documentation was overwhelming.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Quickly read through for few minutes to get an essence of it. Though there are so many details, what I wanted to get done seemed simple. Of course, practicality is always different. That's where experience is gained or comes from ;)</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 20, </span><span style="font-family: "courier new", courier, monospace;">Spring Boot 3.2.0,</span><span style="font-family: "courier new" , "courier" , monospace;"> maven 3.9.6 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div><div style="text-align: left;">I quickly applied it to the project, and immediately ran into a couple of Gotchas. It took me couple of hours to explore and set it up finding issues and fixing settings. This post is just about those issues that I ran into.</div><div style="text-align: left;"><br /></div><h2 style="text-align: left;">Use Spaces, not Tabs in code</h2><div style="text-align: left;">I prefer SPACES to TABS in code. It is good practice to use SPACE instead of TAB in code formatting. This cannot be a personal preference. But most of IDEs still come with TAB as default setting and many developers don't even pay attention to it. In code reviews, the formatting goes off due to SPACESs vs. TABs and is always annoying.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">I use IntelliJ IDEA for my development. Whenever I install IntelliJ, the very first thing I would setup is to use spaces instead of tabs (Go to <span style="font-family: courier;">Preferences > Editor > Code Style > Java</span>, and uncheck <span style="font-family: courier;">Use tab character</span>). With that all code that I write will only use spaces and not tabs. The next thing is to setup to show whitespaces (Go to <span style="font-family: courier;">Preferences > Editor > Appearance</span>, and check <span style="font-family: courier;">Show whitespaces</span>), to easily distinguish Tabs and Spaces by this setting on.</div><div style="text-align: left;"><h3>Gotcha-1</h3></div><div style="text-align: left;">The <a href="https://github.com/diffplug/spotless/blob/main/plugin-maven/README.md" target="_blank">Spotless maven plugin documentation</a> describes many details. The setting I wanted for code check was found at a couple of places in there under <span style="font-family: courier;"><indent></span> tag. At only one place the setting <span style="font-family: courier;"><spaces>true</spaces></span> is specified. Though I tried few different things like <span style="font-family: courier;">false</span> for <span style="font-family: courier;"><tabs></span>, and <span style="font-family: courier;">true</span> for <span style="font-family: courier;"><spaces></span> etc., the <span style="font-family: courier;">spotless:check</span> maven goal was still suggesting to change code with spaces to tabs.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">We also have to use <a href="https://github.com/google/styleguide/tree/gh-pages" target="_blank">google code style settings</a>: <span style="font-family: courier;">eclipse-formatter.xml</span>, that I took from the other project to use. This is described under Java > eclipse jdt in the spotless documentation. It's a google code style Java settings guide in xml format ;)</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The setting id that is used for what I wanted was like: <span style="font-family: courier;"><setting id="org.eclipse.jdt.core.formatter.tabulation.char" value="tab"/></span>. Changing that <span style="font-family: courier;">"tab"</span> to <span style="font-family: courier;">"space"</span> made spotless happy with spaces in code. ;)</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><h3>Gotcha-2</h3><div><a href="https://github.com/diffplug/spotless/blob/main/plugin-maven/README.md#binding-to-maven-phase" target="_blank">Binding to maven phase</a> : I should have read it carefully. At the minimum the <span style="font-family: courier;"><executions><execution><goal>check</goal></execution></execution></span> is required. Without this <span style="font-family: courier;">./mvnw spotless:check</span> works, but <span style="font-family: courier;">./mvnw clean install</span> which also runs <span style="font-family: courier;">verify</span> will not run spotless check. So, for spotless check goal to be bound to maven verify goal, the above setting is required.</div><div><br /></div><div>A sample spotless settings looks like this:</div><div><br /></div>
<div class="code"> ...
<plugins>
<!-- spotless https://github.com/diffplug/spotless -->
<plugin>
<groupId>com.diffplug.spotless</groupId>
<artifactId>spotless-maven-plugin</artifactId>
<version>${spotless-plugin.version}</version>
<executions>
<execution>
<goals>
<goal><span style="background-color: #fcff01;">check</span></goal>
</goals>
</execution>
</executions>
<configuration>
<java>
<endWithNewline />
<trimTrailingWhitespace />
<cleanthat>
<version>2.18</version>
<mutators>
<mutator>AvoidInlineConditionals</mutator>
<mutator>LiteralsFirstInComparisons</mutator>
<mutator>UnnecessaryImport</mutator>
<mutator>UnnecessaryModifier</mutator>
<mutator>UseUnderscoresInNumericLiterals</mutator>
<mutator>UseDiamondOperator</mutator>
</mutators>
</cleanthat>
<eclipse>
<version>4.26</version>
<file><span style="background-color: #fcff01;">${project.basedir}/eclipse-formatter.xml</span></file>
</eclipse>
</java>
</configuration>
</plugin>
...
</plugins>
</div>
<div><br /></div>
<h3 style="text-align: left;"><span>💡 </span>TIPS</h3>
<div><b>Spotless handy maven goals</b></div><div>Check: <span style="font-family: courier;">./mvnw spotless:check</span></div><div>Apply: <span style="font-family: courier;">./mvnw spotless:apply</span></div><div><span style="font-family: courier;"><br /></span></div><h3 style="text-align: left;">Conclusion</h3><div>Any new exploration in Maven world never goes smooth for me. There are always bumps along the way. I do not like copy and paste at all. After all, if everything works in the first place when quick-copy-paste-tool is used, there is not much left to learn ;) With the Generative AI getting trained to write code, learning will soon become a rare thing!</div><div><br /></div><h3 style="text-align: left;">References</h3><div><ul style="text-align: left;"><li><a href="https://github.com/diffplug/spotless" target="_blank">Spotless</a></li><li><a href="https://github.com/diffplug/spotless/blob/main/plugin-maven/README.md" target="_blank">Spotless Maven Plugin</a></li><li><a href="https://github.com/diffplug/spotless/blob/main/plugin-maven/README.md#binding-to-maven-phase" target="_blank">Binding Spotless goal to Maven Phase</a></li><li><a href="https://github.com/diffplug/spotless/blob/main/plugin-maven/README.md#eclipse-jdt" target="_blank">Eclipse jdt</a></li><li><a href="https://github.com/google/styleguide/tree/gh-pages" target="_blank">Google style guides</a></li><li><a href="https://github.com/solven-eu/cleanthat/blob/master/MUTATORS.generated.MD" target="_blank">Cleanthat Mutators</a> </li></ul></div></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-62953584267314492562023-11-30T17:15:00.001-05:002023-11-30T17:20:20.408-05:00Docker maven plugin - Spring Boot, Redis : Gotcha<div style="text-align: left;">The <a href="https://github.com/fabric8io/docker-maven-plugin" target="_blank">docker-maven-plugin</a> comes in handy for managing Docker images and containers in integration tests. It can be used to build images or run. Multiple images can be configured to be built or run depending on the need of your application. This post focuses on run aspect of this plugin; specifically running Redis image in Docker container and a potential issue that one might run into.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 20, </span><span style="font-family: "courier new", courier, monospace;">Spring Boot 3.1.5,</span><span style="font-family: "courier new" , "courier" , monospace;"> maven 3.8.6 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">The Scenario</h3><div style="text-align: left;">After going through some painful hoops, I upgraded a Spring Boot application from 2.6.3 to 3.1.5 with the two-step recommended approach 2.6.x to 2.7.x and then to 3.1.x. The application uses Redis for caching needs. All worked well at the end. The app was built through concourse CI/CD pipeline and successfully deployed as Kubernetes workload to <span style="font-family: courier;">int</span> and <span style="font-family: courier;">cert</span> environments and has been successfully running for few weeks.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">I started to work on fixing some critical and high <a href="https://snyk.io/" target="_blank">Snyk</a> reported open-source dependency vulnerabilities. Suddenly, integration tests around Redis started to fail with the following exception:</div><div style="text-align: left;"><br /></div>
<div class="code">[ERROR] com.hmhco.api.assessmentservice.service.AssessmentServiceCacheIT.testCache -- Time elapsed: 0.478 s <<< ERROR!
org.springframework.data.redis.RedisConnectionFailureException: Unable to connect to Redis
at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$ExceptionTranslatingConnectionProvider.translateException(LettuceConnectionFactory.java:1604)
at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$ExceptionTranslatingConnectionProvider.getConnection(LettuceConnectionFactory.java:1535)
at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$SharedConnection.getNativeConnection(LettuceConnectionFactory.java:1360)
at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$SharedConnection.getConnection(LettuceConnectionFactory.java:1343)
at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getSharedConnection(LettuceConnectionFactory.java:1061)
at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getConnection(LettuceConnectionFactory.java:400)
at org.springframework.data.redis.cache.DefaultRedisCacheWriter.execute(DefaultRedisCacheWriter.java:272)
at org.springframework.data.redis.cache.DefaultRedisCacheWriter.clean(DefaultRedisCacheWriter.java:189)
at org.springframework.data.redis.cache.RedisCache.clear(RedisCache.java:220)
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">The above exception stack-trace didn't give much clue. After looking into few things including redis test configurations, dependencies including transitive dependencies etc. it was still puzzling as it was actually working few weeks ago, and all of sudden only locally the integration test-cases around <span style="font-family: courier;">redis</span> started to fail.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The <span style="font-family: courier;">docker-maven-plugin</span> actually logs that <span style="font-family: courier;">redis:latest</span> started, before running integration tests like below:</div><div style="text-align: left;"><br /></div>
<div class="code">...
[INFO] DOCKER> [redis:latest] "redis-test": Start container 395afa51913b
...
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">The last thing I looked at was the docker containers:</div><div style="text-align: left;"><br /></div>
<div class="code">$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
395afa51913b redis "docker-entrypoint.s…" About a minute ago Exited (1) About a minute ago redis-1
b03c67ae628e postgres:12.4 "docker-entrypoint.s…" About a minute ago Up About a minute 0.0.0.0:5433->5432/tcp postgres-1
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">That showed like the container started but exited for some reason. I also looked at the for Docker Dashboard for any additional details.</div><div style="text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMnGiaJ1hA2wfzESYPAGENs_g482tH1pTE97_wlHOnjUDUBuX-nhtD1oKFaG5sLfJembJ2ENvDg9_E7n69GGZB5YTWI0ct66HMbvLK_rijdH7TpY62aw03O-VY0s58uFKQL8MK8eDLVtSIxmyCTmoW0L0Bv9iU962TNL2bJt_BsqdOtTMQYPzu/s2492/Screen%20Shot%202023-11-29%20at%205.56.47%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1430" data-original-width="2492" height="368" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMnGiaJ1hA2wfzESYPAGENs_g482tH1pTE97_wlHOnjUDUBuX-nhtD1oKFaG5sLfJembJ2ENvDg9_E7n69GGZB5YTWI0ct66HMbvLK_rijdH7TpY62aw03O-VY0s58uFKQL8MK8eDLVtSIxmyCTmoW0L0Bv9iU962TNL2bJt_BsqdOtTMQYPzu/w640-h368/Screen%20Shot%202023-11-29%20at%205.56.47%20PM.png" width="640" /></a></div><br /><div style="text-align: left;">Clicking redis-1 container that exited shows logs and the reason: <span style="font-family: courier;"># Fatal: Can't initialize Background Jobs. Error message: Operation not permitted</span></div><div style="text-align: left;">Also gives Redis version detail: <span style="font-family: courier;">7.2.3</span></div><div style="text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlVMYF8c0WhiggBT7XrHXix8dDk0cMtsxrUFQxBRJL7IqIGsXdTSgS-UWGC93UUrxIWrw-dXKm3C0UR8APjX6gls4PuXooJ_1OvEIbto667Y4NR_Vb4eDIL7cLq7Vo5nq8Kp5_rPMPDDvVV6YyQfFqsXpIPvub_fgM0MAJJwmAIWn6ZqkJSApj/s2500/Screen%20Shot%202023-11-29%20at%205.57.04%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1438" data-original-width="2500" height="368" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlVMYF8c0WhiggBT7XrHXix8dDk0cMtsxrUFQxBRJL7IqIGsXdTSgS-UWGC93UUrxIWrw-dXKm3C0UR8APjX6gls4PuXooJ_1OvEIbto667Y4NR_Vb4eDIL7cLq7Vo5nq8Kp5_rPMPDDvVV6YyQfFqsXpIPvub_fgM0MAJJwmAIWn6ZqkJSApj/w640-h368/Screen%20Shot%202023-11-29%20at%205.57.04%20PM.png" width="640" /></a></div><br /><div style="text-align: left;">That's the issue. Redis started but exited and hence the test failed with the exception - <span style="font-family: courier;">Unable to connect to Redis</span> </div><div style="text-align: left;"><br /></div><div style="text-align: left;">After googling about that initialization issue (<span style="font-family: courier;"># Fatal: Can't initialize Background Jobs. Error message: Operation not permitted</span>), people recommended to try few specific Redis versions. After some trail and error, I had luck with version 6.2.6.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The logs can also be fetched from command-line by executing for the CONTAINER ID (395afa51913b ) or the NAME (redis-1):</div>
<div class="code">$ docker logs 395afa51913b</div>
<div style="text-align: left;"><br /></div><h3 style="text-align: left;">The Fix</h3><div style="text-align: left;">In the <span style="font-family: courier;">docker-maven-plugin</span> configuration, the image tag can be specified for specific version 6.2.6 as shown below:</div><div style="text-align: left;"><br /></div>
<div class="code">
<properties>
<redis.test.port>6381</redis.test.port>
</properties>
...
<plugin>
<groupId>io.fabric8</groupId>
<artifactId>docker-maven-plugin</artifactId>
<version>0.43.4</version>
<executions>
<execution>
<id>start</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
</execution>
<execution>
<id>stop</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
<configuration>
<images>
<image>
<build>
<cleanup>true</cleanup>
<tags>
<tag>latest</tag>
</tags>
</build>
<external>
<type>properties</type>
<prefix>postgres.docker</prefix>
</external>
</image>
<image>
<name>redis<span style="background-color: #fcff01;">:6.2.6</span></name>
<alias>redis-test</alias>
<run>
<ports>
<port>${redis.test.port}:6379</port>
</ports>
</run>
</image>
</images>
</configuration>
</plugin>
</div>
<div style="text-align: left;"><br /></div><h3 style="text-align: left;">TIP</h3><div style="text-align: left;"><a href="https://testcontainers.com/modules/redis/" target="_blank">Testcontainers</a> is another choice to look into for integration tests.</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">GOTCHA</h3><div>It's Gotcha in a Gotcha post ;) I always run into these kinds of issues whenever I explore anything anytime in maven world.</div><div><br /></div><div>The <span style="font-family: courier;">docker-maven-plugin</span> docker goals <span style="font-family: courier;">start</span> and <span style="font-family: courier;">stop</span> are bound to <a href="https://maven.apache.org/surefire/maven-failsafe-plugin/examples/skipping-tests.html" target="_blank">maven failsafe plugin</a> <span style="font-family: courier;">pre-integration-test</span> and <span style="font-family: courier;">post-integration-test</span> phases of <span style="font-family: courier;">integration-test</span> goal. But if you skip tests by passing argument <span style="font-family: courier;">-DskipTests</span> which actually would skip running all unit and integration tests, I expected docker images not to be run. For instance <span style="font-family: courier;">./mvnw clean install -DskipTests</span> command to compile, build, package and install all modules by skipping tests. However, in this case, the images do get run: started and stopped. I couldn't find a way to get a hold on this, could be a bug or issue in the plugin by design, not sure.</div><div><br /></div><div>That was happening right after <span style="font-family: courier;"><a href="https://docs.spring.io/spring-boot/docs/current/maven-plugin/reference/htmlsingle/#integration-tests" target="_blank">spring-boot-maven-plugin</a></span>'s package goal. That plugin also gets bound to <span style="font-family: courier;">pre-integration-test</span> and <span style="font-family: courier;">post-integration-test</span> phases when specified. We don't have these executions specified for this plugin though.</div><div><br /></div><div>Couldn't figure out a workaround for not getting docker images run when integration test cases are skipped. If anyone has a solution for this, please post a comment, I would appreciate it.</div><div><br /></div><div>The word <span style="font-family: courier;">trivial</span> in software development is actually <span style="font-family: courier;">complex</span>. Things are unnecessarily made super-complex. Software developers get paid for dealing with it anyways ;)</div><div><br /></div><h3 style="text-align: left;">References</h3><div style="text-align: left;"><ul style="text-align: left;"><li><a href="https://github.com/fabric8io/docker-maven-plugin" target="_blank">Docker Maven Plugin - Github Repo</a></li><li><a href="https://dmp.fabric8.io/" target="_blank">Docker Maven Plugin - Documentation</a></li><li><a href="https://hub.docker.com/_/redis" target="_blank">Docker Hub - Redis Images</a></li><li><a href="https://maven.apache.org/surefire/maven-failsafe-plugin/examples/skipping-tests.html" target="_blank">Maven Failsafe skipping tests</a></li><li><a href="https://docs.spring.io/spring-boot/docs/current/maven-plugin/reference/htmlsingle/#integration-tests" target="_blank">Spring Boot Maven Plugin</a></li></ul></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-88282858704635536312023-08-31T17:59:00.001-04:002023-08-31T17:59:14.298-04:00Spring Boot 2.6 to 2.7 - Profile related changes go through increased complexity levels . . .<div style="text-align: left;"><span style="font-size: x-large;">I</span>ndirection is directly proportional to complexity. The more the indirection levels in one direction, the more the turns to take in reverse direction to understand. The higher the indirection levels, the deeper the dive in finding details.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">A simple feature like <a href="https://www.baeldung.com/spring-profiles" target="_blank">Spring Profiles</a> interwoven across several Spring frameworks is not only made it difficult to understand but also is painful to follow through multiple, disconnected sets of details, and multiple documents. Pulling necessary details together, across different versions of each framework, related to one specific feature is even more painful.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 20, </span><span style="font-family: "courier new", courier, monospace;">Spring Boot 2.6.3, </span><span style="font-family: "courier new" , "courier" , monospace;">Spring Boot 2.7.14, maven 3.9.3 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div><div style="text-align: left;"><b><a href="https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.profiles" target="_blank">Spring Profiles</a></b> is a pretty simple to understand indirection, abstracted out and externalized with several benefits. <a href="https://spring.io/projects/spring-framework" target="_blank">Spring</a> - as a framework to support all possible ways that this feature can be supported, also started to evolve. Cloud and cloud related frameworks extended it further increasing the levels of indirection and thus its complexity.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">In my recent attempt upgrading a Spring Boot application from Spring Boot 2.6.x to 2.7.x, failed to get deployed quietly with no ERRORS or WARNINGS in Kubernetes (K8S) environment. I noticed a log message about <b>multiple active profiles</b> in logs, and started digging deeper into why there were more than one active profile. There was certainly an interwoven behavioral change between these two Spring Boot minor versions around active profiles with different frameworks working together on profile related configurations, mainly coming in from <span style="font-family: courier;">bootstrap</span> and <span style="font-family: courier;">application</span> yaml files.</div><div style="text-align: left;"><br /></div><h3>The scenario</h3><div>I did migrate a Spring Boot 2.6.3 application successfully to 2.7.14 and tested locally with <span style="font-family: courier;">local</span> profile. The application also depends on Spring framework support for <b>Vault</b>, and <b>Consul</b>. The default active profile (<span style="font-family: courier;">spring.profiles.active</span>) is set to <span style="font-family: courier;">test</span> in <span style="font-family: courier;">application.yml</span> file. When the application is run locally an explicit active profile parameter is passed (<span style="font-family: courier;">-Dspring-boot.run.profiles=local</span>) to the maven goal: <span style="font-family: courier;">spring-boot:run</span>. So, profile: <span style="font-family: courier;">local</span> passed through maven build option gets passed as Java option, takes the precedence, and overrides the default active profile <span style="font-family: courier;">test</span>. The default active profile <span style="font-family: courier;">test</span> is good for the default profile active as there is no explicit profile specified during maven test phase when it runs unit and integration tests. Environment specific configuration files like <span style="font-family: courier;">application-int.yml</span>, <span style="font-family: courier;">application-int.yml</span> have <span style="font-family: courier;">spring.config.activate.on-profile</span> set to respective environment. So, environment specific configuration (e.g. <span style="font-family: courier;">application-int.yml</span>) is considered when active profile is set to specific environment (e.g. <span style="font-family: courier;">int</span>). All looked good before and after the upgrade.</div><div><br /></div><div>When the application gets built through concourse pipeline, after successful build including unit and integration tests run, it automatically gets deployed to both <span style="font-family: courier;">int</span> and <span style="font-family: courier;">cert</span>. The deployment script sets Java option <span style="font-family: courier;"><b>-Dspring.profiles.active=<env></b></span>, thus overriding the default active profile <span style="font-family: courier;">test</span> with specific env as command line option takes the precedence over configuration files. So, all worked as expected for local profile after the upgrade and went for deployment with the same expectations.</div><div><br /></div><div>Also, initially the application had one set of yaml files (<span style="font-family: courier;">application</span>) when Apache Mesos was the deployment platform. Later it got migrated to K8S deployment with added Vault and Consul support, which forced a new set of yaml configuration files (<span style="font-family: courier;">bootstrap</span>). Mainly Vault and Consul required certain properties to be configured through <span style="font-family: courier;">bootstrap</span> yaml set. So after the application got migrated from Apache Mesos to K8S, it ended up having two different sets of application properties: <span style="font-family: courier;">bootstrap.yml</span> set and <span style="font-family: courier;">application.yml</span> set, along with their counterpart env specific files for <span style="font-family: courier;">local</span>, <span style="font-family: courier;">test</span>, <span style="font-family: courier;">int</span>, <span style="font-family: courier;">cert</span> and <span style="font-family: courier;">prod</span>. With that there were 2 sets of 7 yaml files for each set. The first set: <span style="font-family: courier;">bootstrap, bootstrap-local, </span><span style="font-family: courier;">bootstrap-test,</span><span style="font-family: courier;"> </span><span style="font-family: courier;">bootstrap-dev, </span><span style="font-family: courier;">bootstrap-int, </span><span style="font-family: courier;">bootstrap-cert</span> and<span style="font-family: courier;"> </span><span style="font-family: courier;">bootstrap-prod.</span> The second set: <span style="font-family: courier;">application, </span><span style="font-family: courier;">application</span><span style="font-family: courier;">-local, </span><span style="font-family: courier;">application</span><span style="font-family: courier;">-test, </span><span style="font-family: courier;">application</span><span style="font-family: courier;">-dev, </span><span style="font-family: courier;">application</span><span style="font-family: courier;">-int, </span><span style="font-family: courier;">application</span><span style="font-family: courier;">-cert</span> and <span style="font-family: courier;">application</span><span style="font-family: courier;">-prod</span>.</div><div><br /></div><h3 style="text-align: left;">The issue</h3><div style="text-align: left;">The upgraded application (upgraded to Spring Boot 2.7.14) failed to come up successfully after it got deployed to K8S cluster through Concourse CI/CD pipeline with logs showing two active profiles during the application startup after deployment onto both <span style="font-family: courier;">int</span> and <span style="font-family: courier;">cert</span>. For instance on int logs, both <span style="font-family: courier;">test</span> and <span style="font-family: courier;">int</span> were logged in as active profiles.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">It was bit hard to dig into where this profiles handling of active profile getting overridden by the Java option passed was broken - instead of overriding, it was appended to active profiles. Spring allows more than one profile to be specified to be active separated by comma treating it as a list of active profiles. When multiple profiles are active, each profile specific configuration is read in the order of environments that appear in the list, and configuration properties get consolidated, with the later environment related properties replacing the previous environment related if there are same properties configured in both. In this case, it's like merging multiple environment specific configurations.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">In my case, <span style="font-family: courier;">test</span> profile was default active. The active profile passed through Java options, instead of replacing the default list which has just one element <span style="font-family: courier;">test</span>, it was getting prepended to the list. So in int the active profiles were: int, test and similarly in cert: cert, test. The <span style="font-family: courier;">test</span> profile being the last element in the list, was taking the precedence and the application in both <span style="font-family: courier;">int</span> and <span style="font-family: courier;">cert</span> failed to start up as <span style="font-family: courier;">test</span> environment specific configuration was not good for <span style="font-family: courier;">int</span> or <span style="font-family: courier;">cert</span>.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><h3>The fix (better one) - separating out test configurations into it's own configuration area under test</h3></div><div style="text-align: left;">All <span style="font-family: courier;">application</span>, and <span style="font-family: courier;">bootstrap</span> yaml files resided under <span style="font-family: courier;">src/main/resources</span> directory.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The fix I did for this issue involved the following changes:</div><div style="text-align: left;">1. Removed default active profile configuration property set in <span style="font-family: courier;">application.yml</span> file.</div><div style="text-align: left;">2. Moved test profile specific configuration files (<span style="font-family: courier;">application-test.yml</span> and <span style="font-family: courier;">bootstrap-test.yml</span>) that contain <span style="font-family: courier;">spring.config.activate.on-profile</span> property set to <span style="font-family: courier;">test</span> along with other <span style="font-family: courier;">test</span> environment related properties under <span style="font-family: courier;">src/test/resources</span> directory.</div><div style="text-align: left;">3. Added new <span style="font-family: courier;">application.yml </span>and <span style="font-family: courier;">bootstrap.yml</span> files under <span style="font-family: courier;">test/resources</span> directory. Both just contain one property <span style="font-family: courier;">spring.profiles.active</span> set to <span style="font-family: courier;">test</span>.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">With this separation, all configuration files including the base and environment specific reside under <span style="font-family: courier;">src/main/resources</span> directory. There is no active profile set at all in any of these configuration files under <span style="font-family: courier;">src/main/resources</span>. In other words, active profile is always passed in as an option for all environments: <span style="font-family: courier;">local</span>, <span style="font-family: courier;">dev</span>, <span style="font-family: courier;">int</span>, <span style="font-family: courier;">cert</span>, and <span style="font-family: courier;">prod</span>.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Test environment configuration files: <span style="font-family: courier;">application.yml</span>, <span style="font-family: courier;">bootstrap.yml</span>, <span style="font-family: courier;">application-test.yml</span>, and <span style="font-family: courier;">bootstrap-test.yml</span> reside under <span style="font-family: courier;">src/test/resources</span> directory.</div><div style="text-align: left;">Both <span style="font-family: courier;">application.yml</span>, and <span style="font-family: courier;">bootstrap.yml</span> have active profile set to <span style="font-family: courier;">test</span>. So spring testing framework considers these configurations when found under <span style="font-family: courier;">src/test/resources</span> during unit and integration tests. That way test environment active profile and it's associated test specific configurations are totally isolated from all other env configurations into its own configuration area: <span style="font-family: courier;">src/test/resources</span>. </div><div style="text-align: left;"><br /></div><div style="text-align: left;">This is more cleaner approach. Also isolates test configurations under <span style="font-family: courier;">src/test/resources</span> area which are not even bundled into the application jar.</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">Summary</h3><div style="text-align: left;">Enterprise Java's unnecessary overcomplexity gave birth to Spring Framework and it has quickly become popular and a de-facto Java framework since then. The core of it is based on the simple Dependency Injection (or Inversion of Control - IoC) design pattern. It started to grow into every corner of the technology concept. It is not simple anymore, in my opinion. There are too many layers of details one needs to know. With tons and tons Spring-eco-system frameworks, it has become even more complex. A framework started to simplify Enterprise Java development has grown too big to be(come) complex.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">A simple Java framework like JUnit itself is getting complex. So, no wonder Spring being in there already. Software developers absolutely love indirection and complexity. In Software Development simplicity is a rare quality. Even if some simplicity exists, it slowly becomes complex over a period of time. Simple becomes complex, complex only becomes super complex ;)</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Happy software development, and enjoy the complexity!</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">References</h3><div style="text-align: left;"><ul style="text-align: left;"><li><a href="https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.profiles" target="_blank">Spring Profiles - Spring Documentation</a></li><li><a href="https://www.baeldung.com/spring-profiles" target="_blank">Spring Profiles</a></li></ul></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-29164201978413693642023-08-28T16:43:00.000-04:002023-08-28T16:43:11.364-04:00IntelliJ - the Community Edition rescued the broken Ultimate Edition . . . ;)<div>It sound funny to say that - <a href="https://www.jetbrains.com/products/compare/?product=idea&product=idea-ce" target="_blank">IntelliJ Community Edition</a> (free edition) rescued Ultimate Edition (paid edition). But that's what really put me back on Ultimate Edition after many months of switching to Community Edition for my day-to-day development.</div><div><br /></div><div><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">IntelliJ Community Edition 2023.2.1</span><span style="font-family: "courier new", courier, monospace;">, </span><span style="font-family: "courier new", courier, monospace;">IntelliJ Ultimate Edition 2023.2.1, </span><span style="font-family: "courier new" , "courier" , monospace;">maven 3.9.3 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div><span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div><h3 style="text-align: left;">The Issue</h3><div>I stopped using IntelliJ Ultimate Edition as for some weird reason it got stuck with <b>maven dependencies broken paths </b>issue. Specifically, <span style="font-family: courier;">SLF4J</span> and <span style="font-family: courier;">javax</span> libraries were shown in maven dependency broken paths in red. It happened few months ago. One day one of my maven projects that uses Lombok and had classes annotated with @Slf4J suddenly couldn't recognize the <span style="font-family: courier;">log</span> (logger object) statements. When I checked project module dependencies there were broken maven dependency paths.</div><div><br /></div><div>I tried all recommended and possible ways to recover from this issue like: Invalidate Caches and restart the IDE(A), blowing away specific libraries maven cache <span style="font-family: courier;">~/.m2/libraries/.../*.*</span> and letting it build again, blowing away entire maven local cache <span style="font-family: courier;">~/.m2/*.*</span> and letting it build again, downloading dependency jars and explicitly setting dependency paths in the project/module settings, reinstalling the Ultimate Edition, even reinstalling different version of the Ultimate Edition etc. Nothing worked. I spent lot of time few times since then and couldn't get it back to working. Finally gave up and moved to <a href="https://www.jetbrains.com/products/compare/?product=idea&product=idea-ce" target="_blank">Community Edition</a>. Same projects that were having broken dependency issues in Ultimate Edition, when opened in Community edition, had no issues with those dependencies, same maven local cache paths are used by both. The Ultimate Edition complains, the Community Edition doesn't. Alas!</div><div><br /></div><h3 style="text-align: left;">The Fix</h3><div>Here is what I did to fix it.</div><div><br /></div><div>Started IntelliJ Ultimate Edition. At the startup there is a Customize link, click that and click Import Settings... link as shown below:</div><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0qnSTa8PlLfFfFXuo2p7R_FFfAejTaMy6pY0Kk8ovU5XqkyFK85pPW1jye3Kv4IvREs3JByUc7SMVnn2s2y4vToevwC-tIHlF3Dt6-5Q15K38qmXAxYbu8xGZs1MQMemEqLDMAYodcFvINCpw3zK2DrtkErToTgmNE73qE3zznEfulOPpexZl/s1390/Screen%20Shot%202023-08-28%20at%202.04.58%20PM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="898" data-original-width="1390" height="414" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0qnSTa8PlLfFfFXuo2p7R_FFfAejTaMy6pY0Kk8ovU5XqkyFK85pPW1jye3Kv4IvREs3JByUc7SMVnn2s2y4vToevwC-tIHlF3Dt6-5Q15K38qmXAxYbu8xGZs1MQMemEqLDMAYodcFvINCpw3zK2DrtkErToTgmNE73qE3zznEfulOPpexZl/w640-h414/Screen%20Shot%202023-08-28%20at%202.04.58%20PM.png" width="640" /></a></div><br /><div>Selected the Community Edition settings directory (<span style="font-family: courier;">~/Library/Application Support/JetBrains/IdeaIC2023.2</span>) of Community Edition in which my projects were fine with dependencies. It prompted to take a backup of current settings. I did that and imported ommunity Edition settings. It opened projects and the issue of broken dependency paths was gone.</div><div><br /></div><div>It seemed like, there was some broken dependency path setting saved in the Ultimate Edition settings that was stuck and not getting fixed by any means.<br /><div><br /></div><h3 style="text-align: left;">Some Internals</h3><div>IntelliJ saves all it's settings under the specific version's area. On Mac, by default, this area is under <span style="font-family: courier;">~/Library/Application Support/JetBrains</span> dir. The community edition directories start with <span style="font-family: courier;">IdeaIC<version></span> (e.g <span style="font-family: courier;">IdeaIC2023.2</span>) and ultimate edition directories start with <span style="font-family: courier;">IntelliJIdea<version></span> (e.g.<span style="font-family: courier;"> IntelliJIdea2023.2</span>). The <span style="font-family: courier;">options</span> sub-directory is where plugins settings get saved. </div><div><div><br /></div><h3 style="text-align: left;">Plugin Settings</h3><div>Plugin settings are pat of IntelliJ settings and get stored in xml files under <span style="font-family: courier;">options</span> sub-directory of specific IntelliJ version's settings directory. For instance, <a href="https://plugins.jetbrains.com/plugin/1833-awesome-editor" target="_blank">awesome editor plugin</a> is a simple and pretty neat plugin which lets add image backgrounds. I did setup plugin awesome editor to display different kinds of images for different types of files. To get all the settings from one IntelliJ version to another, just copy the plugin settings file (in this case it is: <span style="font-family: courier;">awesome-editor-3.xml</span>). </div><div><br /></div><div>Here is an example to copy plugin settings set in Ultimate edition to Community Edition.</div><div><b>NOTE</b>: Once copied restart IntelliJ Community Edition.</div><div><br /></div>
<div class="code">
$ cd ~/Library/"Application Support"/JetBrains
$ ls -al |grep IdeaIC
drwxr-xr-x 17 pottepalemg 163264107 544 Jul 26 2022 IdeaIC2022.1
drwxr-xr-x 20 pottepalemg 163264107 640 Nov 30 2022 IdeaIC2022.2
drwxr-xr-x 20 pottepalemg 163264107 640 Mar 30 15:23 IdeaIC2022.3
drwxr-xr-x 20 pottepalemg 163264107 640 Jul 7 11:31 IdeaIC2023.1
drwxr-xr-x 20 pottepalemg 163264107 640 Aug 28 10:13 IdeaIC2023.2
$ ls -al |grep IntelliJIdea
drwxr-xr-x 22 pottepalemg 163264107 704 Aug 24 15:43 IntelliJIdea2020.1
drwxr-xr-x 19 pottepalemg 163264107 608 Nov 19 2020 IntelliJIdea2020.2
drwxr-xr-x 21 pottepalemg 163264107 672 Mar 2 2022 IntelliJIdea2020.3
drwxr-xr-x 20 pottepalemg 163264107 640 Jul 9 2021 IntelliJIdea2021.1
drwxr-xr-x 24 pottepalemg 163264107 768 Feb 9 2022 IntelliJIdea2021.2
drwxr-xr-x 25 pottepalemg 163264107 800 Apr 12 2022 IntelliJIdea2021.3
drwxr-xr-x 26 pottepalemg 163264107 832 Sep 23 2022 IntelliJIdea2022.1
drwxr-xr-x 25 pottepalemg 163264107 800 Nov 30 2022 IntelliJIdea2022.2
drwxr-xr-x 25 pottepalemg 163264107 800 Jun 28 13:26 IntelliJIdea2022.3
drwxr-xr-x 24 pottepalemg 163264107 768 Aug 16 15:40 IntelliJIdea2023.1
drwxr-xr-x 22 pottepalemg 163264107 704 Aug 28 10:36 IntelliJIdea2023.2
drwxr-xr-x 11 pottepalemg 163264107 352 Aug 25 15:35 IntelliJIdea2023.2-backup
$ find . -name awesome*
./IntelliJIdea2023.2/options/awesome-editor-3.xml
$ ls -ltr ./IdeaIC2023.2/options | grep awesome*
$ cp ./IntelliJIdea2023.2/options/awesome-editor-3.xml ./IdeaIC2023.2/options
</div>
<div><br /></div><h3 style="text-align: left;">TIPS</h3><div><b>Getting back lost database connection settings into Ultimate Edition</b></div><div><br /></div><div>IntelliJ Ultimate Edition comes bundled with Database plugin that supports all features that are available in <a href="https://www.jetbrains.com/datagrip/" target="_blank">DataGrip</a> (an SQL IDE which is a product of JetBrains). I had database connections set to connect to various PostgreSQL databases (local, int, cert, prod etc.) which I lost by importing Community Edition settings. But I had a settings backup prompted and done for the Ultimate Edition settings that my broken Ultimate Edition was setup with when I imported Community Edition Settings. The backup directory is also listed in the above list of Ultimate Editions setting directories. To get those settings back onto my new settings imported from Community Edition, I had to repeat the <span style="font-family: courier;">Customize</span> > <span style="font-family: courier;">Import Settings...</span> step two more times. First time pointing it to the backup directory and selecting and copying all database connections settings to the clipboard. Second time pointing it to the Community Edition Directory and pasting those connection settings from the clipboard. This was the only way I could get those database connections copied. I got all connection settings except passwords fro every connection. I had to set password for every single connection individually. This was not possible by simply copying xml files like I did for awesome editor plugin.</div></div></div><div><br /></div><div><h3>Summary</h3></div><div>No software application or tool is bug-free. Applications do crash, tools do get corrupted. There is no one solution that works or fixes a similar issue for everybody. Some stupid, nasty, unknown, not very well documented internals of tools do take up lot of time to discover a fix that works for your situation and may help some others who get into similar situation.</div><div><br /></div><div>Hope this blog post on my discovery saves someone's time sometime when that someone bumps into it.</div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-644567044623630832023-08-23T09:25:00.002-04:002023-08-23T10:12:38.933-04:00Spring Batch - upgrade from 4.3.x to 5.0.x (breaking changes) . . .It makes sense to expect some breaking changes between two major versions. I recently ran into this when upgrading a Spring Boot based Spring Batch application from Spring Boot 2.7.14 to Spring Boot 3.1.2.<div><br /></div><div>The application is a Spring Boot command-line runnable app, which takes the batch job name(s) and input data file(s) (job related datafile arg, and filename) and processes those files by kicking off those jobs. This application required few major changes to get it functioning. I had to read the docs to identify few things that were already deprecated in 4.x and are removed in 5.x. also finding out on new things put in place.<div><br /></div><div><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 20, Spring Boot 2.7.14, </span><span style="font-family: "courier new", courier, monospace;">Spring Boot 3.1.2, </span><span style="font-family: "courier new", courier, monospace;">Spring Batch 4.3.8, </span><span style="font-family: "courier new", courier, monospace;">Spring Batch 5.0.2, </span><span style="font-family: "courier new" , "courier" , monospace;">maven 3.9.3 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div><br /></div><div>The following is the summary of high level changes. I am not going into details in this post as these changes are easy enough to understand at a high level.</div><div style="text-align: left;"><br />
1. The in memory map datasource in earlier versions was deprecated. Now it's removed. So, it requires to put in a configuration for this. The following is an example.</div><div style="text-align: left;">
<div class="code">
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.repository.support.JobRepositoryFactoryBean;
import org.springframework.batch.support.transaction.ResourcelessTransactionManager;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseBuilder;
import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseType;
import org.springframework.transaction.PlatformTransactionManager;
import javax.sql.DataSource;
/**
* In memory data source configuration for batch jobs.
* Defines {@link DataSource}, {@link PlatformTransactionManager} and {@link JobRepository} beans.
*
* @author Giri
* created Aug 16, 2023
*/
@Configuration
public class InMemoryBatchRepositoryConfig {
@Bean
public DataSource <span style="background-color: #fcff01;">inMemoryDataSource</span>() {
EmbeddedDatabaseBuilder builder = new EmbeddedDatabaseBuilder();
return builder.setType(EmbeddedDatabaseType.H2)
.addScript("classpath:org/springframework/batch/core/schema-drop-h2.sql")
.addScript("classpath:org/springframework/batch/core/schema-h2.sql")
.generateUniqueName(true)
.build();
}
@Bean
public PlatformTransactionManager <span style="background-color: #fcff01;">resourceLessTransactionManager</span>() {
return new ResourcelessTransactionManager();
}
@Bean
public JobRepository jobRepository() throws Exception {
JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
factory.setDataSource(inMemoryDataSource());
factory.setTransactionManager(resourceLessTransactionManager());
factory.afterPropertiesSet();
return factory.getObject();
}
}
</div>
<br />
2. @EnableBatchProcessing annotation takes new properties: <span style="font-family: courier;">dataSourceRef</span>, <span style="font-family: courier;">transactionManagerRef</span>.</div><div style="text-align: left;">
<div class="code">
@Slf4j
@Configuration
@EnableBatchProcessing(dataSourceRef = "<span style="background-color: #fcff01;">inMemoryDataSource</span>", transactionManagerRef = "<span style="background-color: #fcff01;">resourceLessTransactionManager</span>")
public class MyJobConfig {
@Value("${data_filename:NONE}")
String dataFilename;
...
}
</div>
<br />
3. <span style="font-family: courier;">JobBuilderFactory</span>, <span style="font-family: courier;">StepBuilderFactory</span> are removed. Use <span style="font-family: courier;">JobBuilder</span> and <span style="font-family: courier;">StepBuilder</span> instead.</div><div style="text-align: left;">
<div class="code">
@Slf4j
@Configuration
@EnableBatchProcessing(dataSourceRef = "inMemoryDataSource", transactionManagerRef = "resourceLessTransactionManager")
public class MyJobConfig {
...
@Bean
public Job myJob(
JobRepository jobRepository,
Step myJobStep,
JobCompletionNotificationListener jobCompletionNotificationListener) {
return new <span style="background-color: #fcff01;">JobBuilder</span>("myJob", jobRepository)
.listener(jobCompletionNotificationListener)
.flow(myJobStep)
.end()
.build();
}
@Bean
public Step myJobStep(
JobRepository jobRepository,
PlatformTransactionManager platformTransactionManager,
ItemFailureLoggerListener itemFailureLoggerListener,
StepListener stepListener){
var step = new <span style="background-color: #fcff01;">StepBuilder</span>("myJobStep", jobRepository)
.<MyDomainObject, String> chunk(10, platformTransactionManager)
.reader(myDomainObjectReader()) //input
.processor(myDomainObjectProcessor(isGm5)) //transformer/processor
.writer(myDomainObjectWriter())//output
.listener((ItemProcessListener) itemFailureLoggerListener)
.build();
step.registerStepExecutionListener(stepListener);
return step;
}
...
}
</div>
<br />
4. Unlike previous versions, jobs won't start by setting the property: <span style="font-family: courier;">spring.batch.job.names</span> to a comma separated names of jobs. Need to explicitly start the job launcher. The following is a code snippet of the main application which takes a job name passed from command line by property <span style="font-family: courier;">spring.batch.job.name</span> and launches that job.</div><div style="text-align: left;"><br /></div>
<div class="code">
@Slf4j
@SpringBootApplication
public class BatchApplication implements CommandLineRunner, InitializingBean {
@Value("${spring.batch.job.name:NONE}")
private String jobName;
@Autowired
ApplicationContext applicationContext;
@Autowired
JobLauncher jobLauncher;
@Override
public void afterPropertiesSet() {
try {
validateJobName();
} catch (Exception ex) {
log.error(ex.getMessage());
System.exit(SpringApplication.exit(applicationContext, () -> 1));
}
}
public static void main(String[] args) {
SpringApplication app = new SpringApplicationBuilder(ScoringEtlApplication.class)
.web(WebApplicationType.NONE)
.logStartupInfo(false)
.build(args);
app.run(args);
}
@Override
public void run(String... args) {
log.debug("Beans Count:{} Beans:{}", applicationContext.getBeanDefinitionCount(), applicationContext.getBeanDefinitionNames());
Job job = (Job)applicationContext.getBean(jobName);
JobParameters jobParameters = new JobParametersBuilder()
.addString("jobID", String.valueOf(System.currentTimeMillis()))
.toJobParameters();
try {
jobLauncher.run(job, jobParameters);
log.info("Finished running job(s): {} with args: {}", jobName, Arrays.stream(args).collect(Collectors.joining(", ")));
} catch (JobExecutionAlreadyRunningException | JobRestartException | JobInstanceAlreadyCompleteException | JobParametersInvalidException e) {
throw new RuntimeException(e);
}
}
/**
* Validates jobName and displays usage and appropriate error message upon validation failure.
*/
private void validateJobName() throws URISyntaxException {
String errorMessage = null;
if (jobName.isEmpty() || jobName.equals("NONE")) {
errorMessage = "No job(s) specified. Please, specify valid job_name(s)";
}
if(errorMessage != null) {
String runningJar = this.getClass().getClassLoader().getClass()
.getProtectionDomain()
.getCodeSource()
.getLocation()
.toURI()
.getPath();
// display Usage
log.info("""
USAGE:
Run a specific job:
java -jar <runnableJar> \\
--spring.batch.job.name=<job_name> \\
--<job_arg_data_filename>="job_data_file.csv"
""".replace("runningJar", runningJar));
throw new IllegalArgumentException(errorMessage);
} else {
List<string> allJobNames = Arrays.stream(applicationContext.getBeanNamesForType(Job.class)).toList();
log.debug("Available Jobs: {}", allJobNames);
log.info("Running Job: {}", jobName);
}
}
}
</string></div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">Other classes referenced in Job configurations:</div>
<div class="code">
public class JobCompletionNotificationListener extends JobExecutionListenerSupport {
@Value("${spring.batch.job.name:default-job}")
private String jobName;
@Override
public void beforeJob(JobExecution jobExecution) {
log.info("JOB: {} - About to start.", jobName);
}
@Override
public void afterJob(JobExecution jobExecution) {
if(jobExecution.getStatus() == BatchStatus.COMPLETED) {
var from = jobExecution.getStartTime();
var to = jobExecution.getEndTime();
log.info("JOB: {} - Finished running. Took {} milliseconds. Verify results.", jobName, ChronoUnit.MILLIS.between(from, to));
}
}
}
@Slf4j
@Component
public class ItemFailureLoggerListener extends ItemListenerSupport</div>
<br />
<h3 style="text-align: left;">TIPS</h3><div style="text-align: left;">When running the command line application, if there are warnings: <span style="font-family: courier;">Using deprecated '-debug' fallback for parameter name resolution. Compile the affected code with '-parameters' instead or avoid its introspection: MyJobConfig,</span> then suppress it by setting the following maven-compiler-plugin setting:</div>
<div class="code">
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven-compiler-plugin.version}</version>
<configuration>
<source>${javac.source.version}</source>
<target>${javac.target.version}</target>
<release>${javac.release.version}</release>
<compilerArgs>
<arg>-parameters</arg>
</compilerArgs>
</configuration>
</plugin>
</div>
<br />
<h3 style="text-align: left;">References</h3><div style="text-align: left;"><ul style="text-align: left;"><li><a href="https://docs.spring.io/spring-batch/docs/5.0.2/reference/html/index-single.html" target="_blank">Spring Batch 5.0.2 Reference Documentation</a></li><li><a href="https://github.com/spring-projects/spring-batch/wiki/Spring-Batch-5.0-Migration-Guide" target="_blank">Spring Batch Migration Guide</a></li></ul></div></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-46096871222026221382023-08-11T17:35:00.037-04:002023-12-11T18:22:10.447-05:00Spring boot - your own banner, actuator, version details etc . . .<div style="text-align: left;">Art is good for eyes. Spring boot out of the box comes with a nice text banner of it's name : <span style="font-family: courier;">Spring Boot</span> and displays the version right below the banner. Out of the box, the banner mode is set on and it shows up when the application gets started. There are several articles available on customizing this banner. Also, spring boot documentation has a brief section on thus as well (link in the resources).</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 20, Spring Boot 2.7.14, maven 3.9.3 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><br /></div><div style="text-align: left;">It's always good to customize the banner and see your application name whenever it comes up. Of course, you can display more good-to-have version details like: Spring Boot version, Java version, Your application version, and other available properties along with the banner.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Following is the quick list of steps to have a custom banner in your application.</div><div style="text-align: left;"><ul style="text-align: left;"><li>Generate a text banner for your application name. There are several sites for doing this. The one such is: <a href="https://springhow.com/spring-boot-banner-generator/">https://springhow.com/spring-boot-banner-generator/</a>. Generate a text banner and download the text file.</li><li>Edit the file and add the following properties for versions at the bottom:</li></ul>
<div class="code">
____ _ ____ ___ __
| __ ) ___ ___ | |_ / ___|_ __ __ _ __ _| \ \ / / __ ___
| _ \ / _ \ / _ \| __| | | _| '__/ _` |/ _` | |\ \ / / '_ ` _ \
| |_) | (_) | (_) | |_ | |_| | | | (_| | (_| | | \ V /| | | | | |
|____/ \___/ \___/ \__| \____|_| \__,_|\__,_|_| \_/ |_| |_| |_|
:: Spring Boot :: ${spring-boot.version}
:: Running on Java :: ${java.version}
:: Application :: ${project.version}
</div>
<ul style="text-align: left;"><li>Place the text file with file-name: <span style="font-family: courier;">banner.txt</span> under <span style="font-family: courier;">src/java/resources</span> folder.</li><li>In your maven build file (<span style="font-family: courier;">pom.xml</span>), make sure that you turn on maven filtering and add the resource directory for the extra version properties added to be resolved during the build.</li><li>Also, make sure you have <span style="font-family: courier;">maven-resources-plugin</span> configured to filter resources.</li></ul>
<div class="code"> <build>
<<span style="background-color: #fcff01;">resources</span>>
<resource>
<filtering><span style="background-color: #fcff01;">true</span></filtering>
<directory><span style="background-color: #fcff01;">${project.basedir}/src/main/resources/</span></directory>
</resource>
</resources>
<<span style="background-color: #fcff01;">testResources</span>>
<testResource>
<filtering><span style="background-color: #fcff01;">true</span></filtering>
<directory>${project.basedir}/src/test/resources/</directory>
</testResource>
</testResources>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId><span style="background-color: #fcff01;">maven-resources-plugin</span></artifactId>
<version>3.3.1</version>
<executions>
<execution>
<id>filter-resources</id>
<phase>process-resources</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<resources>
<!-- # Filtered Resources -->
<resource>
<directory>${project.basedir}/src/main/resources/</directory>
<span style="background-color: #fcff01;"><filtering>true</filtering></span>
</resource>
</resources>
<outputDirectory>${project.build.directory}</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
...
</build>
</div>
</div><div style="text-align: left;"><br /></div><h3>Gotcha-1</h3><div>When you run maven build goal(s) that also runs test-cases like for e.g. <span style="font-family: courier;">./mvnw clean install</span>, the banner shows up with all properties resolved for every integration test-case. However, for the custom properties used in the banner to be filtered and shown you need to add the same custom property in <span style="font-family: courier;">application.yml</span> if you happen to separate out test configurations under <span style="font-family: courier;">test/resources</span>. Also, make sure that you have enabled <<span style="font-family: courier;">testResources></span> filtering as well as shown above.</div><div><br /></div><h3>Gotcha-2 (Spring Boot 3.x)</h3><div>The above <span style="font-family: courier;">${project.version}</span> doesn't work in Spring boot 3.x. In this case a custom application version property (e.g. <span style="font-family: courier;">app.version</span>) can be defined in <span style="font-family: courier;">application.yml</span> or <span style="font-family: courier;">application.properties</span> and that can be used in the <span style="font-family: courier;">banner.txt</span> file.</div>
<div>E.g.<span style="font-family: courier;"> application.yml</span></div>
<div class="code">spring:
application:
name: @project.name@
# custom property for banner
<span style="background-color: #fcff01;">app:
version: @project.version@
</span></div>
<br />
<div class="code">
____ _ ____ ___ __
| __ ) ___ ___ | |_ / ___|_ __ __ _ __ _| \ \ / / __ ___
| _ \ / _ \ / _ \| __| | | _| '__/ _` |/ _` | |\ \ / / '_ ` _ \
| |_) | (_) | (_) | |_ | |_| | | | (_| | (_| | | \ V /| | | | | |
|____/ \___/ \___/ \__| \____|_| \__,_|\__,_|_| \_/ |_| |_| |_|
:: Spring Boot :: ${spring-boot.version}
:: Running on Java :: ${java.version}
:: Application :: ${<span style="background-color: #fcff01;">app.version</span>}
</div>
<br />
<h3 style="text-align: left;">Actuator</h3><div>Actuator provides production-ready endpoints for monitoring the application. Just adding the following dependency in pom.xml will do. Once the application is up, check <span style="font-family: courier;">http://localhost:8080/actuator</span></div><div><br /></div>
<div class="code"> <dependency>
<groupId>org.springframework.boot</groupId>
<artifactId><span style="background-color: #fcff01;">spring-boot-starter-actuator</span></artifactId>
</dependency>
</div>
<br />
However, by default not all end-points are enabled. Only <span style="font-family: courier;">/actuator</span>, and <span style="font-family: courier;">/actuator/health</span> end-points are enabled. By setting the property <span style="font-family: courier;">management.endpoints.web.exposure.include=*</span> in <span style="font-family: courier;">application.properties</span> or corresponding <span style="font-family: courier;">application.yml</span> will activate all end-points. <div><br /></div><div><b>Application information</b></div><div><br /></div><div>The <span style="font-family: courier;">/actuator/info</span> endpoint displays application information. By default there is no information. So, <span style="font-family: courier;">http://localhost:8080/actuator/info</span> displays empty JSON:</div>
<div class="code">{}</div>
<div><br /></div><b>
Maven plugin - spring-boot-maven-plugin</b><div><br /><div>This plugin comes with build execution goal: <span style="font-family: courier;">build-info</span> which is run by default and creates build information file: <span style="font-family: courier;">build-info.properties</span> under <span style="font-family: courier;">target/classes/META_INF</span> directory. Properties listed in this file are available through the endpoint: http://localhost:8080/actuator/info.</div><div>The following is an example of build-info goal execution configuration:</div><div><br /><div class="code"><plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<span style="background-color: #fcff01;"><execution>
<!-- Useful info on /actuator/info -->
<id>build-info</id>
<goals>
<goal>build-info</goal>
</goals>
</execution></span>
</executions>
</plugin>
</div>
<div><br /></div><div>The above configuration generates <span style="font-family: courier;">build-info.properties</span> file with pre-defined build info properties like:</div>
<div class="code">build.artifact=boot-graalvm
build.group=com.example
build.name=boot-graalvm
build.time=2023-09-22T16\:04\:27.970Z
build.version=0.0.1-SNAPSHOT
</div>
<br />
<div>With the above file generated, the <span style="font-family: courier;">/actuator/info</span> endpoint response would look like:</div>
<div class="code">{
"build": {
"artifact": "boot-graalvm",
"name": "boot-graalvm",
"time": "2023-09-22T16:04:27.970Z",
"version": "0.0.1-SNAPSHOT",
"group": "com.example"
}
}
</div>
<div> <div style="text-align: left;">Additional custom properties can be added by configuring the <span style="font-family: courier;">build-info</span> goal. For instance to add Java version and the Spring Boot version that the application is running with, the following additional configuration can be added:</div>
<div class="code"> ...
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>3.1.2</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>boot-graalvm</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>boot-graalvm</name>
<description>GraalVm project for Spring Boot</description>
<properties>
<java.version>20</java.version>
<spring.boot.version>${parent.version}</spring.boot.version>
</properties>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<!-- Useful info on /actuator/info -->
<id>build-info</id>
<goals>
<goal>build-info</goal>
</goals>
<span style="background-color: #fcff01;"><configuration>
<additionalProperties>
<java.version>${java.version}</java.version>
<spring.boot.version>${spring.boot.version}</spring.boot.version>
</additionalProperties>
</configuration></span>
</execution>
</executions>
...
</div>
<br />
Which would result with additional properties in the <span style="font-family: courier;">build-info.properties</span> file like:
<div class="code">build.java.version=20
build.spring.boot.version=3.1.2
</div>
<br />The <span style="font-family: courier;">/actuator/info</span> endpoint response looks like:
<div class="code">{
"build": <span style="background-color: #fcff01;">{
"java": {
"version": "20"
},
"spring": {
"boot": {
"version": "3.1.2"
}
}</span>,
"version": "0.0.1-SNAPSHOT",
"artifact": "boot-graalvm",
"name": "boot-graalvm",
"time": "2023-09-22T18:52:39.193Z",
"group": "com.example"
}
}
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Java information</b></div><div style="text-align: left;"><b><br /></b></div><div style="text-align: left;">Info endpoint has several info contributors like build, env, java, git etc. By default they are disabled. Enabling these would show related information collected by Spring.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">To enable java info, add <span style="font-family: courier;">management.info.java.enabled=true</span> in <span style="font-family: courier;">application.properties</span> or <span style="font-family: courier;">management.info.java.enabled:true</span> in application.yml.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">This will show the following java info details collected like:</div>
<div class="code">{
"build": {
"java": {
"version": "20"
},
"spring": {
"boot": {
"version": "3.1.2"
}
},
"version": "0.0.1-SNAPSHOT",
"artifact": "boot-graalvm",
"name": "boot-graalvm",
"time": "2023-09-22T18:52:39.193Z",
"group": "com.example"
},
<span style="background-color: #fcff01;">"java": {
"version": "20",
"vendor": {
"name": "Amazon.com Inc.",
"version": "Corretto-20.0.0.36.1"
},
"runtime": {
"name": "OpenJDK Runtime Environment",
"version": "20+36-FR"
},
"jvm": {
"name": "OpenJDK 64-Bit Server VM",
"vendor": "Amazon.com Inc.",
"version": "20+36-FR"
}
}</span>
}
</div>
<h3 style="text-align: left;">TIP</h3><div style="text-align: left;">These build properties generated and available through <span style="font-family: courier;">/actuator/info</span> are also are available through <span style="font-family: courier;">BuildProperties</span> object which can be auto-wired into any of the Spring managed beans and be accessed. For instance these properties can be outputted on swagger-ui page.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><a href="https://github.com/gpottepalem/boot-graalvm/tree/main" target="_blank">Reference Application</a></div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">References</h3><div style="text-align: left;"><ul style="text-align: left;"><li><a href="https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.spring-application.banner" target="_blank">Spring Boot Documentation - Customizing the Banner</a></li><li><a href="https://springhow.com/spring-boot-banner-generator/" target="_blank">Generate banner text file</a></li><li><a href="https://docs.spring.io/spring-boot/docs/current/reference/html/actuator.html" target="_blank">Actuator</a></li><li><a href="https://reflectoring.io/spring-boot-info-endpoint/" target="_blank">Actuator Info endpoint</a></li><li><a href="https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#appendix.test-auto-configuration" target="_blank">Spring Boot Test Slices</a></li></ul></div></div></div></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-70455805988391918582023-08-05T07:21:00.009-04:002023-09-21T15:41:00.605-04:00Java bytecode - compiler version options and compatibilities . . .<div style="text-align: left;">One of many strengths of Java platform is its backward compatibility with the language. As language keeps evolving and moving forward, the good old syntax is still supported for backward compatibility. However, the compiler adds certain indicative options for specifying version details. The <span style="font-family: courier;">--source</span>, <span style="font-family: courier;">--target</span> are two such compiler (<span style="font-family: courier;">javac</span>) options. From Java 9 onwards a third option <span style="font-family: courier;">--release</span> got added to this mix. Getting a good understanding of these options is not trivial without actually experiencing all three. When compiling source code of a single class you may not need to specify these options. But in Java project when building with maven like build system, one needs to understand these options and their implications.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 20, Spring Boot 2.7.15, maven 3.9.3 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div><div style="text-align: left;"><h3>The maven-compiler-plugin</h3></div><div style="text-align: left;"><div>Maven build system uses <a href="https://maven.apache.org/plugins/maven-compiler-plugin/index.html" target="_blank">maven-compiler-plugin</a> for compiling source code. This <a href="https://maven.apache.org/plugins/maven-compiler-plugin/index.html" target="_blank">plugin documentation</a> upfront talks about <span style="font-family: courier;">source</span> and <span style="font-family: courier;">target</span> options and highly recommends to change these in plugin configuration. In order to change these per application/module needs, one needs to look under the hood for understanding.</div><div><br /></div><div>Various extra Java compiler options can be specified in the <a href="https://maven.apache.org/plugins/maven-compiler-plugin/index.html" target="_blank">maven-compiler-plugin</a> configuration. The actual Java compiler options related to version are: <span style="font-family: courier;">--source</span>, <span style="font-family: courier;">--target</span> and <span style="font-family: courier;">--release</span> that can be specified and passed to the compiler during code compilation through <a href="maven-compiler-plugin" target="_blank">maven-compiler-plugin</a> configuration. This can be done in two different ways in <span style="font-family: courier;">pom.xml</span>:</div><div><br /></div><div>1. Through maven properties: <span style="font-family: courier;">maven.compiler.source</span>, <span style="font-family: courier;">maven.compiler.target</span> and <span style="font-family: courier;">maven.compiler.release</span> as highlighted below:</div><div><br />
<div class="code">
<properties>
<<span style="background-color: #fcff01;">maven.compiler.source</span>>20</maven.compiler.source>
<<span style="background-color: #fcff01;">maven.compiler.target</span>>20</maven.compiler.target>
<<span style="background-color: #fcff01;">maven.compiler.target</span>>20</maven.compiler.target>
</properties>
</div><div><br /></div><div>If these properties are not explicitly defined, maven compiler plugin uses 1.8 for <span style="font-family: courier;">source</span> and <span style="font-family: courier;">target</span>.</div><div><br /></div>
2. Through the plugin configuration settings as highlighted below. <b>Note</b> - For convenience defined extra properties and used for <span style="font-family: courier;">source</span>, <span style="font-family: courier;">target</span> and <span style="font-family: courier;">release</span> but straight version numbers can be used.</div><div><br />
<div class="code">
...
<properties>
<java.version>20</java.version>
<javac.source.version>${java.version}</javac.source.version>
<javac.target.version>${java.version}</javac.target.version>
<javac.release.version>${java.version}</javac.release.version>
<!-- Maven plugins -->
<maven-compiler-plugin.version>3.11.0</maven-compiler-plugin.version>
</properties>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven-compiler-plugin.version}</version>
<configuration>
<<span style="background-color: #fcff01;">source</span>>${javac.source.version}</source>
<<span style="background-color: #fcff01;">target</span>>${javac.target.version}</target>
<<span style="background-color: #fcff01;">release</span>>${javac.release.version}</release>
<compilerArgs>
<arg>-Xlint:all</arg>
</compilerArgs>
</configuration>
</plugin>
</plugins>
</pluginManagement>
...
</div>
<div><br /></div><div>If no special configuration is required, it doesn't require even to specify maven-compiler-plugin. If specified, the above are two ways to control/change default 1.8 set by the plugin for these options which eventually get passed to the Java compiler (<span style="font-family: courier;">javac</span>) during code compilation of sources (both under <span style="font-family: courier;">src</span> and <span style="font-family: courier;">test</span>. </div><div><br /></div><div><b>Note</b> from Java 9 onwards, the values to these options are not like 1.7, 1.8 but must be 7 and 8.</div><div><br /></div><div><h3>Java 20</h3></div><div>Java 20 compiler doesn't support version 7 for <span style="font-family: courier;">source</span>, <span style="font-family: courier;">target</span> anymore. The supported releases are 8 through 20. So, for any reason if maven compiler plugin is set explicitly with 1.7, build fails with <span style="color: red;">ERRORS</span> saying: <span style="color: red; font-family: courier;">Source option 7 is no longer supported. Use 8 or later.</span> , and <span style="font-family: courier;"><span style="color: red;">Target option 7 is no longer supported. Use 8 or later.</span> </span></div><div><br /></div><div>Now it's time to understand what these options actually tell the compiler, <span style="font-family: courier;">javac</span>. The compiler's help option (<span style="font-family: courier;">javac -help</span>) lists all available options and a brief description about each option. The <span style="font-family: courier;">-source</span>, <span style="font-family: courier;">-target</span>, <span style="font-family: courier;">-release</span> options descriptions are helpful to some extent.</div>
<br />
<div class="code">
--source <release>, -source <release>
Provide source compatibility with the specified Java SE release. Supported releases: 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20
--target <release>, -target <release>
Generate class files suitable for the specified Java SE release. Supported releases: 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20
--release <release>
Compile for the specified Java SE release. Supported releases: 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20</div></div><div><br /></div><div>To understand these compiler options better, we can compile a simple Java application class with just main method.</div><div><br /></div><span style="font-family: courier;">
HelloJava.java</span><br />
<div class="code">
import java.util.Properties;
public class HelloJava {
public static void main(String[] args) {
Properties systemProperties = System.getProperties();
System.out.println(String.format("Hello Java %s!", systemProperties.getProperty("java.vm.specification.version")));
systemProperties.entrySet().stream()
.filter(entry -> entry.getKey().toString().startsWith("java"))
.<span style="background-color: #fcff01;">toList()</span>.stream()
.forEach(entry ->
System.out.println(entry.getKey() + "=" + systemProperties.getProperty(entry.getKey().toString()))
);
}
}
</div>
<b>Note</b> - The above class prints system properties that start with "java" with their values. It uses <span style="font-family: courier;">toList()</span> method that Java 16 added to <span style="font-family: courier;">Stream</span> class. With this the expectation is- the code should not be compiled for Java/JVM version less than 16.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><h3>Java compiler version options</h3><div>Let's compile the class with different Java versions and compiler options.</div><div> </div></div><div style="text-align: left;"><div class="code">// <span style="background-color: #fcff01;">compile with Java 20: no options specified</span>
$ sdk use java 20.0.2-amzn
// check: major version
$ javap -verbose HelloJava.class | grep major
major version: 64
// run on Java 20: works
$ javac HelloJava.java
"Hello Java 20!"
// switch to Java 17 and run: fails with LinkageError
$ sdk use java 17.0.1.12.1-amzn
$ java HelloJava
Error: LinkageError occurred while loading main class HelloJava
java.lang.UnsupportedClassVersionError: HelloJava has been compiled by a more recent version of the Java Runtime (class file version 64.0), this version of the Java Runtime only recognizes class file versions up to 61.0
// switch to Java 15 and run: fails with LinkageError
$ sdk use java 15.0.2.7.1-amzn
$ java HelloJava
Error: LinkageError occurred while loading main class HelloJava
java.lang.UnsupportedClassVersionError: HelloJava has been compiled by a more recent version of the Java Runtime (class file version 64.0), this version of the Java Runtime only recognizes class file versions up to 59.0
// switch to Java 8 and run: fails with UnsupportedClassVersionError Exception
$ sdk use java 8.0.352-amzn
$ java HelloJava
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.UnsupportedClassVersionError: HelloJava has been compiled by a more recent version of the Java Runtime (class file version 64.0), this version of the Java Runtime only recognizes class file versions up to 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:601)
</div>
<br />
<div>So. when compiled with a specific version of java compiler (in this case Java 20 and no version options are specified), it gets compiled with default target of the Java compiler which is 20. The class generated <b>cannot be run on prior JVM versions</b> (prior to 20). To find the target of JVM code the byte-code is generated for, use <span style="font-family: courier;">javap -verbose HelloJava.class | grep major</span>.</div><div><br /></div>
<div>Now, let's try compiling for target 17 and try to run on different JVM versions.</div><div><br /></div>
<div class="code">
// <span style="background-color: #fcff01;">compile with Java 20 for target 17: --source and --target options specified</span>
$ sdk use java 20.0.2-amzn
$ javac --source=17 --target=17 HelloJava.java
warning: [options] system modules path not set in conjunction with -source 17
1 warning
// check: major version
$ javap -verbose HelloJava.class | grep major
major version: 61
// run on Java 20: works
$ java HelloJava
Hello Java 20!
// switch to Java 17 and run: works
$ sdk use java 17.0.1.12.1-amzn
// run on Java 17: works
$ java HelloJava
Hello Java 17!
// swicth to Java 15 and run: fails with LinkageError
$ sdk use java 15.0.2.7.1-amzn
$ java HelloJava
Error: LinkageError occurred while loading main class HelloJava
java.lang.UnsupportedClassVersionError: HelloJava has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 59.0
// compile with Java 20 for target 17: --source and --target options specified
$ sdk use java 20.0.2-amzn
$ javac --source=15 --target=15 HelloJava.java
warning: [options] system modules path not set in conjunction with -source 15
1 warning
// check: major version
$ javap -verbose HelloJava.class | grep major
major version: 59
// switch to Java 17 and run: works
$ sdk use java 17.0.1.12.1-amzn
$ java HelloJava
Hello Java 17!
// swicth to Java 15 and run: fails with NoSuchMethodError
$ sdk use java 15.0.2.7.1-amzn
$ java HelloJava
Hello Java 15!
Exception in thread "main" java.lang.NoSuchMethodError: 'java.util.List java.util.stream.Stream.toList()'
at HelloJava.main(HelloJava.java:15)
// switch to Java 8 and run: fais with UnsupportedClassVersionError
$ sdk use java 8.0.352-amzn
$ java HelloJava
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.UnsupportedClassVersionError: HelloJava has been compiled by a more recent version of the Java Runtime (class file version 59.0), this version of the Java Runtime only recognizes class file versions up to 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:601)
// compile with Java 20 for target 8: --source and --target options specified
$ sdk use java 20.0.2-amzn
$ javac --source=8 --target=8 HelloJava.java
javac --source=8 --target=8 HelloJava.java
warning: [options] bootstrap class path not set in conjunction with -source 8
warning: [options] source value 8 is obsolete and will be removed in a future release
warning: [options] target value 8 is obsolete and will be removed in a future release
warning: [options] To suppress warnings about obsolete options, use -Xlint:-options.
4 warnings
// check: major version
$ javap -verbose HelloJava.class | grep major
major version: 52
// switch to Java 20 and run: works
$ sdk use java 20.0.2-amzn
$ java HelloJava
Hello Java 20!
// switch to Java 17 and run: works
$ sdk use java 17.0.1.12.1-amzn
$ java HelloJava
Hello Java 17!
// swicth to Java 15 and run: fails with NoSuchMethodError
$ sdk use java 15.0.2.7.1-amzn
$ java HelloJava
Hello Java 15!
Exception in thread "main" java.lang.NoSuchMethodError: 'java.util.List java.util.stream.Stream.toList()'
at HelloJava.main(HelloJava.java:15)
</div>
<br />So, when compiled for a target version, it cannot be run on prior JVM versions.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">
Let's try target 7.<div class="code">
$ sdk use java 20.0.2-amzn
$ javac <span style="background-color: #fcff01;">--source=7 --target=7</span> HelloJava.java
warning: [options] bootstrap class path not set in conjunction with -source 7
error: <span style="background-color: #fcff01;">Source option 7 is no longer supported. Use 8 or later</span>.
error: <span style="background-color: #fcff01;">Target option 7 is no longer supported. Use 8 or later</span>.
</div>Java version 7 is not supported anymore.</div><div style="text-align: left;"><br />
Let's experience --release option.<div class="code">
// compile with Java 20 using options --source and --target options specified
$ sdk use java 20.0.2-amzn
$ javac --source=17 --target=17 --release=17 HelloJava.java
error: <span style="background-color: #fcff01;">option --source cannot be used together with --release</span>
error: <span style="background-color: #fcff01;">option --target cannot be used together with --release</span>
Usage: javac <options> <source files>
use --help for a list of possible options
// compile with Java 20 for target 17: --release options specified
$ sdk use java 20.0.2-amzn
$ javac --release=17 HelloJava.java
// check: major version
$ javap -verbose HelloJava.class | grep major
major version: 61
// switch to Java 20 and run: works
$ sdk use java 20.0.2-amzn
$ java HelloJava
Hello Java 20!
// switch to Java 17 and run: works
$ sdk use java 17.0.1.12.1-amzn
$ java HelloJava
Hello Java 17!
// switch to Java 15 and run: fails with NoSuchMethodError
$ sdk use java 15.0.2.7.1-amzn
$ java HelloJava
Hello Java
Exception in thread "main" java.lang.NoSuchMethodError: 'java.util.List java.util.stream.Stream.toList()'
at HelloJava.main(HelloJava.java:14)
// compile with Java 20 for target 15: --release options specified
$ sdk use java 20.0.2-amzn
$ javac --release=15 HelloJava.java
HelloJava.java:14: error: cannot find symbol
.toList().stream()
^
symbol: method toList()
location: interface Stream<Entry<Object,Object>>
1 error
// compile with Java 20 for target 15: --source --target options specified
$ sdk use java 20.0.2-amzn
$ javac --source=15 --target=15 HelloJava.java
warning: [options] system modules path not set in conjunction with -source 15
1 warning
// switch to Java 15 and run: fails with NoSuchMethodError
$ sdk use java 15.0.2.7.1-amzn
$ java HelloJava
Exception in thread "main" java.lang.NoSuchMethodError: 'java.util.List java.util.stream.Stream.toList()'
at HelloJava.main(HelloJava.java:14)
</div><br />
So, <b>--release </b>option does a strict compilation time checks to see if the code is compliant with the release target and fails to compile if a method not supported in target version is used in the code. This makes sure the compiled class works on target release (the target JVM version that the code is released to run on). Whereas, the --target option doesn't do code compliance checks during compilation time, it simply compiles code but fails during runtime. So, <span style="font-family: courier;"><b>--release</b></span> seems like <b>better option</b> to leverage when specifying.</div><div style="text-align: left;"><br /><h3 style="text-align: left;">Implications of version options</h3><div><ul style="text-align: left;"><li><b>No compiler options specified:</b> The code gets compiled with default target as the version of the Java compiler.</li><li><b>All 3 options <span style="font-family: courier;">--source</span>, <span style="font-family: courier;">--target</span> and <span style="font-family: courier;">--release</span> specified:</b> Not allowed.</li><li><b>Options <span style="font-family: courier;">--source</span>, <span style="font-family: courier;">--target</span> specified</b>: 1) Both can be same (e.g. 17, 17). 2) The option: --source can be lower version (e.g. 15) and --target can be higher version (e.g. 17), but not the other way. If --source is higher version (e.g.17) and --target is lower version (e.g.15), compiler fails with a warning: warning: <span style="font-family: courier;">source release 17 requires target release 17</span>, it doesn't get compiled.</li><li><b>Only option <span style="font-family: courier;">--source</span>:</b> The option --source can be any version but default target would be the version of Java compiler being used.</li><li><b>Only option <span style="font-family: courier;">--target</span>:</b> The option: --target cannot be lower than the version of Java compiler being used because default source would be the version of the compiler being used. A lower target version gets into compiler option source higher, target lower and fails compilation. E.g <span style="font-family: courier;">javac --target=19 HelloJava.java</span> with Java 20 compiler fails with <span style="font-family: courier;">warning: target release 19 conflicts with default source release 20</span> and code doesn't get compiled.</li><li> <b>Only option <span style="font-family: courier;">--release</span>:</b> Strict code check during compilation to make sure that compiled code gets compiled and works on the target version. Also, the byte-code generated runs only on the release specified and higher, but doesn't run on any lower versions.</li><ul><li>Compiling using Java 20, and no --release option or --release=20 results with major version 64 (Java 20).</li><li>Compiling using Java 20 with --release=19 results with major version 63 (Java 19).</li><li>Compiling using Java 20 with --release=17 results with major version 61 (Java 17) and would result with LinkageError when run on lower version other than 17. Works on 17 and higher.</li></ul></ul></div></div><div><h3>The maven-compiler-plugin variations with these options</h3></div><div>Maven, out of the box with no <a href="https://maven.apache.org/plugins/maven-compiler-plugin/index.html" target="_blank">maven-compiler-plugin</a> specified in <span style="font-family: courier;">pom.xml</span>, has the following variations with it's compiler version option properties (maven.compiler.source, maven.compiler.target and maven.compiler.release).</div><div><br /></div>
<div class="code">
<properties>
<maven.compiler.source>20</maven.compiler.source>
<maven.compiler.target>20</maven.compiler.target>
<maven.compiler.release>20</maven.compiler.release>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
</div>
<div><ul style="text-align: left;"><li><b>No version properties are specified: </b>(No <span style="font-family: courier;">maven.compiler.source</span>, <span style="font-family: courier;">maven.compiler.target</span> and <span style="font-family: courier;">maven.compiler.release</span> properties): Build fails with compilation ERRORS: <span style="color: red;">Source option 5 is no longer supported. Use 8 or later.</span> and <span style="color: red;">Target option 5 is no longer supported. Use 8 or later.</span></li><li><b>All 3 <b>version properties are</b> specified: </b>The option: maven.compiler.release is ignored, it can be any junk. Only source and target properties matter. <b>The value for target option cannot be less than the source</b>. For instance, source 20, target 19 fails build with warning: source release 20 requires target release 20.</li><li><b>Only source </b><b>version property is specified</b>: When only source is specified, <b>target must also be specified</b>. Otherwise, maven build fails with: Fatal error compiling: warning: source release 20 requires target release 20</li><li><b>Only target <b><b>version property</b></b> is specified</b>: When only target is specified, <b>source must also be specified</b>. Otherwise, maven build fails with: <span style="color: red;">Source option 5 is no longer supported. Use 8 or later.</span></li><li><span>The release property specified: When release is specified and is 8 or later, this takes the precedence. <span style="background-color: #fcff01;"><b>Make a special NOTE of it.</b> When <span style="font-family: courier;">release</span> is specified, it takes the precedence and <span style="font-family: courier;">source</span>, <span style="font-family: courier;">target</span> options are ignored. In this case, any non-sense value will make the build work and code gets compiled for the release version specified. But this must be 8 or later.</span></span></li></ul><div>With <a href="https://maven.apache.org/plugins/maven-compiler-plugin/index.html" target="_blank">maven-compiler-plugin</a> specified in <span style="font-family: courier;">pom.xml</span>, has the following variations with these options specified in the plugin configuration (<span style="font-family: courier;"><configuration></span>): </div></div><div><br /></div>
<div class="code">
<properties>
<maven.compiler.source>20</maven.compiler.source>
<maven.compiler.target>20</maven.compiler.target>
<maven.compiler.release>20</maven.compiler.release>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
...
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.11.0</version>
<configuration>
<source>20</source>
<target>20</target>
<release>20</release>
</configuration>
</plugin>
</plugins>
</build>
</div>
<br />With the above way of having both defined set of properties, and maven-compiler-plugin configuration, the configuration values override the property values defined. If no configuration is specified for the maven-compiler-plugin, it uses the set of properties defined. With configuration <source>, <target>, and <release> taking the precedence, the variations are as follows:<br /><ul><li><b>No <configuration> specified for the plugin, but set of properties are defined: </b>It uses defined set of properties if exist, release takes the precedence over target and source.</li><li><b>No <b>properties are defined, and no </b><configuration> is specified for the plugin : </b>It defaults to target 1.8.</li><li><b>All 3 <b>configuration options are</b> specified: </b>The release configuration takes the precedence and is built for the release version.</li><li><b>No properties are set, and only source </b><b>configuration option is specified</b>: When only source is specified, <b>target must also be specified</b>. Otherwise, maven build fails with: Fatal error compiling: warning: source release 20 requires target release 20</li><li><b>No properties are set, and only target </b><b>configuration option is specified</b>: Code gets compiled for the target specified.</li><li><b>No properties are set, and both target and release </b><b>configuration options are specified</b>: The release option takes the precedence and code gets compiled for the release specified.</li></ul><div><h3 style="text-align: left;">Summary</h3><div>For Java 9 and after, use <span style="font-family: courier;">--release</span> option.</div><div>For older versions prior to Java 9 use <span style="font-family: courier;">--source</span> and <span style="font-family: courier;">--target</span> options.</div></div><div><br /></div><div><h3 style="text-align: left;">TIPS</h3><div><ul style="text-align: left;"><li>Use <a href="https://sdkman.io/" target="_blank">SDKMAN</a> to install multiple Java versions and easily switch between different versions.</li><li>If You see noisy warning: <span style="font-family: courier;"><b>Using deprecated '-debug' fallback for parameter name resolution. Compile the affected code with '-parameters' instead or avoid its introspection:</b></span>, then add compiler argument <span style="font-family: courier;"><b><arg>-parameters</arg></b></span> to the maven-compiler-plugin configuration as shown below:</li></ul></div>
<div class="code">
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven-compiler-plugin.version}</version>
<configuration>
<source>${javac.source.version}</source>
<target>${javac.target.version}</target>
<release>${javac.release.version}</release>
<compilerArgs>
<arg>-Xlint:all</arg>
<span style="background-color: #fcff01;"><arg>-parameters</arg></span>
</compilerArgs>
</configuration>
</plugin>
</div>
<br />
<h3 style="text-align: left;">References</h3><div style="text-align: left;"><ul style="text-align: left;"><li><a href="https://maven.apache.org/plugins/maven-compiler-plugin/" target="_blank">Maven Compiler Plugin</a></li><li><a href="https://maven.apache.org/plugins/maven-compiler-plugin/examples/set-compiler-source-and-target.html" target="_blank">Maven - setting compiler source target versions</a></li><li><a href="https://maven.apache.org/plugins/maven-compiler-plugin/examples/set-compiler-release.html" target="_blank">Maven - setting compiler release target version</a></li><li><a href="https://javaalmanac.io/bytecode/versions/" target="_blank">Java bytecode versions</a></li><li><a href="https://openjdk.org/jeps/247" target="_blank">The --release option</a></li><li><a href="https://www.baeldung.com/java-compiler-release-option" target="_blank">What Is the –release Option in the Java 9 Compiler?</a></li><li><a href="https://docs.oracle.com/javase/specs/jvms/se20/html/jvms-4.html" target="_blank">Java class File Format</a></li><li><a href="https://docs.oracle.com/en/java/javase/20/index.html" target="_blank">Java 20 Home</a></li></ul></div></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-32128075691549129702023-01-26T15:25:00.010-05:002023-01-27T18:42:45.415-05:00Spring Boot - WebTestClient Gotcha . . .<h3 style="text-align: left;">The Scenario</h3><div style="text-align: left;">Recently I had a scenario to write an integration test in a Spring Boot micro-service (say <span style="font-family: courier;">MyService</span>) which hosts it's own data in it's own PostgreSQL database in order to verify it's data against another Spring Boot micro-service which hosted master data in it's own database. The database contains a specific entity data set (say <span style="font-family: courier;">Item</span>), a fixed set of items, used for a specific purpose. Another micro-service (say <span style="font-family: courier;">OtherService</span>) uses one of the end-points of <span style="font-family: courier;">MyService</span> by passing entity ids (<span style="font-family: courier;">Item</span> ids) for some processing that <span style="font-family: courier;">MyService</span> is capable of. The <span style="font-family: courier;">OtherService</span> hosts master data of those Entities (<span style="font-family: courier;">Item</span>s) in its own database. But, the data hosted for each entity by both services is entirely different, and each has its own business with it's own data set. Only ids are common. <span style="font-family: courier;">OtherService</span> data is considered master data. So, a <span style="font-family: courier;">MyService</span> integration test requires to make sure that there are no entities missing in it's database. <span style="font-family: courier;">OtherService</span>, offers an end-point to get it's hosted <span style="font-family: courier;">Item</span> entities.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 17, Spring Boot 3.0.2, maven 3.8.5 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div><div style="text-align: left;">So, <span style="font-family: courier;">WebTestClient</span> seemed like better choice to leverage to make an API request and get the master list of <span style="font-family: courier;">Item</span>s and check it's database to see if the count of <span style="font-family: courier;">Item</span>s and list of <span style="font-family: courier;">Item</span> ids match.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">In <span style="font-family: courier;">MyService</span>, there was an integration test already in place written to check most of it's lookup like database entities; one test method for testing each entity's data set. This was a natural integration test to extend by adding another integration test method for testing <span style="font-family: courier;">Item</span> entity set. But this one goes beyond it's database, out to <span style="font-family: courier;">OtherService,</span> making an API request to to get the master list of <span style="font-family: courier;">Item</span>s to verify against.</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">The Issue</h3><div style="text-align: left;">Simply Auto-wiring <span style="font-family: courier;">WebTestClient</span> like:</div>
<div class="code">@Autowired
privateWebTestClient webTestClient;
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">broke the auto-configuration by the following exception:</div>
<div class="code">org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'com.hmhco.sgm.scoring.api.persistence.LookupRepositoriesDataIT': Unsatisfied dependency expressed through field 'webTestClient'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.springframework.test.web.reactive.server.WebTestClient' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.springframework.test.web.reactive.server.WebTestClient' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
</div>
<div style="text-align: left;"><br /></div><h3>The Solution</h3><div style="text-align: left;">A quick Googling suggested to try the following, <b>which actually worked</b> (<b>NOTE</b>: <span style="font-family: courier;">WebTestClient</span> requires <span style="font-family: courier;">org.springframework.boot:spring-boot-starter-webflux</span> dependency which we already had in <span style="font-family: courier;">MyService</span> app <span style="font-family: courier;">pom.xml</span>:</div>
<div class="code">@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">But this solution brings up the embedded web server (Tomcat) starting it at a random port which is actually unnecessary for this test. All I need is an instance of <span style="font-family: courier;">WebTestClient</span> to make a HTTP Get request to <span style="font-family: courier;">OtherService</span> end-point.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Here is the code snippet for this solution:</div>
<div class="code">@SpringBootTest<span style="background-color: #fcff01;">(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)</span>
@ActiveProfiles("test")
@RunWith(SpringRunner.class)
public class MyServiceDataIT {
@Value("${spring.other-service-api.get-Items-end-point}")
private String apiEndPoint;
@Value("${spring.other-service-api.trusted_token}")
private String trustedToken;
<span style="background-color: #fcff01;">@Autowired
private WebTestClient webTestClient;</span>
...
@Test
public void itemLookup_has_no_missing_items() throws Exception {
// given: request spec
var requestSpec = webTestClient.get()
.uri(apiEndPoint)
.header(HttpHeaders.ACCEPT, MediaType.APPLICATION_JSON_VALUE)
.header(HttpHeaders.AUTHORIZATION, trustedToken);
// expect: end-point used to get items, succeeds and get JSON response body
var response = requestSpec.exchange()
.expectStatus().isOk()
.expectBody(String.class)
.returnResult()
.getResponseBody();
// and: convery JSON response to List
List<Map<String, String>> items = new ObjectMapper().readValue(response, List.class);
// verify: items size matches with what we have in database
...
// verify: item ids match with what we have in database
...
}
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><h3>Little more Research</h3><div style="text-align: left;">But a follow-up reading of <a href="https://docs.spring.io/spring-boot/docs/2.6.3/reference/htmlsingle/#features.testing.spring-boot-applications" target="_blank">Spring Boot Documentation</a> made me dig bit deeper. Actually, I did not need a full web environment to be up with embedded servers started and listening on random port that the above change brings in, which also satisfied auto-configuration need for <span style="font-family: courier;">WebTestClient</span>.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Spring Boot auto configuration has a set of annotations with which you can pick and chose the ones that you really need. Apparently there is one for <span style="font-family: courier;">WebTestClient</span>, <span style="font-family: courier;">@AutoConfigureWebTestClient</span>. That sounded like the way to go instead of making having the complete test web environment up and running.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">So, thought the following would work, but it also failed with the above, same exception:</div>
<div class="code">@SpringBootTest
@AutoConfigureWebTestClient
@AutoConfigureWebFlux
@ActiveProfiles("test")
@RunWith(SpringRunner.class)
public class MyServiceDataIT {
@Autowired
private WebTestClient webTestClient;
...
}
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">After little more investigation, I found that we do have dependency: <span style="background-color: white; color: #080808; font-family: "JetBrains Mono", monospace; font-size: 9.8pt;">spring-boot-starter-hateoas</span> that <a href="https://docs.spring.io/spring-boot/docs/2.5.x/reference/htmlsingle/#features.developing-web-applications.spring-mvc.spring-hateoas" target="_blank">Spring Boot Documentation</a> clearly warns on this saying it is meant to be specifically for Spring MVC and should not be used with WebFlux. The alternative in this case is to use: org.springframework.hateoas:spring-hateoas instead. Well, tried that too with the above annotations, but ended up with the same exception.</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">Better and cleaner solution</h3><div style="text-align: left;">The better and cleaner solution is not to have the embedded web server started, but WebTestClient instance created to make a HTTP Get request. So the obvious solution is to ditch dependency injection for WebTestClient and create an instance.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The following is the code snippet:</div>
<div class="code"><span style="background-color: #fcff01;">@SpringBootTest</span>
@ActiveProfiles("test")
@RunWith(SpringRunner.class)
public class MyServiceDataIT {
@Value("${spring.other-service-api.get-Items-end-point}")
private String apiEndPoint;
@Value("${spring.other-service-api.trusted_token}")
private String trustedToken;
...
@Test
public void itemLookup_has_no_missing_items() throws Exception {
// given: web test client
<span style="background-color: #fcff01;">WebTestClient webTestClient = WebTestClient</span>
.bindToServer()
.baseUrl(apiEndPoint)
.build();
// and: request spec
var requestSpec = webTestClient
.get()
.header(HttpHeaders.ACCEPT, MediaType.APPLICATION_JSON_VALUE)
.header(HttpHeaders.AUTHORIZATION, trustedToken);
// expect: end-point used to get items, succeeds and get JSON response body
var response = requestSpec.exchange()
.expectStatus().isOk()
.expectBody(String.class)
.returnResult()
.getResponseBody();
// and: convery JSON response to List
List<Map<String, String>> items = new ObjectMapper().readValue(response, List.class);
// verify: items size matches with what we have in database
...
// verify: item ids match with what we have in database
...
}
</div>
<div style="text-align: left;"><br /></div><h3>TIP</h3><div>Sometimes, WebTestClient times out if the response takes more time than the default value 5000 milliseconds (5 seconds) by throwing the following exception:</div>
<div class="code">java.lang.IllegalStateException: Timeout on blocking read for 5000 MILLISECONDS
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:123)
at reactor.core.publisher.Mono.block(Mono.java:1734)
</div>
<div><br /></div><div>This timeout can be configured in two ways.</div><div><br /></div><div>1. If WebTestClient is @Autowired with web server starting on a random port, by using annotation @AutoConfigureWebTestClient as shown below:</div>
<div class="code">@SpringBootTest<span style="background-color: #fcff01;">(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)</span>
<span style="background-color: #fcff01;">@AutoConfigureWebTestClient(timeout = "10000") // millis, 10 seconds</span>
@ActiveProfiles("test")
@RunWith(SpringRunner.class)
public class MyServiceDataIT {
@Value("${spring.other-service-api.get-Items-end-point}")
private String apiEndPoint;
@Value("${spring.other-service-api.trusted_token}")
private String trustedToken;
<span style="background-color: #fcff01;">@Autowired
private WebTestClient webTestClient;</span>
...
}
</div>
<div><br /></div><div>2. If WebTestClient is NOT @AutoWired and the embedded web server is not started, then by setting response timeout while building WebTestClient instance as shown below:</div>
<div class="code"> @Test
public void itemLookup_has_no_missing_items() throws Exception {
// given: web test client
<span style="background-color: #fcff01;">WebTestClient webTestClient = WebTestClient</span>
.bindToServer()
.baseUrl(apiEndPoint)
.build()
<span style="background-color: #fcff01;">.mutate()</span>
<span style="background-color: #fcff01;">.responseTimeout(Duration.ofSeconds(10))</span>
<span style="background-color: #fcff01;">.build()</span>;
...
}
</div>
<div><br /></div><h3 style="text-align: left;">Conclusion</h3><div style="text-align: left;">These days, most of the times, solutions can be found by googling or on stackoverflow. But, sometimes a clea(ne)r solution takes time to explore and try out.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">A software developer's life is never easy, it's always challenging by simple things that look simple on the surface, yet complex under the hoods. Spring Boot framework is no exception in this regard ;)</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Here is the link to GitHub repo that contains example code for two different integration test-cases of the two solutions mentioned in this post to try out: <a href="https://github.com/gpottepalem/spring-boot-gotchas" target="_blank">spring-boot-gotchas GitHub repo</a></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><h3>References</h3><div><ul><li><a href="https://docs.spring.io/spring-boot/docs/3.0.2/reference/htmlsingle/#features.testing.spring-boot-applications" target="_blank">Testing Spring Boot Applications</a></li></ul></div></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-56704881471823801392022-09-22T14:04:00.050-04:002024-03-03T18:52:02.592-05:00Enums - all the way to persistence . . .<div style="text-align: left;">In any application there would be a need for pre-defined ordered set of constants. Enum is a data type good for such cases. Java has <span style="font-family: courier;">enum</span> since 1.5. Databases do support <span style="font-family: courier;">enum</span> type and PostgreSQL has a special support for this since release 8.3. JPA and Spring Data is a good match to use in modern Java applications for persistence, especially in Spring Boot applications. </div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 17, Spring Boot 2.6,7</span><span style="font-family: "courier new" , "courier" , monospace;"> </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Example Scenario -</b> A persistable entity object in a Spring Boot micro-service application with JPA and PostgreSQL DB.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>DDL Script</b></div><div style="text-align: left;">
<div class="code">-- create enum type genders
CREATE TYPE genders AS ENUM(
'MALE', 'FEMALE'
);
-- create people table
CREATE TABLE people(
id VARCHAR(36) PRIMARY KEY,
first_name VARCHAR(50) NOT NULL,
last_name VARCHAR(50) NOT NULL,
gender genders NOT NULL
);
-- Unique Constraints
ALTER TABLE people
ADD CONSTRAINT people_fname_lname_uk UNIQUE (first_name, last_name);
</div>
</div><div style="text-align: left;"><a href="https://dbfiddle.uk/9XbWLnCf" target="_blank">DB Fiddle</a></div><div style="text-align: left;"><b><br /></b></div><div style="text-align: left;"><b>Maven dependencies: </b><span style="font-family: courier;">pom.xml</span></div>
<div class="code"> ...
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.6.3</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-validation</artifactId>
</dependency>
<dependency>
<groupId><span style="background-color: #fcff01;">com.vladmihalcea</span></groupId>
<artifactId><span style="background-color: #fcff01;">hibernate-types-55</span></artifactId>
<version>2.16.2</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.20</version>
<scope>provided</scope>
</dependency>
...
</dependencies>
...
</div>
<div style="text-align: left;"><b><br /></b></div><div style="text-align: left;"><b>Enum:</b> <span style="font-family: courier;">Gender.java</span></div><div style="text-align: left;"><span style="font-family: courier;">
<div class="code">import lombok.AllArgsConstructor;
@AllArgsConstructor
public enum Gender {
MALE("Male"), FEMALE("Female");
String genderName;
}
</div>
</span></div><div style="text-align: left;"><b><br /></b></div><div style="text-align: left;"><b>Domain Object:</b> <span style="font-family: courier;">Person.java</span></div><div style="text-align: left;">
<div class="code">import com.vladmihalcea.hibernate.type.basic.PostgreSQLEnumType;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
import org.hibernate.annotations.Type;
import org.hibernate.annotations.TypeDef;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.Table;
import javax.persistence.UniqueConstraint;
import javax.validation.constraints.NotNull;
import java.util.UUID;
@Entity
@Table(
name = "people",
uniqueConstraints = {
@UniqueConstraint(
columnNames = {"firstName", "lastName"},
name = "people_fname_lname_uk"
)
}
)
@TypeDef(
name = "pgsql_enum",
typeClass = PostgreSQLEnumType.class
)
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class Person {
@Id
@GeneratedValue(strategy = GenerationType.AUTO)
@Column(length = 36, nullable = false, updatable = false)
@Type(type="org.hibernate.type.UUIDCharType")
private UUID id;
@NotNull
String firstName;
@NotNull
String lastName;
@NotNull
@Enumerated(EnumType.STRING)
@Type(type = "pgsql_enum")
Gender gender;
}
</div>
</div><div style="text-align: left;"><b><br /></b></div><div style="text-align: left;"><b>JPA Repository:</b> <span style="font-family: courier;">PersonRepository.java</span></div><div style="text-align: left;">
<div class="code">import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Query;
import java.util.UUID;
public interface PersonRepository extends JpaRepository<Person, UUID> {
Person findByFirstNameAndLastNameAndGender(String firstName, String lastName, Gender gender);
@Query(value = "SELECT * FROM people WHERE first_name = :firstName AND last_name = :lastName AND gender = CAST(:#{#gender.name()} as genders)", nativeQuery = true)
Person findByFirstNameAndLastNameAndGenderNQ(String firstName, String lastName, Gender gender);
}
</div>
</div><div style="text-align: left;"><b><br /></b></div><div style="text-align: left;"><b>JPA Repository Test:</b> <span style="font-family: courier;">PersonRepositoryIT.java</span></div><div style="text-align: left;">
<div class="code">import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.jdbc.AutoConfigureTestDatabase;
import org.springframework.boot.test.autoconfigure.orm.jpa.DataJpaTest;
import org.springframework.boot.test.autoconfigure.orm.jpa.TestEntityManager;
import org.springframework.test.context.ActiveProfiles;
import org.springframework.test.context.junit4.SpringRunner;
import java.util.List;
import static org.assertj.core.api.Assertions.assertThat;
@ActiveProfiles("test")
@RunWith(SpringRunner.class)
@DataJpaTest
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
public class PersonRepositoryIT {
@Autowired
TestEntityManager testEntityManager;
@Autowired
PersonRepository personRepository;
@Test
public void findBy_returns_expected() {
// given: data in DB
List.of(
Person.builder().firstName("Giri").lastName("Potte").gender(Gender.MALE),
Person.builder().firstName("Boo").lastName("Potte").gender(Gender.MALE)
).forEach(testEntityManager::persist);
testEntityManager.flush();
// expect:
assertThat(
personRepository.findByFirstNameAndLastNameAndGender("Giri", "Potte", Gender.MALE)
).isNotNull()
.extracting("firstName", "lastName", "gender")
.isEqualTo(List.of("Giri", "Potte", Gender.MALE));
// expect:
assertThat(
personRepository.findByFirstNameAndLastNameAndGenderNQ("Boo", "Potte", Gender.MALE)
).isNotNull()
.extracting("firstName", "lastName", "gender")
.isEqualTo(List.of("Boo", "Potte", Gender.MALE));
}
}
</div>
</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">TIP</h3><div style="text-align: left;">From Database side, sometimes, it's handy to have some SQLs dealing with enum type to create, drop, list values, add a new value, modify existing value etc.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">I was just fiddling with PostgreSQL enum type and the following is the link:</div><div style="text-align: left;"><a href="https://dbfiddle.uk/dAIr6GmG" target="_blank">PostgreSQL - enum - DB Fiddle</a> </div><div style="text-align: left;"><a href="https://dbfiddle.uk/2_cq2exP" target="_blank">PostgreSQL - Find Max - DB Fiddle</a></div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">References</h3><div style="text-align: left;"><ul style="text-align: left;"><li><a href="https://www.baeldung.com/jpa-persisting-enums-in-jpa" target="_blank">Persisting Enums in JPA</a></li><li><a href="https://stackoverflow.com/questions/851758/java-enums-jpa-and-postgres-enums-how-do-i-make-them-work-together" target="_blank">Java enums, JPA, PostgreSQL enums - make them work together - Stackoverflow </a></li><li><a href="https://www.baeldung.com/spring-data-derived-queries" target="_blank">Derived method queries in Spring Data JPA Repositories</a></li><li><a href="https://github.com/vladmihalcea/hibernate-types" target="_blank">Hibernate Types</a></li><li><a href="https://spring.io/blog/2014/07/15/spel-support-in-spring-data-jpa-query-definitions" target="_blank">SpEL support in Spring Data JPA @Quey Definitions</a></li><li><a href="https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#jpa.query-methods.at-query" target="_blank">Spring Data JSP @Query documentation</a></li></ul></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-16542366473661514942022-09-17T09:42:00.007-04:002023-01-26T16:12:25.700-05:00Do more with less - the way to write code, what Groovy taught me . . .<div style="text-align: left;">I was not happy at all going back to verbose native Java, leaving my years of happy Groovy and Grails development on JVM. The shrinking market for Groovy pushed me out, but I am not away from it. I still stay with it and write Groovy code on any given day. At least on the side, when I explore and experience some of new Java language features along the way. One question always comes to my mind is - how did we do it in Groovy, why didn't we have to worry about this obvious expectations. The comparison always pleasantly proves that Groovy stood by it's literal word-meaning- very pleasant, to work with, and a very well thought-out and super consistent language on JVM right from day-1.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 17, G</span><span style="font-family: "courier new" , "courier" , monospace;">roovy 4.0.2 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><br /></div><div style="text-align: left;">In my recent Java code, I was dealing with a list of request objects that come in a specific order, and had to give the response objects back in the same order after doing some internal processing that included database query for the list by ids with Spring Data's <span style="font-family: courier;">findAllBy</span> conventional interface. In between I built a map from the request list for faster lookup of specific object's request details. I wanted to retain the order in the map and then the Java's functional feature threw me out to know some internals and come back before take things granted for to use it.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The following is a simple code snippet comparing both Java and Groovy features in the same Groovy script. That's another beauty of Groovy, you can write both Java and Groovy code in one class file.</div><div style="text-align: left;"><br /></div>
<div class="code">import java.util.stream.Collectors
// a data item
record Item(Integer id, String name){}
//groovy
List<item> items = [
new Item(1, 'One'), new Item(2, 'Two'), new Item(3, 'Three'),
new Item(4, 'Four'), new Item(5, 'Five'), new Item(6, 'Six'),
new Item(7, 'Seven'), new Item(8, 'Eight'), new Item(9, 'Nine'),
new Item(10, 'Ten'),
]
println "Groovy\n======"
println "Items[id:Integer, name:String]: ${items}"
def itemsMap = items.collectEntries{[it.name(), it.id()]}
println "Items map by name (order preserved): ${itemsMap}"
println "Items map keys (order preserved): ${itemsMap.keySet()}"
// java
List<item> itemsJava = List.of(
new Item(1, "One"), new Item(2, "Two"), new Item(3, "Three"),
new Item(4, "Four"), new Item(5, "Five"), new Item(6, "Six"),
new Item(7, "Seven"), new Item(8, "Eight"), new Item(9, "Nine"),
new Item(10, "Ten")
);
System.out.println("Java\n====");
System.out.println("Items[id:Integer, name:String]:" + itemsJava);
var itemsMapJava = itemsJava.stream()
.collect(
Collectors.toMap(
item -> item.name,
item -> item.id
)
);
System.out.println("Items map by name (order NOT preserved): " + itemsMapJava);
System.out.println("Items map keys (order NOT preserved): " + itemsMapJava.keySet().stream().toList());
var itemsMapOrderPreserved = itemsJava.stream()
.collect(
Collectors.toMap(
item -> item.name,
item -> item.id,
(key1, key2) -> key1, // key conflict resolver
LinkedHashMap::new // pass the underlying map implementation you want, to preserve the order, defaults to HashMap
)
);
System.out.println("Items map by name (order preserved): " + itemsMapOrderPreserved);
System.out.println("Items map keys (order preserved): " + itemsMapOrderPreserved.keySet().stream().toList());
</item></item></div>
<br />
<div style="text-align: left;">The output:</div><br />
<div class="code">Groovy
======
Items[id:Integer, name:String]: [Item[id=1, name=One], Item[id=2, name=Two], Item[id=3, name=Three], Item[id=4, name=Four], Item[id=5, name=Five], Item[id=6, name=Six], Item[id=7, name=Seven], Item[id=8, name=Eight], Item[id=9, name=Nine], Item[id=10, name=Ten]]
Items map by name (order preserved): [One:1, Two:2, Three:3, Four:4, Five:5, Six:6, Seven:7, Eight:8, Nine:9, Ten:10]
Items map keys (order preserved): [One, Two, Three, Four, Five, Six, Seven, Eight, Nine, Ten]
Java
====
Items[id:Integer, name:String]:[Item[id=1, name=One], Item[id=2, name=Two], Item[id=3, name=Three], Item[id=4, name=Four], Item[id=5, name=Five], Item[id=6, name=Six], Item[id=7, name=Seven], Item[id=8, name=Eight], Item[id=9, name=Nine], Item[id=10, name=Ten]]
Items map by name (order NOT preserved): [Eight:8, Five:5, Six:6, One:1, Four:4, Nine:9, Seven:7, Ten:10, Two:2, Three:3]
Items map keys (order NOT preserved): [Eight, Five, Six, One, Four, Nine, Seven, Ten, Two, Three]
Items map by name (order preserved): [One:1, Two:2, Three:3, Four:4, Five:5, Six:6, Seven:7, Eight:8, Nine:9, Ten:10]
Items map keys (order preserved): [One, Two, Three, Four, Five, Six, Seven, Eight, Nine, Ten]
</div>
<div style="text-align: left;"><br /></div><h3 style="text-align: left;">Conclusion</h3><div>Java started to evolve, changing faster than it used to be and faster than the community can catch up- all for good. It's slowly getting to be less verbose. Still, certain obvious expectations are not so obvious in Java.</div><div><br /></div><div>Groovy shined next to legacy Java, still shines next to modern Java, on the JVM.</div><div><br /></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-48417074687392707972022-05-25T16:21:00.010-04:002023-01-20T08:03:27.725-05:00Maven - finer control of running tests . . .<div style="text-align: left;">There should always be something new to learn everyday. Otherwise, life is boring. In software development, it is even more boring if we don't explore to learn anything new.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Maven is popular open source build tool but comes with a price tag of "Time", consumes much of your time - the most precious of all. I've just leaned a better way to take a finer control of running specific unit/integration test. Two years ago, when I was back to Maven-Java world from Gradle-Groovy/Grails world, I had to do quite a bit of exploration. I still end up doing now and then even after a couple of years. Welcome to the Maven world ;)</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 18, maven 3.8.5 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div><div style="text-align: left;"><a href="https://maven.apache.org/surefire/maven-surefire-plugin/index.html" target="_blank">Surefire</a> and <a href="https://maven.apache.org/surefire/maven-failsafe-plugin/index.html" target="_blank">Failsafe</a> are maven plugins used for unit and integration tests. Out-of-the-box, with no additional configuration, you can simply skip running all tests by passing the command line argument to define the property skipTests like : <span style="font-family: courier;">-DskipTests</span>. This skips all tests, both unit and integration which is also same setting it to true: <span style="font-family: courier;">-DskipTests=true</span>. So, specifying or specifying by setting it with <span style="font-family: courier;">true</span> is same. By default, this property is set with <span style="font-family: courier;">false</span>. So, not specifying at all on the command line, or specifying it with false like: <span style="font-family: courier;">-DskipTests=false</span> is same.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">With Surefire and Failsafe plugins and no additional configurations, tests can be run like shown( assuming that we have a multi-module project with <span style="font-family: courier;">my-service-api</span> as a module and running maven commands for the api module from the root project) below:</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Run all unit tests:</b></div><div style="text-align: left;"><span style="font-family: courier;"> ./mvnw -pl my-service-api clean test</span></div><div style="text-align: left;"><b>Run specific unit test:</b></div><div style="text-align: left;"><span style="font-family: courier;"> ./mvnw -pl my-service-api clean test -Dtest=MyService1</span></div><div style="text-align: left;"><b>Run specific unit test method:</b></div><div style="text-align: left;"><span style="font-family: courier;"> ./mvnw -pl my-service-api clean test -Dtest=MyService1#myServiceTest1</span></div><div style="text-align: left;"><span style="font-family: courier;"><br /></span></div><div style="text-align: left;"><b>Run all integration tests:</b></div><div style="text-align: left;"><span style="font-family: courier;"> ./mvnw -pl my-service-api clean integration-test</span><span style="font-family: courier;"> </span></div><div style="text-align: left;"><b>Run specific integration test:</b></div><div style="text-align: left;"><span style="font-family: courier;"> ./mvnw -pl my-service-api clean integration-test -Dit.test=MyServiceIT1</span></div><div style="text-align: left;"><b>Run Run specific integration test method:</b></div><div style="text-align: left;"><span style="font-family: courier;"> ./mvnw -pl my-service-api clean integration-test </span><span style="font-family: courier;">-Dit.test=MyServiceIT1#method1</span></div><div style="text-align: left;"><br /></div><div style="text-align: left;">The problem with this is, running integration test, runs all unit-tests. If you DO NOT want unit tests to be run, when integration test/tests are run, we need to have a finer control to turn off unit tests.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The following explicit surefire and failsafe plugin configuration by three additional properties will give the finer control with the -DskipTests flag intact.</div><div style="text-align: left;"><br /></div>
<div class="code">
<properties>
...
<!-- For finer control of running tests -->
<span style="background-color: #fcff01;"><skipTests>false</skipTests></span>
<span style="background-color: #fcff01;"><skipUTs>${skipTests}</skipUTs></span>
<span style="background-color: #fcff01;"><skipITs>${skipTests}</skipITs></span>
</properties>
<build>
<plugins>
...
<!-- surefire for unit tests -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>${maven-surefire-plugin.version}</version>
</dependency>
</dependencies>
<configuration>
<skipTests><span style="background-color: #fcff01;">${skipUTs}</span></skipTests>
</configuration>
</plugin>
<!-- failsafe for integration tests -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${maven-failsafe-plugin.version}</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>${maven-surefire-plugin.version}</version>
</dependency>
</dependencies>
<executions>
<execution>
<id>integration-tests</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<argLine>${integrationTestCoverageAgent}</argLine>
<classesDirectory>${project.build.outputDirectory}</classesDirectory>
<includes>
<include>**/*IT.java</include>
</includes>
</configuration>
</execution>
</executions>
<configuration>
<skipTests><span style="background-color: #fcff01;">${skipITs}</span></skipTests>
</configuration>
</plugin>
...
</plugins>
...
</build>
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">With the above three defined additional properties and set to default: <span style="font-family: courier;">false</span> for both the plugins, all tests run out of the box. Now, with their own separate properties defined for unit-tests and integration-tests, unit tests can be set not to run when we want to run specific integration test or all like:</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Run specific integration test, don't run unit tests:</b></div><div style="text-align: left;"><span style="font-family: courier;"> ./mvnw -pl my-service-api clean integration-test <span style="background-color: #fcff01;">-DskipUTs=true</span> -Dit.test=MyServiceIT1</span></div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Run specific integration test, don't run unit tests (same as above, no explicit <span style="font-family: courier;">true</span>):</b></div><div style="text-align: left;"><span style="font-family: courier;"> </span><span style="font-family: courier;">./mvnw -pl my-service-api clean integration-test </span><span style="background-color: #fcff01; font-family: courier;">-DskipUTs</span><span style="font-family: courier;"> -Dit.test=MyServiceIT1</span></div><div style="text-align: left;"><span style="font-family: courier;"><br /></span></div><div style="text-align: left;">But using <span style="font-family: courier;">verify</span> goal instead of <span style="font-family: courier;">integration-test</span> goal is better. Reason in TIP.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Run specific integration test (BETTER)</b>:</div><div style="text-align: left;"><span style="font-family: courier;"> </span><span style="font-family: courier;">./mvnw -pl my-service-api clean </span><span style="background-color: #fcff01; font-family: courier;">verify</span><span style="font-family: courier;"> </span><span style="background-color: #fcff01; font-family: courier;">-DskipUTs</span><span style="font-family: courier;"> -Dit.test=MyServiceIT1</span></div><div style="text-align: left;"><span style="font-family: courier;"><div style="font-family: Times;"><br /></div><div style="font-family: Times;"><b>Run specific integration test (CLEANER)</b>:</div><div style="font-family: Times;"><span style="font-family: courier;"> </span><span style="font-family: courier;">./mvnw -pl my-service-api clean </span><span style="background-color: #fcff01; font-family: courier;">post-integration-test</span><span style="font-family: courier;"> </span><span style="background-color: #fcff01; font-family: courier;">-DskipUTs</span><span style="font-family: courier;"> -Dit.test=MyServiceIT1</span></div><div style="font-family: Times;"><span style="font-family: courier;"><br /></span></div><div style="font-family: Times;"><span style="font-family: courier;"><span style="font-family: Times;">But using </span><span style="font-family: courier;">post-integration-test</span><span style="font-family: Times;"> </span><span style="font-family: Times;">goal instead of</span><span style="font-family: Times;"> </span><span style="font-family: courier;">integration-test</span><span style="font-family: Times;"> </span><span style="font-family: Times;">goal is cleaner. Reason in TIP.</span></span></div><div style="font-family: Times;"><span style="font-family: courier;"><span style="font-family: Times;"><br /></span></span></div></span></div><h3 style="text-align: left;">TIP</h3><div style="text-align: left;">Use <span style="font-family: courier;">verify</span> goal instead of <span style="font-family: courier;">integration-test</span>. If you have Docket container started for DB in integration test phase, it will not be stopped after finishing running tests. This will make your subsequent runs fail to start the container. You will running docker command to remove the container (docker rm -f <postgres-image-name>.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The goal: <span style="font-family: courier;">verify</span> is a way to have cleaner way of executing integration test cases in this case.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The goal <span style="font-family: courier;">verify</span> not only runs integration test case(s) but also verifies code coverage for coverage threshold check. The build might result with FAILURE at the end due to coverage threshold not being met. That is ok, we know that we are excluding unit tests and only running partial integration tests and won't expect coverage threshold to be met. But, it will execute cleanly stopping the containers if started any.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><div>The goal: <span style="font-family: courier;">post-integration-test</span> is another cleaner way to execute integration test cases.</div><div><br /></div><div>Unlike the goal <span style="font-family: courier;">verify</span>, this runs all phases up to post-integration-test (pre-integration-test, integration-test and post-integration-test) leaving out the last verify phase that checks code coverage. This will execute cleanly stopping the containers if started.</div><div><br /></div><div>A typical docker container plugin configurations to start before running integration test(s) and stop after running test(s) looks like:</div><div><br /></div></div>
<div class="code">
<plugin>
<groupId>io.fabric8</groupId>
<artifactId>docker-maven-plugin</artifactId>
<version>0.33.0</version>
<executions>
<execution>
<id><span style="background-color: white;">start</span></id>
<span style="background-color: #fcff01;"><phase>pre-integration-test</phase></span>
<goals>
<span style="background-color: #fcff01;"><goal>start</goal></span>
</goals>
</execution>
<execution>
<id>stop</id>
<span style="background-color: #fcff01;"><phase>post-integration-test</phase></span>
<goals>
<span style="background-color: #fcff01;"><goal>stop</goal></span>
</goals>
</execution>
</executions>
<configuration>
<images>
<image>
<external>
<type>properties</type>
<prefix>postgres.docker</prefix>
</external>
</image>
</images>
</configuration>
</plugin>
</div>
<h3 style="text-align: left;"><br /></h3><h3 style="text-align: left;">References</h3><p></p><ul style="text-align: left;"><li><a href="https://giri-tech.blogspot.com/2021/08/maven-multi-module-java-project-code.html" target="_blank">Maven - multi-module Java project code coverage</a> </li><li><a href="https://stackoverflow.com/questions/6612344/prevent-unit-tests-but-allow-integration-tests-in-maven" target="_blank">Stackoverflow</a></li><li><a href="https://maven.apache.org/surefire/maven-surefire-plugin/examples/skipping-tests.html" target="_blank">Surefire Skipping Tests</a></li><li><a href="https://maven.apache.org/surefire/maven-failsafe-plugin/examples/skipping-tests.html" target="_blank">Failsafe Skipping Tests</a></li><li><a href="https://books.sonatype.com/mvnref-book/reference/running-sect-options.html" target="_blank">Maven command line options</a></li><li><a href="https://maven.apache.org/surefire/maven-failsafe-plugin/" target="_blank">Maven Failsafe plugin lifecycle phases</a></li></ul><p></p>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-74453636349613820192022-05-07T10:25:00.004-04:002022-06-27T10:29:31.591-04:00Keep your Maven builds DRY - leverage placeholder feature in multi-module project for version . . .<div style="text-align: left;">Another Maven blog post in a row, makes me feel like digging into Maven never ends ;). It's an XML world anyway, and requires considerable effort in making any small feature change to work.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">I created a maven multi-module Spring Boot micro-service application a couple of years ago which is a key service for the business. Every time when there is a feature change, or a new feature addition, I always look for opportunities to upgrade tech-stack. Of course, Maven cannot be left behind. I keep upgrading maven wrapper, the tech-stack, and make build scripts better following the <a href="https://en.wikipedia.org/wiki/Don%27t_repeat_yourself#:~:text=%22Don't%20repeat%20yourself%22,data%20normalization%20to%20avoid%20redundancy." target="_blank">DRY</a> programming principle.</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">Problem Context</h3><div style="text-align: left;">The Spring Boot micro-service is a maven multi-module build project with about 4 sub modules (<span style="font-family: courier;">lib</span>, <span style="font-family: courier;">domain</span>, <span style="font-family: courier;">etl</span>, and <span style="font-family: courier;">api</span>). The root module and all sub-modules have semantic <<span style="font-family: courier;">version></span> tag specified. Due to some limitations that I ran into earlier with an older maven version, the semantic <<span style="font-family: courier;">version></span> tag value in all modules was repeated. So, every time when there is a version change, we had to update value in all modules. Our CI environment tags every pull request that gets merged into <span style="font-family: courier;">master</span>/<span style="font-family: courier;">main</span> Git branch by appending timestamp, and git-commit-id to the semantic version tag specified in the build scripts like: <span style="font-family: courier;"><semantic-version><span style="background-color: #fcff01;">-</span><timestamp in YYYYMMddHHmm format><span style="background-color: #fcff01;">.</span><short-commitId></span> (e.g. <span style="font-family: courier;">2.0.1-202205070824.174cd82</span>), thus making every commit a release candidate. However, the <a href="https://semver.org/" target="_blank">semantic version</a> that we specify in maven builds is what we decide to change based on the nature of the feature. </div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 17, Spring Boot 2.5.6, maven 3.8.5 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div><div style="text-align: left;">Starting from 3.5.0 maven started to allow placeholders for versions. A property (e.g. <span style="font-family: courier;">my-version</span>) can be defined in root module with a value and the property can be used as a placeholder for <version> tags in all modules including the root module like: <span style="font-family: courier;"><b><version>${my-version}</version></b></span>. This feature alone might work for a single module project, but doesn't work when you have a multi-module maven build project. An extra plugin is needed for it to work. Otherwise your placeholders in sub-module dependencies are not resolved and replaced with its value and causes errors on instances like when you run any maven goal for a specific sub-module that has root module specified in <span style="font-family: courier;"><parent></span> block. Several blogposts and Stackoverflow question-answer references only talk about this feature with example XML snippets. </div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">Maven Flatten Plugin</h3><div style="text-align: left;">The missing important piece is the <a href="https://www.mojohaus.org/flatten-maven-plugin/index.html" target="_blank">Maven Flatten Plugin</a>. This plugin makes the feature complete. Maven documentation about this feature does talk about this. But due to the nature of today's fast-paced development and not that great Maven's documentation, developers rely more on Stackoverflow and other direct Google hits. I also went through this, but finally ended up reading <a href="https://maven.apache.org/maven-ci-friendly.html" target="_blank">maven documentation</a>, and test trials to make it work.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Following are examples <span style="font-family: courier;">pom.xml</span> snippets of this feature.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Root module's <span style="font-family: courier;">pom.xml</span></div>
<div class="code"><project ...>
<groupId>com.giri.services</groupId>
<artifactId>my-service</artifactId>
<span style="background-color: #fcff01;"><version>${my-service.version}</version></span>
<modules>
<module>my-lib</module>
<module>my-service-domain</module>
<module>my-service-etl</module>
<module>my-service-api</module>
</modules>
<properties>
<span style="background-color: #fcff01;"><my-service.version>2.1.0-SNAPSHOT</my-service.version></span>
...
</properties>
<build>
...
<plugins>
...
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId><span style="background-color: #fcff01;">flatten-maven-plugin</span></artifactId>
<version>1.2.7</version>
<configuration>
<updatePomFile>true</updatePomFile>
<flattenMode>resolveCiFriendliesOnly</flattenMode>
</configuration>
<executions>
<execution>
<id>flatten</id>
<phase>process-resources</phase>
<goals>
<goal>flatten</goal>
</goals>
</execution>
<execution>
<id>flatten.clean</id>
<phase>clean</phase>
<goals>
<goal>clean</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">Sub-module's <span style="font-family: courier;">pom.xml</span></div>
<div class="code"><project ...>
...
<parent>
<groupId>com.giri.services</groupId>
<artifactId>my-service</artifactId>
<version><span style="background-color: #fcff01;">${my-service.version}</span></version>
</parent>
<artifactId>my-service-domain</artifactId>
<packaging>jar</packaging>
...
</project>
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">With this, next time when I want to bump up revision numbers, I only need to change the value in one <span style="font-family: courier;">pom.xml</span> file from the root, unlike 5 earlier. That makes it <a href="https://en.wikipedia.org/wiki/Don%27t_repeat_yourself#:~:text=%22Don't%20repeat%20yourself%22,data%20normalization%20to%20avoid%20redundancy." target="_blank">DRY</a>.</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">TIPS</h3><div style="text-align: left;"><b>IntelliJ inline Error in sub-module's <span style="font-family: courier;">pom.xml</span> file</b></div><div style="text-align: left;">IntelliJ IDEA still complains with an error saying <span style="background-color: #eeeeee; font-family: courier;">Properties in parent definition are prohibited</span> for the inline placeholder in sub-module's <span style="font-family: courier;"><parent></span> block though you set it to use maven wrapper of your app or maven installed on your system which is higher than 3.5.0, or whatever through IDEA preferences for Maven Build.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Simply ignore this. Your build works both inside IDEA or outside from command line.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Extra . files generated</b></div><div style="text-align: left;">Also, notice that there are extra <span style="font-family: courier;">.flattened-pom.xml</span> files generated in root and every sub-module folders. Just let them hang around there.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Avoid conflicting placeholder names</b></div><div style="text-align: left;">I wouldn't use <span style="background-color: #f7f7f9; color: #dd1144; font-family: Monaco, Menlo, Consolas, "Courier New", monospace; font-size: 12px; white-space: nowrap;">${revision} </span>as as the placeholder name for this feature as specified in the maven document when I also have the <a href="http://smartcodeltd.co.uk/release-candidate-maven-plugin/getting-started.html" target="_blank">release candidate plugin</a>. Release candidate plugin has this placeholder name reserved for git-commit-id.</div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">References</h3><div style="text-align: left;"><ul style="text-align: left;"><li><a href="https://maven.apache.org/maven-ci-friendly.html" target="_blank">Maven CI friendly versions</a></li><li><a href="https://www.mojohaus.org/flatten-maven-plugin/index.html" target="_blank">Maven Flatten Plugin</a></li><li><a href="http://smartcodeltd.co.uk/release-candidate-maven-plugin/getting-started.html" target="_blank">Release Candidate Plugin</a></li></ul></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-50766014571450981832022-05-02T13:36:00.014-04:002022-08-09T15:11:19.043-04:00Maven - running Java app . . .<div style="text-align: left;">When <a href="https://maven.apache.org/" target="_blank">Maven</a> is the build tool and you have no choice, you need to factor in additional amount of development time dealing with its build file: the <span style="font-family: courier;">pom.xml</span>. Any new additional feature eats up unexpected time.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 17, maven 3.8.5 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div><div style="text-align: left;">Recently, I had to work on a task that involved writing a new Plain Old Java Application (POJA) with a main method. This application is new addition to the existing half-a-dozen Java applications family residing in a Maven module in a multi-module project. Each application is a kind of ETL app (<b>E</b>xtract, <b>T</b>ransform, and <b>L</b>oad data) with XML, Excel spread-sheets as data sources. The transformed output is <a href="https://github.com/flyway/flyway" target="_blank">Flyway</a> SQL script. Sounds simple! But, not really!!</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The existing applications family has a strong contract with their parent class through Inheritance (OO model) & Copy-and-paste (popular dev-model) for sharing code, inherit about 50 final and non-final Static constants Also, every application's specific run requires changing constant values for every run and checkin latest code changes. Huh, old messy way of maintaining apps. Joining the legacy family, following the inheritance model (to keep the family relationship), at least I wanted to 1) Eliminate Copy-and-paste sharing model 2) Add CLI support to pass in run specific values for constants.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">In Groovy world, no specific library is needed for developing Java apps with CLI support. It comes with <a href="https://docs.groovy-lang.org/latest/html/gapi/groovy/cli/commons/CliBuilder.html" target="_blank">CliBuilder</a>. For Java, googling found me <a href="https://picocli.info/" target="_blank">Picocli</a>. But Maven got in the way giving bit hard time for executing the application. After going through some stackoverflow explanations and my experiments, learnt that two ways of running the app is possible. Both require to leverage the plugin: <a href="https://www.mojohaus.org/exec-maven-plugin/" target="_blank">Exec Maven Plugin</a>.</div><div style="text-align: left;"><span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div><div style="text-align: left;">1. Configure the plugin - if there are multiple applications, multiple executions can be setup. </div><div style="text-align: left;"><br /></div>
<div class="code">
...
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.4.0</version>
<executions>
<!-- mvn compile <span style="background-color: #fcff01;">exec:java@my-etl-app</span> -->
<execution>
<id><span style="background-color: #fcff01;">my-etl-app</span></id>
<goals>
<goal>java</goal>
</goals>
<configuration>
<mainClass><span style="background-color: #fcff01;">com.giri.etl.MyEtlAppPicoCli</span></mainClass>
</configuration>
</execution>
<!-- mvn compile <span style="background-color: #fcff01;">exec:java@my-etl-app-new</span> -->
<execution>
<id><span style="background-color: #fcff01;">my-etl-app-new</span></id>
<goals>
<goal>java</goal>
</goals>
<configuration>
<mainClass><span style="background-color: #fcff01;">com.giri.etl.MyEtlAppNewPicoCli</span></mainClass>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">With this the application can be run by specifying the goal <span style="font-family: courier;">exec:java</span> like:</div><div style="text-align: left;"><span style="font-family: courier;"><div class="code">mvn compile exec:java@my-etl-app</div></span><span style="font-family: arial;"> </span></div><div style="text-align: left;">assuming that the code is compiled and the other dependent modules are built.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">2. Configure the plugin in Profiles - if there are multiple applications, multiple profiles can be setup. </div><div style="text-align: left;"><br /></div>
<div class="code">
...
<profiles>
<!-- mvn <span style="background-color: #fcff01;">exec:java -Pmy-etl-app</span> -->
<profile>
<id><span style="background-color: #fcff01;">my-etl-app</span></id>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.4.0</version>
<configuration>
<mainClass><span style="background-color: #fcff01;">com.giri.etl.MyEtlAppPicoCli</span></mainClass>
</configuration>
</plugin>
</plugins>
</build>
</profile>
<!-- mvn <span style="background-color: #fcff01;">exec:java -Pmy-etl-app-new</span> -->
<profile>
<id><span style="background-color: #fcff01;">my-etl-app-new</span></id>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.4.0</version>
<configuration>
<mainClass><span style="background-color: #fcff01;">com.giri.etl.MyEtlAppNewPicoCli</span></mainClass>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
</div>
<div><br /></div>With this the application can be run by specifying the goal <span style="font-family: courier;">exec:java</span> like:<div><span style="font-family: courier;"><div class="code">mvn compile exec:java -Pmy-etl-app</div></span></div><div>assuming that the code is compiled and the other dependent modules are built.</div><div><br /><h3 style="text-align: left;">TIP-1</h3><div>There is also a standard way of specifying the main application class directly like:</div><div><span style="font-family: courier;"><div class="code">mvn compile exec:java -Dexec.mainClass="<span style="background-color: #fcff01;">com.giri.etl.MyEtlAppPicoCli</span>" -Dexec.args="-y=2022"</div></span></div><div><br /></div><div>To pass application arguments to the Java app when running the maven goal: <span style="font-family: courier;">exec:java</span>, use <span style="font-family: courier;">-Dexec.args</span> like: </div><div><div class="code">mvn compile exec:java -Pmy-etl-app -Dexec.args="-<arg1>=<value1> -<arg2>=<value2>"</div></div><div><br /></div><div>E.g. </div>
<div><span style="font-family: courier;"><div class="code">mvn exec :java -Pmy-etl-app -Dexec.args="-y=2022 -run-extra-checks=false"</div></span></div><div>Assuming that <span style="font-family: courier;">-y</span>, <span style="font-family: courier;">-run-extra-checks</span> are specific CLI arguments added to my app by leveraging <a href="https://picocli.info/" target="_blank">Picocli</a> Framework.</div><div><div style="text-align: left;"><br /></div><h3>TIP-2</h3></div><h4 style="text-align: left;"><a href="https://docs.oracle.com/en/java/javase/18/language/preview-language-and-vm-features.html" target="_blank">Java Preview Feature</a> maven support (both compile-time, and run-time)</h4><div><a href="https://docs.oracle.com/en/java/javase/18/language/preview-language-and-vm-features.html" target="_blank">Java preview features</a> are disabled by default. If any of the preview features are used in the code, it must be enabled explicitly by using the command line option <span style="font-family: courier;">--enable-preview</span> for both compiler and runtime.</div><div><br /></div><div><b>Compile-time</b></div><div>For compile-time support in maven builds, the <span style="font-family: courier;">maven-compiler-plugin</span> must be configured to pass the compiler flag as shown below:</div><div><br /></div>
<div class="code">
</build>
<plugins>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.7.0</version>
<configuration>
<release>${jdk.version}</release>
<source>${jdk.version}</source>
<target>${jdk.version}</target>
<compilerArgs>
<arg>-Xlint:all</arg>
<arg><span style="background-color: #fcff01;">--enable-preview</span></arg>
</compilerArgs>
</configuration>
</plugin>
</plugins>
</build>
</div>
<div><br /></div><div><b>Run-time</b></div><div>With <a href="https://www.mojohaus.org/exec-maven-plugin/" target="_blank">Exec Maven Plugin</a> used for running the Java application, the run-time preview features enabling is also necessary. The above <span style="font-family: courier;">maven-compiler-plugin</span> configuration is only good for compilation. For runtime system in this scenario, a <span style="font-family: courier;">jvm.config</span> file needs to exist in the <span style="font-family: courier;">.mvn</span> folder of the project where you are running the application from.</div><div>The content of this file should be just:</div><div><span style="font-family: courier;">--enable-preview</span></div><div><span style="font-family: courier;"><br /></span></div><h3 style="text-align: left;">
References</h3><div><div><ul style="text-align: left;"><li><a href="https://www.mojohaus.org/exec-maven-plugin/" target="_blank">Exec Maven Plugin</a> </li></ul></div></div></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-59213571157756287722021-12-08T18:23:00.001-05:002021-12-11T05:28:50.994-05:00Maven Dependencies Fix - Need to support both Log4j (ver 1) & Log4j2 (ver 2) but exclude Log4j (ver 1) . . .<div style="text-align: left;">Software development never gets simple. A simple logging in modern Java applications is not simple enough. There are several logging frameworks: <a href="https://docs.oracle.com/javase/8/docs/api/java/util/logging/package-summary.html" target="_blank">java.util.logging</a>, log4j, <a href="https://logging.apache.org/log4j/2.x/" target="_blank">log4j2</a>, <a href="http://www.slf4j.org/" target="_blank">SLF4J</a>, <a href="http://logback.qos.ch/" target="_blank">Logback</a> etc. A typical Java application these days brings in one, or more, or even all of these dependencies. It's often a confusion which one is better, which one to use, which one is in action, which one to configure, and how to configure etc. It becomes worse if you need to exclude any.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">I was recently working on a task to fix Security Vulnerabilities detected and reported by a commercial SaaS tool that scans Java project codebase and it's dependent libraries. The tool scans and generates a report listing vulnerabilities detected by categorizing each into <span style="font-family: courier;">Critical</span>/<span style="font-family: courier;">High</span>/<span style="font-family: courier;">Medium</span>/<span style="font-family: courier;">Low</span>. Obviously, when there is an unpatched dependent library marked <span style="font-family: courier;">Critical</span>, that draws superior-attention and brings in a worry with the word: "hacking"- a scary or not-scary word in Software Engineering, it depends ;).</div><div style="text-align: left;"><br /></div><div style="text-align: left;">This post is NOT about the messy path that Java logging has been going on right from the beginning. It's about dealing with a need to support both <span style="font-family: courier;">log4j</span> (ver 1) and <span style="font-family: courier;">log4j2</span> (ver 2) in a Java project but want to exclude <span style="font-family: courier;">log4j</span> from the dependency list to fix the security vulnerability reported in reference to the <a href="https://nvd.nist.gov/vuln/detail/CVE-2019-17571" target="_blank">specific vulnerability reported on log4j </a>in the <a href="https://nvd.nist.gov/" target="_blank">National Vulnerability Database</a>. </div><div style="text-align: left;"><br /></div><div style="text-align: left;">To deal with this issue, all you ned to do is exclude <span style="font-family: courier;">log4j</span> from the dependency library that depends on it, and add an explicit dependency of <span style="font-family: courier;">log4j2</span> and the <span style="font-family: courier;">log4j1-2 bridge</span>.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">The following is a snipper of maven <span style="font-family: courier;">pom.xml</span>.</div>
<div class="code">
...
<properties>
<log4j.version>2.15.0</log4j.version>
</properties>
<dependencies>
...
<dependency>
<groupId>com.some.lib</groupId>
<artifactId>somelib-need-log4j</artifactId>
<exclusions>
<!--
Fix Security Vulnerability reported with log4j-1.2.17, the last discontinued version of log4j 1.
Exclude log4j 1 and depend on the log4j 1 to 2 bridge and bring in log4j 2 explicitly
-->
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
</exclusions>
</dependency>
<!--
Fix Security Vulnerabily reported with log4j-1.2.17
Log4j 1 to 2 bridge to forward log4j 1 calls to 2
-->
<!-- log4j 1 to 2 bridge -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-1.2-api</artifactId>
<version>${log4j.version}</version>
</dependency>
<!-- log4j 2 -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>${log4j.version}</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>${log4j.version}</version>
</dependency>
...
</dependencies>
...
</div>
<div style="text-align: left;"><br /></div><h3 style="text-align: left;">TIP</h3><div style="text-align: left;">Leverage maven <span style="font-family: courier;">dependency:tree </span>goal to see which dependency's transitive dependencies bring in the library that needs an attention.</div><div style="text-align: left;"><br /></div>
<div class="code">
mvn dependency:tree
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div><h3 style="text-align: left;">References</h3><div style="text-align: left;"><ul style="text-align: left;"><li><a href="https://nvd.nist.gov/vuln/detail/CVE-2019-17571 " target="_blank">Log4j security vulnerability</a></li><li><a href="https://logging.apache.org/log4j/2.x/manual/index.html" target="_blank">Log4j 2</a></li><li><a href="https://logging.apache.org/log4j/1.2/" target="_blank">Log4j (1.x) - end of life</a></li><li><a href="https://logging.apache.org/log4j/2.x/log4j-1.2-api/" target="_blank">Log4j 1.x to 2.x bridge</a></li><li><a href="https://mvnrepository.com/artifact/log4j/log4j" target="_blank">Maven Log4j (1.x) dependency</a> - 1.2.17 is the last version, no updates since 2021. Also got moved to org.apache.logging.log4j » log4j-core, this means a package change and requires code changes.</li><li><a href="https://mvnrepository.com/artifact/org.apache.logging.log4j" target="_blank">Maven log4j 1.2 dependencies</a></li><li><a href="https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-1.2-api" target="_blank">Maven log4j 1 to 2 bridge dependency</a></li></ul></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0tag:blogger.com,1999:blog-13608249.post-37260463510804951042021-10-21T15:32:00.008-04:002022-02-16T14:25:31.731-05:00Spring boot upgrade from 2.2.x to 2.5.x - Spring Cloud Sleuth Zipkin - log message format change . . .<div style="text-align: left;">I recently upgraded a Spring Boot micro-service application from Spring Boot 2.2.11 to 2.5.3 and Java OpenJDK 15 to Java Amazon Corretto 16.</div><div style="text-align: left;"><br /></div><div style="text-align: left;">Upgrading Java was smooth. Some major and breaking changes that I came across from Spring Boot side include:</div><div style="text-align: left;"><ul style="text-align: left;"><li>Profile changes - documented well</li><li>Dependency changes - Obvious</li><li>Spring Cloud Sleuth Zipkin - Undocumented glitch</li></ul></div><div style="text-align: left;">This post is on the Spring Cloud Sleuth Zipkin - undocumented removal of <span style="font-family: courier;">exportable</span> property.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><b>Environment:</b> <span style="font-family: "courier new" , "courier" , monospace;">Java 16, Spring Boot</span><span style="font-family: "courier new" , "courier" , monospace;"> 2.5.3 </span>on <span style="font-family: "courier new" , "courier" , monospace;">macOS Catalina 10.15.7</span></div><div style="text-align: left;"><br /></div><div style="text-align: left;">We have application logs collected, ingested and indexed in <span style="font-family: courier;"><a href="https://www.datadoghq.com/" target="_blank">Datadog</a></span> by <span style="font-family: courier;"><a href="https://www.elastic.co/logstash/" target="_blank">Logstash</a></span> scrapping <span style="font-family: courier;"><a href="http://mesos.apache.org/" target="_blank">Mesos</a></span> application's stdout. Soon after the upgrade, logs stopped showing up in Datadog. After some investigation, figured out that the pattern parser failed to parse log string as it was expecting log string format like: <span style="font-family: courier;">[service, trace, span, exportable]</span>. But the format after the upgrade was: <span style="font-family: courier;">[service, trace, span]</span>. Basically, the last exportable property with value <span style="font-family: courier;">true</span> or <span style="font-family: courier;">false</span> was missing. </div><div style="text-align: left;"><br /></div><div style="text-align: left;">The trace, span, exportable are injected into log messages by Spring Cloud Sleuth and Zipkin for distributed tracing via log framework MD5. The documentation of Spring Cloud Sleuth seems not updated with this details. :(</div><div style="text-align: left;"><br /></div><div style="text-align: left;">In order to get away with this issue on the Datadog, switching to JSON log message format seemed like a better solution as the JSON is better than strict string pattern matchers. This required an explicit <span style="font-family: courier;">logback</span> definition in <span style="font-family: courier;">resources/logback-spring.xml</span> file (yes, XML is the only way) and a new dependency for JSON logging.</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><span style="font-family: courier;">src/main/resources/logback-spring.xml</span></div>
<div class="code">
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<springProperty scope="context" name="serviceName" source="spring.zipkin.service.name"/>
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
<providers>
<pattern>
<pattern>
{
"timestamp": "%d{yyyy-MM-dd HH:mm:ss.SSS}",
"severity": "%level",
"service": "${serviceName:-}",
"pid": "${PID:-}",
"thread": "%thread",
"class": "%logger{40}",
"trace": "%X{traceId:-}",
"span": "%X{spanId:-}",
"parent": "%X{parentId:-}",
"exportable": "%X{sampled:-}",
"logmessage": "%message"
}
</pattern>
</pattern>
<!-- Additional support needed for logging stack trace in JSON message -->
<!-- https://github.com/logfellow/logstash-logback-encoder -->
<stackTrace>
<throwableConverter class="net.logstash.logback.stacktrace.ShortenedThrowableConverter">
<maxDepthPerThrowable>30</maxDepthPerThrowable>
<maxLength>4096</maxLength>
<shortenedClassNameLength>20</shortenedClassNameLength>
<rootCauseFirst>true</rootCauseFirst>
</throwableConverter>
</stackTrace>
</providers>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="console" />
</root>
</configuration>
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">Maven build file, new dependency for JSON log message: <span style="font-family: courier;">pom.xml</span></div>
<div class="code">
<dependency>
<groupId>net.logstash.logback</groupId>
<artifactId>logstash-logback-encoder</artifactId>
<version>6.6</version>
</dependency>
</div>
<div style="text-align: left;"><br /></div><div style="text-align: left;">With this a sample JSON exception log messages looks like:</div><div style="text-align: left;"><br /></div>
<div class="code">
{
"timestamp": "2021-12-22 12:26:26.552",
"severity": "ERROR",
"service": "my-service-api",
"pid": "65623",
"thread": "http-nio-8080-exec-6",
"class": "c.g.s.e.MyServceImpl",
"trace": "529f2d2d7a65dde4",
"span": "529f2d2d7a65dde4",
"parent": "",
"exportable": "",
"logmessage": "My Exception log mesage.",
"stack_trace": "c.g.s.e.MyException: my exception message\n\tat c.g.s.s.MyServiceImpl.serviceMethodOne(MyServiceImpl.java:210)\n\tat c.g.s.s.MyServiceImpl.serviceMethoTwo(MyServiceImpl.java:280)\n"
}
</div>
<br/>
<h3 style="text-align: left;">References</h3><div style="text-align: left;"><ul style="text-align: left;"><li><a href="https://docs.spring.io/spring-cloud-sleuth/docs/3.0.2/reference/htmlsingle/#features-log-integration-json-logback" target="_blank">JSON Logback with Logstash</a></li><li><a href="https://docs.spring.io/spring-cloud-sleuth/docs/3.0.3/reference/htmlsingle/#how-to-see-spans-in-an-external-system" target="_blank">Spring Cloud Sleuth reference Documentation</a></li><li><a href="https://medium.com/swlh/distributed-tracing-in-micoservices-using-spring-zipkin-sleuth-and-elk-stack-5665c5fbecf" target="_blank">Distributed Tracing</a></li></ul></div><div style="text-align: left;"><br /></div>Giridhar Pottepalemhttp://www.blogger.com/profile/06947525071574509288noreply@blogger.com0