Indirection is directly proportional to complexity. The more the indirection levels in one direction, the more the turns to take in reverse direction to understand. The higher the indirection levels, the deeper the dive in finding details.
A simple feature like Spring Profiles interwoven across several Spring frameworks is not only made it difficult to understand but also is painful to follow through multiple, disconnected sets of details, and multiple documents. Pulling necessary details together, across different versions of each framework, related to one specific feature is even more painful.
Environment: Java 20, Spring Boot 2.6.3, Spring Boot 2.7.14, maven 3.9.3 on macOS Catalina 10.15.7
Spring Profiles is a pretty simple to understand indirection, abstracted out and externalized with several benefits. Spring - as a framework to support all possible ways that this feature can be supported, also started to evolve. Cloud and cloud related frameworks extended it further increasing the levels of indirection and thus its complexity.
In my recent attempt upgrading a Spring Boot application from Spring Boot 2.6.x to 2.7.x, failed to get deployed quietly with no ERRORS or WARNINGS in Kubernetes (K8S) environment. I noticed a log message about multiple active profiles in logs, and started digging deeper into why there were more than one active profile. There was certainly an interwoven behavioral change between these two Spring Boot minor versions around active profiles with different frameworks working together on profile related configurations, mainly coming in from bootstrap and application yaml files.
The scenario
I did migrate a Spring Boot 2.6.3 application successfully to 2.7.14 and tested locally with local profile. The application also depends on Spring framework support for Vault, and Consul. The default active profile (spring.profiles.active) is set to test in application.yml file. When the application is run locally an explicit active profile parameter is passed (-Dspring-boot.run.profiles=local) to the maven goal: spring-boot:run. So, profile: local passed through maven build option gets passed as Java option, takes the precedence, and overrides the default active profile test. The default active profile test is good for the default profile active as there is no explicit profile specified during maven test phase when it runs unit and integration tests. Environment specific configuration files like application-int.yml, application-int.yml have spring.config.activate.on-profile set to respective environment. So, environment specific configuration (e.g. application-int.yml) is considered when active profile is set to specific environment (e.g. int). All looked good before and after the upgrade.
When the application gets built through concourse pipeline, after successful build including unit and integration tests run, it automatically gets deployed to both int and cert. The deployment script sets Java option -Dspring.profiles.active=<env>, thus overriding the default active profile test with specific env as command line option takes the precedence over configuration files. So, all worked as expected for local profile after the upgrade and went for deployment with the same expectations.
Also, initially the application had one set of yaml files (application) when Apache Mesos was the deployment platform. Later it got migrated to K8S deployment with added Vault and Consul support, which forced a new set of yaml configuration files (bootstrap). Mainly Vault and Consul required certain properties to be configured through bootstrap yaml set. So after the application got migrated from Apache Mesos to K8S, it ended up having two different sets of application properties: bootstrap.yml set and application.yml set, along with their counterpart env specific files for local, test, int, cert and prod. With that there were 2 sets of 7 yaml files for each set. The first set: bootstrap, bootstrap-local, bootstrap-test, bootstrap-dev, bootstrap-int, bootstrap-cert and bootstrap-prod. The second set: application, application-local, application-test, application-dev, application-int, application-cert and application-prod.
The issue
The upgraded application (upgraded to Spring Boot 2.7.14) failed to come up successfully after it got deployed to K8S cluster through Concourse CI/CD pipeline with logs showing two active profiles during the application startup after deployment onto both int and cert. For instance on int logs, both test and int were logged in as active profiles.
It was bit hard to dig into where this profiles handling of active profile getting overridden by the Java option passed was broken - instead of overriding, it was appended to active profiles. Spring allows more than one profile to be specified to be active separated by comma treating it as a list of active profiles. When multiple profiles are active, each profile specific configuration is read in the order of environments that appear in the list, and configuration properties get consolidated, with the later environment related properties replacing the previous environment related if there are same properties configured in both. In this case, it's like merging multiple environment specific configurations.
In my case, test profile was default active. The active profile passed through Java options, instead of replacing the default list which has just one element test, it was getting prepended to the list. So in int the active profiles were: int, test and similarly in cert: cert, test. The test profile being the last element in the list, was taking the precedence and the application in both int and cert failed to start up as test environment specific configuration was not good for int or cert.
The fix (better one) - separating out test configurations into it's own configuration area under test
All application, and bootstrap yaml files resided under src/main/resources directory.
The fix I did for this issue involved the following changes:
1. Removed default active profile configuration property set in application.yml file.
2. Moved test profile specific configuration files (application-test.yml and bootstrap-test.yml) that contain spring.config.activate.on-profile property set to test along with other test environment related properties under src/test/resources directory.
3. Added new application.yml and bootstrap.yml files under test/resources directory. Both just contain one property spring.profiles.active set to test.
With this separation, all configuration files including the base and environment specific reside under src/main/resources directory. There is no active profile set at all in any of these configuration files under src/main/resources. In other words, active profile is always passed in as an option for all environments: local, dev, int, cert, and prod.
Test environment configuration files: application.yml, bootstrap.yml, application-test.yml, and bootstrap-test.yml reside under src/test/resources directory.
Both application.yml, and bootstrap.yml have active profile set to test. So spring testing framework considers these configurations when found under src/test/resources during unit and integration tests. That way test environment active profile and it's associated test specific configurations are totally isolated from all other env configurations into its own configuration area: src/test/resources.
This is more cleaner approach. Also isolates test configurations under src/test/resources area which are not even bundled into the application jar.
Summary
Enterprise Java's unnecessary overcomplexity gave birth to Spring Framework and it has quickly become popular and a de-facto Java framework since then. The core of it is based on the simple Dependency Injection (or Inversion of Control - IoC) design pattern. It started to grow into every corner of the technology concept. It is not simple anymore, in my opinion. There are too many layers of details one needs to know. With tons and tons Spring-eco-system frameworks, it has become even more complex. A framework started to simplify Enterprise Java development has grown too big to be(come) complex.
A simple Java framework like JUnit itself is getting complex. So, no wonder Spring being in there already. Software developers absolutely love indirection and complexity. In Software Development simplicity is a rare quality. Even if some simplicity exists, it slowly becomes complex over a period of time. Simple becomes complex, complex only becomes super complex ;)
Happy software development, and enjoy the complexity!
Posts
No comments:
Post a Comment