Upgrading Grails-2.2.1 to Grails-3: Static Assets take a BIG move . . .

I recently upgraded a Grails 2.2.1 web-app to Grails 3.2.1. It was a BIG move forward. Moving static assets (JavaScripts, CSS and Images) to their new assets directory, organizing & setting up directives/manifests in their new home directory, making all required changes to views & templates, testing all views for styles & images, and testing views with AJAX functionality involving JavaScripts... overall, it took a considerable amount my time during the whole upgrade efforts.

Following are some key points I have from my efforts:

• Grails 3 doesn't come with Resources plugin and hence you will not have ResourceTagLib in your classpath. If you want, you can probably still use Resources plugin in Grails-3 app, but Asset-Pipeline plugin seems to be the viable option.
• Static assets which can be handled by Asset-Pipeline plugin need a move from web-app directory to grails-app/assets directory, their new home. Files under web-app/css, web-app/js and web-app/images now go into grails-app/assets/stylesheets, grails-app/assets/javascripts and grails-app/assets/images respectively.
• Grails-3 comes with jquery-2.2.0.min.js and bootstrap.js(3.3.6). If your Grails 2.2.1 app was dependent on these, you probably had jquery-1.7.2.min.js and bootstrap.js(2.2.2). If so, you will be better off retaining older versions to start with the upgrade process to eliminate this new variance in upgrading-equation.
• The recommended approach to upgrade Grails 2.x app to Grails 3.x is to first create a new Grails-3 application and start copying all artifacts from old to new locations. Grails-3 documentation's Upgrading section has very well documented details on old and new locations. With respect to static assets, when a new Grails-3 app is created, you will notice images, javascripts and stylesheets sub directories under grails-app/assets. Also, you will have a bunch of static assets already sitting in there. You may have to do some cleanup with these files.
• Modularize your assets
1. AppResources.groovy is Resources plugin’s way of modularizing JavaScripts and CSS files by grouping these static assets into modules. But Asset-Pipeline plugin minifies compresses all JavaScript & CSS files, also enables browser cache, and hence static assets are only served once for all pages. So, it may not be required to group/ modularize static assets. If truly needed, for every module (e.g. module1) in AppResources.groovy, an equivalent manifest/directive with module-name.js (e.g. modul1.js) file can be created listing all it’s dependencies.
2. The directive files application.js and application.css are main manifest files for JavaScripts and CSS.
3. If you have modularized static assets in Grails-2 app, your main static resource AppResources.groovy should be your reference for re-organizing your static assets in Grails-3 app to minimize changes in views and view templates.
4. You can modularize your static assets in Grails-3 the same way as in Grails-2 app with no need for ApprRsources.groovy file but with equivalent module manifests/directives created.
5. Create a one-to-one asset-pipeline directive/manifest file for each of your module defined in AppResources.groovy. For instance, if you have, let's say a common module defined listing all it's dependency resources (both JavaScripts and CSS files), create common.js and common.css asset-pipeline directives that list required JavaScript and CSS dependencies respectively in Grails-3 app.
6. If you have many modules in your application, your grails-app/assets/javascripts and grails-app/assets/stylesheets will get cluttered and mixed with manifest files and actual assets. You will be better off keeping directives separate from actually assets by keeping actual javascript and css assets under grails-app/assets/javascripts/lib and grails-app/assets/stylesheets/shared sub-directories respectively so that asset-pipeline manifest files can be under main grails-app/assets/javascripts and grails-app/assets/stylesheets directories.
• Modify views & view templates and change resources tags to equivalent asset-pipeline tags
1. Replace all <r:script> </r:script> with <asset:script type=”text/javascript”> </asset:script>
2. Replace <r:external file="/static/images/favicon.ico"/> with <asset:link rel='shortcut icon' href="favicon.ico" type="image/x-icon"/>
3. Remove all <r:layoutResources/> in <head></head> and replace all <r:layoutResources/> at the very bottom of layout pages with <asset:deferredScripts/> (This is Asset-Pipeline plugin's equivalent of Resources plugin’s deferring scripts to the bottom of the page)
4. If you have modularized assets in Grails 2.2.1 app, for example, for module 'module1'  replace all <r:require module=”module1”/> with <asset:stylesheet src=”module1”/> and <asset:javascript src=”module1”/> if module1 has both JavaScripts and CSSs in it.
• If there are other static assets like pdf files that are referenced in views by grails resource tag or it's equivalent method call, it can safely be moved from Grails-2's web-app/pdf to Grails-3's grails-app/assets/pdf and be served by Assets-Pipeline plugin. These assets, like images, need no manifest/directive files and it simply works.

Summary

Grails moved away from Resources plugin in favor of Asset-Pipeline plugin starting from 2.4. Upgrading prior versions of Grails 2.4 apps to 3.x certainly requires considerable development and testing efforts with respect to static assets. So, just be prepared for this BIG move.

References

Resources Plugin Docs
Asset-Pipeline Plugin Docs
Asset-Pipeline Plugin - GitHub source code
Grails Team Blog Post on Migrate from Resources Plugin to Asset-Pipeline Plugin
Very nice Introduction to Asset Pipeline Plugin

My previous posts on Upgrading Grails application from 2.2.1 to 3.2.1

• Know test cases run order in Grails-3 app to help solve issues with "test pollution"
• Upgrading Grails-2 application to Grails-3: Spring Security Core Plugin differences
• Upgrading Grails-2 application to Grails-3 - Make static assets available

Know test cases run order in Grails-3 app to help solve issues with "test pollution". . .

In a Grails 3 application that I upgraded recently from Grail 2.2.1 to Grails 3.2.1, it was very puzzling when an integration specification (test case upgraded from Grails 2.2.1) with few feature methods (test case methods) passed locally but failed on the Bamboo CI server. From the kind of failure, it was evident that this particular specification failed due to "data pollution"- some unwanted data hanging around in the database by the time it ran.

The failed specification was an integration test specification and I was under the impression when grails test-app command is run, the test specifications are run in the order of test categories: unit tests, followed by integration tests, followed by functional tests.  If that was the case, the integration test that failed should have never failed as all integration specifications were properly annotated with @Integration and @Rollback and there was no setup() method creating data that couldn't be rolled back by @Rollback annotation causing the pollution. Grails 3 doesn't distinguish integration tests from functional test (at least when they are run as part of integrationTest gradle task), though they are distinguished by code. Typically integration tests extend Spock's Specification and annotated with @Integration and @Rollback, where as functional tests simply extend GebSpec.

Both local and CI server ran test-app task against an empty database. The only difference was: local was on Mac OS X and Bamboo CI server was on Linux. There was a functional test (GebSpec) in the set of integration tests under integration-test/groovy/myapp dir which had a bunch of feature methods. That one obviously did not have any data cleanup methods like: cleanup() or cleanupSpec(). I didn't even want to do any cleanup at the end in that functional spec because it was perfectly fine with local run. That led me to think of test data pollution causing this issue. But the most puzzling question was: "Why this functional test was coming in between and polluting integration test cases?" The only way to find it out was to know the order in which test-cases run and compare local run with Bamboo CI run.

I read through some documentation of Spock and Junit but didn't find an easy way of knowing the order of tests. Spock supports @Stepwise annotation to specify the order, but it was only within a specification. I read through some Gradle documentation about the test task and found a way to tap into the lifecycle of test cases and print the description of each test case that gets run. This helped me finding test-cases run order, and compare local with CI run to nail down the issue. Locally on Mac OS X, test-cases ran in alphabetical order, where as on Bamboo CI server (Linux), they seemed running in random order and one GebSpec functional test that was part of integration tests that got run along with all integration tests during integrationTest task as part of grails test-app task was the culprit. It was a coincidence that the functional specification name alphabetically was the last in the set of integration tests. On Mac OS X, it ran as the very last test case but on Linux it was running in between causing the following test to fail by leaving data in the database and thus leading to "test data pollution".

Following is the code snippet I added in build.gralde that prints each test before it's run:

/** * Configure test and integrationTest tasks. * Added beforeTest closure to get notified before a test is run. The closure simply logs the test descriptor which * indicates the test method that is being executed. Added to help find the test execution order differences between two * test runs or even differences between two systems like local, ci etc. */ test { beforeTest { descriptor -> logger.lifecycle("Running test: " + descriptor) } } integrationTest { beforeTest { descriptor -> logger.lifecycle("Running test: " + descriptor) } }

This will print as shown below, for instance when unit tests are run with the command: grails test-app -unit

:compileJava UP-TO-DATE :compileGroovy :buildProperties :processResources :classes :compileTestJava UP-TO-DATE :compileTestGroovy :processTestResources UP-TO-DATE :testClasses :test Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=2048m; support was removed in 8.0 Running test: Test testIndex(myapp.FirstTestSpec) Running test: Test testList(myapp.FirstTestSpec) Running test: Test testCreate(myapp.FirstTestSpec) Running test: Test testList(myapp.SecondTestSpec) Running test: Test testCreate(myapp.SecondTestSpec) Running test: Test testIndex(myapp.ThirdTestSpec) . . .

Ideally the order in which test cases run should not be a concern at all. But when functional tests and integration tests run together as integration tests, there are high chances of functional tests and integration tests getting mixed up in the sequence of running integration tests, and thus polluting integration tests. So, knowing the order in which test cases run will help solving this problem.

After finding out the issue, I refactored test cases under integration-test/groovy/myapp directory and separated out functional from integration tests into two different folders/packages (myapp.integration and myapp.functional) and even separated out their executions by running unit, integration and functional in 3 steps instead of running all in one step (grails test-app) as follows:

grails -Dgrails.env=development test-app -unit grails -Dgrails.env=development test-app myapp.integration.* -integration grails -Dgrails.env=development test-app myapp.functional.* -integration

This will guarantee that my functional tests which are expected to leave data in the database after run (as they were written) are run as the last group of tests in a bit controlled manner.

References

Gradle Test task documentation