The application is a Spring Boot command-line runnable app, which takes the batch job name(s) and input data file(s) (job related datafile arg, and filename) and processes those files by kicking off those jobs. This application required few major changes to get it functioning. I had to read the docs to identify few things that were already deprecated in 4.x and are removed in 5.x. also finding out on new things put in place.
1. The in memory map datasource in earlier versions was deprecated. Now it's removed. So, it requires to put in a configuration for this. The following is an example.
2. @EnableBatchProcessing annotation takes new properties: dataSourceRef, transactionManagerRef.
3. JobBuilderFactory, StepBuilderFactory are removed. Use JobBuilder and StepBuilder instead.
4. Unlike previous versions, jobs won't start by setting the property: spring.batch.job.names to a comma separated names of jobs. Need to explicitly start the job launcher. The following is a code snippet of the main application which takes a job name passed from command line by property spring.batch.job.name and launches that job.
Environment: Java 20, Spring Boot 2.7.14, Spring Boot 3.1.2, Spring Batch 4.3.8, Spring Batch 5.0.2, maven 3.9.3 on macOS Catalina 10.15.7
The following is the summary of high level changes. I am not going into details in this post as these changes are easy enough to understand at a high level.
1. The in memory map datasource in earlier versions was deprecated. Now it's removed. So, it requires to put in a configuration for this. The following is an example.
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.repository.support.JobRepositoryFactoryBean;
import org.springframework.batch.support.transaction.ResourcelessTransactionManager;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseBuilder;
import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseType;
import org.springframework.transaction.PlatformTransactionManager;
import javax.sql.DataSource;
/**
* In memory data source configuration for batch jobs.
* Defines {@link DataSource}, {@link PlatformTransactionManager} and {@link JobRepository} beans.
*
* @author Giri
* created Aug 16, 2023
*/
@Configuration
public class InMemoryBatchRepositoryConfig {
@Bean
public DataSource inMemoryDataSource() {
EmbeddedDatabaseBuilder builder = new EmbeddedDatabaseBuilder();
return builder.setType(EmbeddedDatabaseType.H2)
.addScript("classpath:org/springframework/batch/core/schema-drop-h2.sql")
.addScript("classpath:org/springframework/batch/core/schema-h2.sql")
.generateUniqueName(true)
.build();
}
@Bean
public PlatformTransactionManager resourceLessTransactionManager() {
return new ResourcelessTransactionManager();
}
@Bean
public JobRepository jobRepository() throws Exception {
JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
factory.setDataSource(inMemoryDataSource());
factory.setTransactionManager(resourceLessTransactionManager());
factory.afterPropertiesSet();
return factory.getObject();
}
}
2. @EnableBatchProcessing annotation takes new properties: dataSourceRef, transactionManagerRef.
@Slf4j
@Configuration
@EnableBatchProcessing(dataSourceRef = "inMemoryDataSource", transactionManagerRef = "resourceLessTransactionManager")
public class MyJobConfig {
@Value("${data_filename:NONE}")
String dataFilename;
...
}
3. JobBuilderFactory, StepBuilderFactory are removed. Use JobBuilder and StepBuilder instead.
@Slf4j
@Configuration
@EnableBatchProcessing(dataSourceRef = "inMemoryDataSource", transactionManagerRef = "resourceLessTransactionManager")
public class MyJobConfig {
...
@Bean
public Job myJob(
JobRepository jobRepository,
Step myJobStep,
JobCompletionNotificationListener jobCompletionNotificationListener) {
return new JobBuilder("myJob", jobRepository)
.listener(jobCompletionNotificationListener)
.flow(myJobStep)
.end()
.build();
}
@Bean
public Step myJobStep(
JobRepository jobRepository,
PlatformTransactionManager platformTransactionManager,
ItemFailureLoggerListener itemFailureLoggerListener,
StepListener stepListener){
var step = new StepBuilder("myJobStep", jobRepository)
.<MyDomainObject, String> chunk(10, platformTransactionManager)
.reader(myDomainObjectReader()) //input
.processor(myDomainObjectProcessor(isGm5)) //transformer/processor
.writer(myDomainObjectWriter())//output
.listener((ItemProcessListener) itemFailureLoggerListener)
.build();
step.registerStepExecutionListener(stepListener);
return step;
}
...
}
4. Unlike previous versions, jobs won't start by setting the property: spring.batch.job.names to a comma separated names of jobs. Need to explicitly start the job launcher. The following is a code snippet of the main application which takes a job name passed from command line by property spring.batch.job.name and launches that job.
@Slf4j
@SpringBootApplication
public class BatchApplication implements CommandLineRunner, InitializingBean {
@Value("${spring.batch.job.name:NONE}")
private String jobName;
@Autowired
ApplicationContext applicationContext;
@Autowired
JobLauncher jobLauncher;
@Override
public void afterPropertiesSet() {
try {
validateJobName();
} catch (Exception ex) {
log.error(ex.getMessage());
System.exit(SpringApplication.exit(applicationContext, () -> 1));
}
}
public static void main(String[] args) {
SpringApplication app = new SpringApplicationBuilder(ScoringEtlApplication.class)
.web(WebApplicationType.NONE)
.logStartupInfo(false)
.build(args);
app.run(args);
}
@Override
public void run(String... args) {
log.debug("Beans Count:{} Beans:{}", applicationContext.getBeanDefinitionCount(), applicationContext.getBeanDefinitionNames());
Job job = (Job)applicationContext.getBean(jobName);
JobParameters jobParameters = new JobParametersBuilder()
.addString("jobID", String.valueOf(System.currentTimeMillis()))
.toJobParameters();
try {
jobLauncher.run(job, jobParameters);
log.info("Finished running job(s): {} with args: {}", jobName, Arrays.stream(args).collect(Collectors.joining(", ")));
} catch (JobExecutionAlreadyRunningException | JobRestartException | JobInstanceAlreadyCompleteException | JobParametersInvalidException e) {
throw new RuntimeException(e);
}
}
/**
* Validates jobName and displays usage and appropriate error message upon validation failure.
*/
private void validateJobName() throws URISyntaxException {
String errorMessage = null;
if (jobName.isEmpty() || jobName.equals("NONE")) {
errorMessage = "No job(s) specified. Please, specify valid job_name(s)";
}
if(errorMessage != null) {
String runningJar = this.getClass().getClassLoader().getClass()
.getProtectionDomain()
.getCodeSource()
.getLocation()
.toURI()
.getPath();
// display Usage
log.info("""
USAGE:
Run a specific job:
java -jar <runnableJar> \\
--spring.batch.job.name=<job_name> \\
--<job_arg_data_filename>="job_data_file.csv"
""".replace("runningJar", runningJar));
throw new IllegalArgumentException(errorMessage);
} else {
List allJobNames = Arrays.stream(applicationContext.getBeanNamesForType(Job.class)).toList();
log.debug("Available Jobs: {}", allJobNames);
log.info("Running Job: {}", jobName);
}
}
}
Other classes referenced in Job configurations:
public class JobCompletionNotificationListener extends JobExecutionListenerSupport {
@Value("${spring.batch.job.name:default-job}")
private String jobName;
@Override
public void beforeJob(JobExecution jobExecution) {
log.info("JOB: {} - About to start.", jobName);
}
@Override
public void afterJob(JobExecution jobExecution) {
if(jobExecution.getStatus() == BatchStatus.COMPLETED) {
var from = jobExecution.getStartTime();
var to = jobExecution.getEndTime();
log.info("JOB: {} - Finished running. Took {} milliseconds. Verify results.", jobName, ChronoUnit.MILLIS.between(from, to));
}
}
}
@Slf4j
@Component
public class ItemFailureLoggerListener extends ItemListenerSupport
TIPS
When running the command line application, if there are warnings: Using deprecated '-debug' fallback for parameter name resolution. Compile the affected code with '-parameters' instead or avoid its introspection: MyJobConfig, then suppress it by setting the following maven-compiler-plugin setting:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven-compiler-plugin.version}</version>
<configuration>
<source>${javac.source.version}</source>
<target>${javac.target.version}</target>
<release>${javac.release.version}</release>
<compilerArgs>
<arg>-parameters</arg>
</compilerArgs>
</configuration>
</plugin>
Posts
No comments:
Post a Comment