Today, we show you Spring Batch Classifier Example that helps you to classify data to multiple files, even different file types, e.g: XML, JSON, Text. If you work in a big company and enjoy the development of large architecture and certainly you need to share data with multiple systems. This is really needed for you.
Spring Batch Classifier Example
Project Setup
Tools and framework that we use:
Spring Boot 2.2.2 RELEASE
Spring Batch 4.2.1
Maven 3.6
Java 8
Project Directory
Our project will have a structure like below
Project Dependencies
We’re using the dependencies like below
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
... <properties> <java.version>8</java.version> </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-batch</artifactId> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-oxm</artifactId> </dependency> <dependency> <groupId>com.thoughtworks.xstream</groupId> <artifactId>xstream</artifactId> <version>1.4.8</version> </dependency> <!-- https://mvnrepository.com/artifact/net.java.dev.stax-utils/stax-utils --> <dependency> <groupId>net.java.dev.stax-utils</groupId> <artifactId>stax-utils</artifactId> <version>20040917</version> </dependency> ... |
Routing data to multiple files using Classifier
Here, we reuse source code from the post Spring Batch Hello World Example and adapt the part of classifier implementation.
First, we create a simple classifier that routes employees of Java department to XML writer and the rest of the employees will be routed to JSON writer.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
package com.javabycode.batch; import org.springframework.batch.item.ItemWriter; import org.springframework.classify.Classifier; import com.javabycode.model.Employee; public class MyClassifier implements Classifier<Employee, ItemWriter<? super Employee>> { private static final long serialVersionUID = 1L; private ItemWriter<Employee> filteredItemWriter; private ItemWriter<Employee> otherItemWriter; public MyClassifier(ItemWriter<Employee> filteredItemWriter, ItemWriter<Employee> otherItemWriter) { this.filteredItemWriter = filteredItemWriter; this.otherItemWriter = otherItemWriter; } @Override public ItemWriter<? super Employee> classify(Employee employee) { // routing employees of Java department to XML writer return employee.getDepartment().contains("Java") ? filteredItemWriter : otherItemWriter; } } |
Adapting classifier to Spring Batch Job
Here, we need to create XML writer and JSON writer beans then adapt the MyClassifier
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 |
package com.javabycode.batch; import java.io.File; import java.util.HashMap; import java.util.Map; import org.springframework.batch.core.Job; import org.springframework.batch.core.Step; import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; import org.springframework.batch.item.file.FlatFileItemReader; import org.springframework.batch.item.file.FlatFileItemWriter; import org.springframework.batch.item.file.builder.FlatFileItemReaderBuilder; import org.springframework.batch.item.file.builder.FlatFileItemWriterBuilder; import org.springframework.batch.item.file.transform.PassThroughLineAggregator; import org.springframework.batch.item.support.ClassifierCompositeItemWriter; import org.springframework.batch.item.xml.StaxEventItemWriter; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.core.io.ClassPathResource; import org.springframework.core.io.FileSystemResource; import org.springframework.oxm.xstream.XStreamMarshaller; import com.javabycode.model.Employee; @Configuration public class HelloWorldJobConfig { @Bean public Job helloWorlJob(JobBuilderFactory jobBuilders, StepBuilderFactory stepBuilders) throws Exception { return jobBuilders.get("myJob").start(helloWorldStep(stepBuilders)).build(); } @Bean public Step helloWorldStep(StepBuilderFactory stepBuilders) throws Exception { return stepBuilders.get("myStep").<Employee, Employee>chunk(10).reader(reader()) // .processor(processor()) .writer(classifierCustomerCompositeItemWriter()).stream(xmlItemWriter()).stream(jsonItemWriter()) .build(); } @Bean public FlatFileItemReader<Employee> reader() { return new FlatFileItemReaderBuilder<Employee>().name("employeeItemReader") .resource(new ClassPathResource("csv/employees.csv")).delimited() .names(new String[] { "firstName", "lastName", "department" }).targetType(Employee.class).build(); } @Bean public ClassifierCompositeItemWriter<Employee> classifierCustomerCompositeItemWriter() throws Exception { ClassifierCompositeItemWriter<Employee> compositeItemWriter = new ClassifierCompositeItemWriter<>(); // adapting your classifier here compositeItemWriter.setClassifier(new MyClassifier(xmlItemWriter(), jsonItemWriter())); return compositeItemWriter; } @Bean public EmployeeItemProcessor processor() { return new EmployeeItemProcessor(); } @Bean public FlatFileItemWriter<String> writer() { return new FlatFileItemWriterBuilder<String>().name("greetingItemWriter") .resource(new FileSystemResource("target/output.txt")).lineAggregator(new PassThroughLineAggregator<>()) .build(); } // XML writer @Bean public StaxEventItemWriter<Employee> xmlItemWriter() throws Exception { String employeeOutputPath = File.createTempFile("employeeOutput", ".xml").getAbsolutePath(); System.out.println(">> XML Output = " + employeeOutputPath); Map<String, Class> aliases = new HashMap<>(); aliases.put("employee", Employee.class); XStreamMarshaller marshaller = new XStreamMarshaller(); marshaller.setAliases(aliases); // Serializing object to XML. IndentingStaxEventItemWriter<Employee> writer = new IndentingStaxEventItemWriter<>(); writer.setRootTagName("employees"); writer.setMarshaller(marshaller); writer.setResource(new FileSystemResource(employeeOutputPath)); writer.afterPropertiesSet(); return writer; } // JSON writer @Bean public FlatFileItemWriter<Employee> jsonItemWriter() throws Exception { String employeeOutputPath = File.createTempFile("employeeOutput", ".json").getAbsolutePath(); System.out.println(">> JSON Output = " + employeeOutputPath); FlatFileItemWriter<Employee> writer = new FlatFileItemWriter<>(); writer.setLineAggregator(new CustomLineAggregator()); writer.setResource(new FileSystemResource(employeeOutputPath)); writer.afterPropertiesSet(); return writer; } } |
Input data
Here, we assume that we need to read employee information from CSV file named employees.csv, its format looks like below:
|
1 2 3 |
David, Pham, Java James, Bon, PHP Tom, Hank, Objective-C |
This file is located in the project folder resources /csv/employees.csv
Running Application
To demo our Spring Batch Classifier Example, we run the below command in the console.
|
1 |
mvn spring-boot:run |
Note: You must be in your project directory before running that command.
While running that command, you will see the output like below
Let’s have a look into two output files:
XML output file
JSON output file
That’s all about Spring Batch Classifier Example.
If you find this post useful, don't hesitate to share it with your friends or other people for growing knowledge together and is an effort to contribute us.
References
Spring Boot Tutorial for Beginners
Spring Batch Frameworks
