Chapter 6.2☕ 16 min read

Item Readers, Processors & Writers

Read one, process one, write a chunk. Memory-friendly batch processing.

01The Concept: Chunk Processing

The Dr. Reddy’s Pharma Assembly Line Analogy:

In a Hyderabad pharma factory, they don’t make one pill at a time.

Reader: A machine drops 100 empty capsules into the tray.
Processor: A machine fills those 100 capsules with medicine.
Writer: A machine seals the 100 capsules and puts them in a box.

Only when the 100 capsules are fully boxed does the next batch of 100 empty capsules drop. If the filling machine jams, the 100 capsules in the tray are discarded, and the factory fixes the machine before dropping the next 100. Spring Batch calls this a “chunk” (commit-interval).

02Technical Explanation

ItemReader<T>: Reads one item at a time from a source (CSV, Database, JSON).
ItemProcessor<T, T>: Transforms the item (e.g., uppercase a name, calculate a discount). Can filter items by returning null.
ItemWriter<T>: Writes a list (chunk) of items to the destination (Database, CSV).
chunk(size): Tells Spring to read and process size items, and then send them all together to the Writer in a single transaction.

03Full Working Code: CSV to Database

Let’s read books from a CSV file, process them (uppercase the author), and write them to an H2 database.

// Entity (Book.java)
package com.devinhyderabad;

import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.GenerationType;

@Entity
public class Book {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
    private String title;
    private String author;

    public Book() {}

04Entity and Batch Configuration

// BatchConfig.java
package com.devinhyderabad;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.job.builder.JobBuilder;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.step.builder.StepBuilder;
import org.springframework.batch.item.ItemProcessor;
import org.springframework.batch.item.database.JpaItemWriter;
import org.springframework.batch.item.database.builder.JpaItemWriterBuilder;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.builder.FlatFileItemReaderBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.ClassPathResource;
import org.springframework.transaction.PlatformTransactionManager;

@Configuration
public class BatchConfig {

    @Bean
    public FlatFileItemReader<Book> csvReader() {
        return new FlatFileItemReaderBuilder<Book>()
                .name("csvReader")
                .resource(new ClassPathResource("books.csv"))
                .delimited()
                .names("title", "author")
                .targetType(Book.class)
                .build();
    }

    @Bean
    public ItemProcessor<Book, Book> uppercaseProcessor() {
        return book -> {
            book.setAuthor(book.getAuthor().toUpperCase());
            return book;
        };
    }

    @Bean
    public JpaItemWriter<Book> jpaWriter(jakarta.persistence.EntityManagerFactory emf) {
        return new JpaItemWriterBuilder<Book>()
                .entityManagerFactory(emf)
                .build();
    }

    @Bean
    public Step csvToDbStep(JobRepository jobRepository, PlatformTransactionManager transactionManager,
                            FlatFileItemReader<Book> reader, ItemProcessor<Book, Book> processor, JpaItemWriter<Book> writer) {
        return new StepBuilder("csvToDbStep", jobRepository)
                .<Book, Book>chunk(10, transactionManager)
                .reader(reader)
                .processor(processor)
                .writer(writer)
                .build();
    }

    @Bean
    public Job csvToDbJob(JobRepository jobRepository, Step csvToDbStep) {
        return new JobBuilder("csvToDbJob", jobRepository)
                .start(csvToDbStep)
                .build();
    }
}

05Why It Matters / Interview Note

Interview Question: “How does Spring Batch handle memory when processing 1 billion records?”

Answer: Spring Batch uses chunk processing. Instead of loading 1 billion records into RAM, the ItemReader reads one record, the ItemProcessor transforms it, and it is held in memory. Once the chunk size (e.g., 100) is reached, the ItemWriter writes them to the DB, and the transaction is committed. The 100 items are then cleared from memory.

Enterprise Note: If a record fails during processing (e.g., bad data), Spring Batch allows you to configure “Skip Logic”. You can tell Spring to skip records that throw a specific exception (e.g., .faultTolerant().skip(InvalidDataException.class).skipLimit(10)), ensuring one bad row doesn’t crash a 10-hour batch job.

Key Takeaways

✅ ItemReader reads one item at a time from CSV, DB, or JSON sources
✅ ItemProcessor transforms items and can filter by returning null
✅ ItemWriter writes a chunk (batch) of items in a single transaction
✅ Chunk size determines how many items are processed before a DB commit
✅ Skip logic prevents one bad record from crashing a batch job

PreviousSpring Batch Intro 📋 All Chapters Next ChapterScheduling Batch Jobs →

Item Readers, Processors &amp; Writers

Key Takeaways

Item Readers, Processors & Writers