Spring Batch에서 지원하는 배치 병렬 처리 방식

  • AsyncItemProcessor / AsyncItemWriter → 한 step 내에서 processor만 병렬 수행해야 할 때
  • Multi-threaded Step → 한 step 내에서 reader, processor, writer를 chunk 단위로 병렬 수행해야 할 때
  • Parallel Steps → 여러 step들을 병렬로 수행해야 할 때
  • Externalizing Batch Process Execution 
    → 외부 remote 서버에서 병렬 수행 필요할 때. (master-worker 모델)
    • Remote Chunking of Step
      → 스텝 내의 Processor, Writer가 무거운 작업이라 외부 remote 서버들에서 병렬로 돌리고 싶을 때
    • Remote Partitioning (Partitioning a Step)
      → Reader 부터가 무거운 작업이라 Reader를 포함한 Step 전체를 쪼개 외부 remote 서버들에서 병렬로 돌리고 싶을 때.
      *꼭 외부가 아니라 local thread로도 가능

 

https://docs.spring.io/spring-batch/docs/current/reference/html/scalability.html

 

AsyncItemProcessor / AsyncItemWriter

 

Multi-threaded Step

 

TaskExecutor의 종류

https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/core/task/TaskExecutor.html

- All Known Implementing Classes 부분 참조

 

https://www.baeldung.com/java-threadpooltaskexecutor-core-vs-max-poolsize

 

Parallel Steps

 

Remote Chunking

https://godekdls.github.io/Spring%20Batch/scalingandparallelprocessing/#73-remote-chunking

https://godekdls.github.io/Spring%20Batch/springbatchintegration/#remote-chunking

 

Remote Partitioning

https://godekdls.github.io/Spring%20Batch/scalingandparallelprocessing/#74-partitioning

https://godekdls.github.io/Spring%20Batch/springbatchintegration/#remote-partitioning

 

참고

https://backtony.github.io/spring/2022-01-29-spring-batch-11/  

https://godekdls.github.io/Spring%20Batch/scalingandparallelprocessing/  

https://docs.spring.io/spring-batch/docs/4.2.x/reference/html/index-single.html#scalability