Salesforce B2C Commerce 24.7 > Administering Your Organization > Jobs > Creating Jobs > Using Job Steps > Creating Custom Job Steps

Chunk-Oriented Script Module

You can define what a custom step does using a chunk-oriented CommonJS script module that reads and processes items in chunks of size S. If the list contains more items than can be processed in one chunk, a new chunk is started. A chunk-oriented script can include any series of processing steps, not just database transactions. A job step that uses a chunk-oriented script allows fine-grained progress monitoring because the number of elements that are written is updated each time a chunk is finished.

Chunk Processing Example

In this example, a list of 15 orders is exported to a file. Each chunk includes four orders.

Table 1. Chunk Processing Example
Chunk	Steps
Chunk 1	Read Order 1. Process Order 1. Read Order 2. Process Order 2. Read Order 3. Process Order 3. Read Order 4. Process Order 4. Write Order 1. Write Order 2. Write Order 3. Write Order 4.
Chunk 2	Read Order 5. Process Order 5. Read Order 6. Process Order 6. Read Order 7. Process Order 7. Read Order 8. Process Order 8. Write Order 5. Write Order 6. Write Order 7. Write Order 8.
Chunk 3	Read Order 9. Process Order 9. Read Order 10. Process Order 10. Read Order 11. Process Order 11. Read Order 12. Process Order 12. Write Order 9. Write Order 10. Write Order 11. Write Order 12.
Chunk 4	Read Order 13. Process Order 13. Read Order 14. Process Order 14. Read Order 15. Process Order 15. Write Order 13. Write Order 14. Write Order 15. This chunk read, processed, and wrote fewer items than the previous chunks, because only three items were left on the list.

Table 2. Chunk-Oriented Script Module Functions
Function	Required or Optional	Purpose
read-function	Required	Returns one item or nothing if there are no more items.
process-function	Required	Transforms items and applies business logic to them. It receives the item returned by the read function, performs a process, and returns one item.The item returned can be the same item that the read function returned if no processing logic is necessary, or it can be a new item of a different type. If the process function returns nothing, then the read function item is filtered and doesn't appear in the list of items to be written later.
write-function	Required	Receives a list of items. The list size matches the chunk size or, if the number of items in the last available chunk is smaller, it is smaller. The write function returns nothing.
total-count-function	Optional	Returns the total number of items that are available. Called by the framework exactly once before chunk processing begins. A known total count allows better monitoring, for example, to show that 50 of 100 items have already been processed.
before-step-function	Optional	Executed before a chunk step begins. Implements logic before all items of all chunks are read, processed, and written.
before-chunk-function	Optional	Executed before a chunk begins. Implements logic before a chunk of S items is read, processed, and written.
after-chunk-function	Optional	Executed after a chunk finishes. Implements logic after a chunk of S items has been read, processed, and written successfully.
after-step-function	Optional	Executed after a chunk step finished successfully. Implements logic after all items of all chunks are read, processed, and written successfully.

When administrators create jobs using Business Manager, they set parameters that are available as scriptable objects for each exposed module function and for the dw.job.JobStepExecution object. The dw.job.JobStepExecutionobject allows read-only access to information about the current step execution and job execution. You can't define the exit status for a chunk-oriented script module. Chunk modules always finish with either OK or ERROR.

The following example shows a chunk-oriented CommonJS module.

var system = require( 'dw/system' );
var catalog = require( 'dw/catalog' );
var job = require( 'dw/job' ); 
var io = require( 'dw/io' );

var products;
var fileWriter;
 
exports.beforeStep = function( parameters, stepExecution )
{
  fileWriter = new io.FileWriter( new io.File( io.File.IMPEX ), parameters.ExportFileName );
  products = catalog.ProductMgr.queryAllSiteProducts();
  fileWriter.writeLine( "<products>" );
}
 
exports.getTotalCount = function( parameters, stepExecution )
{
  return products.count;
}


exports.read = function( parameters, stepExecution )
{
  if( products.hasNext() )
    {
    return products.next();
    }
}
 
exports.process = function( product, parameters, stepExecution )
{
  if( product.isOnline() )
  {
    return "<product>" + product.getID() + "</product>";
  }
}
 
exports.write = function( lines, parameters, stepExecution )
{
  for ( i = 0; i < lines.size(); i++ ) 
  {
    fileWriter.writeLine( lines.get(i) );
  }
} 
 
exports.afterStep = function( success, parameters, stepExecution )
{
  if( success )
  {
    fileWriter.writeLine( "</products>" );
  }
  fileWriter.close();
  products.close();