The custom code of your custom step can be either Java 17 or, until November 5, 2025, Python 3.9.1. This chapter provides a code example for each programming language.

Starting from November 5, 2025, the default Python version will be 3.13. In the meantime, we encourage you to test your affected pipelines with version 3.13 as described here.

Prerequisites

  • You have a paid plan.
  • To execute the following actions, you have to be assigned the Admin role or higher.
  • You use an IDE of your choice (e.g Eclipse) to develop the custom step for your pipeline.

The custom processing logic and its Intellectual Property (IP) belong to the project owner.
The project owner is liable for damages which are caused by the custom processing logic.
We advise you to perform security and validity scans (especially when using open source code).
We also advise you to verify open source license compliance of your custom processing logic.

Code examples

The code examples provided for Java or Python are explained in detail in the following. They can be customized according to your needs.

Proceed as follows

  1. Download the Java or the Python code example and unzip the file.

    Check the provided README file for more information on how to setup the required software.
  2. Change the example according to your needs.
  3. Build the code with your changes.

  4. Zip the file again with the required files.

    The executable-manifest.yaml file must be on top level.
  5. Upload the zip file into a custom step of your Pipeline, refer to Configuring a pipeline.


Java example

File

Description

Required

executable-manifest.yaml

In the manifest, you configure which java class is used for the startup.

YES

src/main/java/com/example/myfirstcustomstep/

AsyncMyFirstCustomStepImpl.class

This is the java class to start your processing step with.

The java class has to listen for incoming data on System.in and print out processing results on System.out.

The name of the class can be changed according to your needs in the executable-manifest.yaml file.

YES

*.jar

Unlike the Python example, all required dependencies are downloaded locally.

When building the artifact with our maven plugin, the dependencies specified in the pom.xml file are automatically added to the jar.

YES

Python example

File

Description

Required

executable-manifest.yaml

In the manifest, you configure which script file is used for the startup.

YES

src/step.py

This is the Python script to start your processing step with.

The Python script has to listen for incoming data on sys.stdin and print out processing results on sys.stdout.

The name of the script can be changed according to your needs in the executable-manifest.yaml file.

YES

requirements.txt

This file is used to switch off the dependency auto detection and enforce specific versions of dependencies.

All packages on the pypi.org website are available to list them here, as long as they run on a Linux Ubuntu 18.04 operating system.

NO

Manifest attributes

Attribute

Description

input

Defines how large input data is fetched and processed.

Supported data types:

  • insights+json: Injected input data is provided as JSON. All input data is embedded directly in the injected data.
  • insights-payloadRef+json: Injected input data is provided as JSON (small files) or with a payload reference (large files). Small input data is embedded directly in the injected data. Large input data has to be fetched from the S3 object storage by using the payload reference and needs to be processed by using streaming. This can prevent memory congestion.

There is an example in the code on how to decide if the input data is processed embedded or is fetched from the S3 object storage.

Example for producing custom events using exchange data holder

You can use the following example with the relevant adjustments to produce a custom event as part of the custom step logic.

private List<CustomEventRef> createCustomEventToDispatchOnEndOfPipeline(List<Document> documents) {
// Note: Ensure you have first created the custom event definition with the name 'allStepsProcessed'.
String customEventName = "allStepsProcessed";
// produce your payload here using the documents
      Document customEventPayload = new Document();
      customEventPayload.put( "completedAt", Instant.now() );
return List.of( new CustomEventRef( customEventName, customEventPayload, CustomEventRef.DispatchType.END_OF_PIPELINE ) );
}

Then set it as sendEvents of the the ExchangeDataHolder like this:

ExchangeDataHolder output = ExchangeDataHolder.fromInput( inputDataHolder );
output.setSendEvents( createCustomEventToDispatchOnEndOfPipeline(output.getDocuments()) );