Deployment
Three main ways to deploy and run Data Caterer:
- Application
- Docker
- Helm
Application
Run the OS native application from downloading the specific OS application here.
Docker
To package up your class along with the Data Caterer base image, you can follow the Dockerfile that is created for you here.
Then you can run the following:
Docker Pre and Post Processing Scripts
Data Caterer supports running custom scripts before and after the main data generation process when deployed via Docker. This is useful for setup tasks, cleanup operations, notifications, or integrating with external systems.
Configuration
Configure pre and post processing scripts using environment variables:
Environment Variable | Description | Default |
---|---|---|
PRE_PROCESSOR_SCRIPT |
Path to script to run before Data Caterer execution | (empty) |
POST_PROCESSOR_SCRIPT |
Path to script to run after Data Caterer execution | (empty) |
POST_PROCESSOR_CONDITION |
When to run post processor: success , failure , always |
success |
Usage Example
docker run -d \
-e PRE_PROCESSOR_SCRIPT="/opt/app/scripts/setup.sh" \
-e POST_PROCESSOR_SCRIPT="/opt/app/scripts/cleanup.sh" \
-e POST_PROCESSOR_CONDITION="always" \
-v /path/to/scripts:/opt/app/scripts \
datacatering/data-caterer:0.16.11
Script Execution Behavior
- Pre-processor: Runs before Data Caterer starts
- If the script fails, Data Caterer execution is stopped
- Script must be executable and return exit code 0 for success
- Post-processor: Runs after Data Caterer completes, based on condition:
success
: Only runs if Data Caterer exit code is 0failure
: Only runs if Data Caterer exit code is non-zeroalways
: Runs regardless of Data Caterer exit code- If post-processor fails, the original Data Caterer exit code is preserved
Error Handling
- Scripts are executed with bash and include comprehensive error logging
- Missing script files generate warnings but don't stop execution
- All script output is logged with clear prefixes (pre/post processor)
- The final exit code is always Data Caterer's original exit code
Helm
Link to sample helm on GitHub here
Update the configuration to your own data connections and configuration or own image created from above.