0.15.2
Deployed: 18-09-2025
Latest feature and fixes for Data Catering include:
- Various performance improvements
- Don't call
df.rddwhen zipping with index in foreign key logic - Don't call
df.rddwhen checking for unique values - When passing metadata to nested fields, don't re-create dataframe
- Use
unionByNameinstead of checking if dataframe is empty then runningunion - Set
enableSinkMetadatato false by default - New unique value checking logic using bloom filters
- Add
uniqueBloomFilterNumItemsto generation config - Add
uniqueBloomFilterFalsePositiveProbabilityto generation config - Check unique generation tuning documentation here
- Add
- Update default Spark memory settings
- Update
nettyandjsonsmartlibraries due to vulnerabilities - Add
enableUniqueCheckOnlyInBatchto Scala and Java API - Check configuration documentation here