Announcing the release of Apache Samza 1.5.1
We are thrilled to announce the release of Apache Samza 1.5.1.
In 1.5 release, we enabled transactional state by default for all samza jobs. We identified a critical bug related to trimming the state and have addressed the issue in 1.5.1. For applications that are already on Samza 1.5.0, updating your dependencies to use Samza 1.5.1 should be sufficient to upgrade. For applications that are on version 1.4 & below, please follow the upgrade instructions below.
Upgrading your application to Apache Samza 1.5.0
ConfigFactory is deprecated as Job Runner does not load full job config anymore. Instead, ConfigLoaderFactory is introduced to be executed on ClusterBasedJobCoordinator to fetch full job config. If you are using the default PropertiesConfigFactory, simply switching to use the default PropertiesConfigLoaderFactory will work, otherwise if you are using a custom ConfigFactory, kindly creates its new counterpart following ConfigLoaderFactory.
Configs related to job submission must be explicitly provided to Job Runner as it is no longer loading full job config anymore. These configs include
- Configs directly related to job submission, such as yarn.package.path, job.name etc.
- Configs needed by the config loader on AM to fetch job config, such as path to the property file in the tarball, all of such configs will have a job.config.loader.properties prefix.
- Configs that users would like to override
Full list of the job submission configurations can be found here
Alternative way when submitting job,
can be simplified to
where submission.properties contains
In case of a problem in Samza 1.5.1, users can rollback to Samza 1.4 and keep the old start up flow using config-path & config-factory.
SAMZA-2578 Excessive trimming during transactional state restore