HIVE-8852 - Update new spark progress API for local submitted job monitoring [Spark Branch] (Rui Li via Brock)