public class StorageListener
extends org.apache.spark.storage.BlockStatusListener
This class is thread-safe (unlike JobProgressListener)
Constructor and Description |
---|
StorageListener(StorageStatusListener storageStatusListener) |
Modifier and Type | Method and Description |
---|---|
scala.collection.mutable.Map<java.lang.Object,RDDInfo> |
_rddInfoMap() |
void |
onStageCompleted(SparkListenerStageCompleted stageCompleted) |
void |
onStageSubmitted(SparkListenerStageSubmitted stageSubmitted) |
void |
onTaskEnd(SparkListenerTaskEnd taskEnd)
Assumes the storage status list is fully up-to-date.
|
void |
onUnpersistRDD(SparkListenerUnpersistRDD unpersistRDD) |
scala.collection.Seq<RDDInfo> |
rddInfoList()
Filter RDD info to include only those with cached partitions
|
scala.collection.Seq<StorageStatus> |
storageStatusList() |
public StorageListener(StorageStatusListener storageStatusListener)
public scala.collection.mutable.Map<java.lang.Object,RDDInfo> _rddInfoMap()
public scala.collection.Seq<StorageStatus> storageStatusList()
public scala.collection.Seq<RDDInfo> rddInfoList()
public void onTaskEnd(SparkListenerTaskEnd taskEnd)
taskEnd
- (undocumented)public void onStageSubmitted(SparkListenerStageSubmitted stageSubmitted)
public void onStageCompleted(SparkListenerStageCompleted stageCompleted)
public void onUnpersistRDD(SparkListenerUnpersistRDD unpersistRDD)