SampleRecord

Description:

Samples the records of a FlowFile based on a specified sampling strategy (such as Reservoir Sampling). The resulting FlowFile may be of a fixed number of records (in the case of reservoir-based algorithms) or some subset of the total number of records (in the case of probabilistic sampling), or a deterministic number of records (in the case of interval sampling).

Additional Details...

Tags:

record, sample

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

Name	Default Value	Allowable Values	Description
Record Reader		Controller Service API: RecordReaderFactory Implementations: JsonPathReader AvroReader XMLReader WindowsEventLogReader ReaderLookup Syslog5424Reader GrokReader ScriptedReader CSVReader SyslogReader ParquetReader JsonTreeReader CEFReader	Specifies the Controller Service to use for parsing incoming data and determining the data's schema
Record Writer		Controller Service API: RecordSetWriterFactory Implementations: AvroRecordSetWriter ScriptedRecordSetWriter JsonRecordSetWriter ParquetRecordSetWriter RecordSetWriterLookup FreeFormTextRecordSetWriter XMLRecordSetWriter CSVRecordSetWriter	Specifies the Controller Service to use for writing results to a FlowFile
Sampling Strategy	reservoir	Interval Sampling Probabilistic Sampling Reservoir Sampling	Specifies which method to use for sampling records from the incoming FlowFile
Sampling Interval			Specifies the number of records to skip before writing a record to the outgoing FlowFile. This property is only used if Sampling Strategy is set to Interval Sampling. A value of zero (0) will cause no records to be included in theoutgoing FlowFile, a value of one (1) will cause all records to be included, and a value of two (2) will cause half the records to be included, and so on. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Sampling Probability			Specifies the probability (as a percent from 0-100) of a record being included in the outgoing FlowFile. This property is only used if Sampling Strategy is set to Probabilistic Sampling. A value of zero (0) will cause no records to be included in theoutgoing FlowFile, and a value of 100 will cause all records to be included in the outgoing FlowFile.. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Reservoir Size			Specifies the number of records to write to the outgoing FlowFile. This property is only used if Sampling Strategy is set to reservoir-based strategies such as Reservoir Sampling or Weighted Random Sampling. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
Random Seed			Specifies a particular number to use as the seed for the random number generator (used by probabilistic strategies). Setting this property will ensure the same records are selected even when using probabilistic strategies. Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)

Relationships:

Name	Description
success	The FlowFile is routed to this relationship if the sampling completed successfully
failure	If a FlowFile fails processing for any reason (for example, any record is not valid), the original FlowFile will be routed to this relationship
original	The original FlowFile is routed to this relationship if sampling is successful

Reads Attributes:

None specified.

Writes Attributes:

None specified.

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component requires an incoming relationship.

System Resource Considerations:

Resource	Description
MEMORY	An instance of this component can cause high usage of this system resource. Multiple instances or high concurrency settings may result a degradation of performance.