GetAzureEventHub

Description:

Receives messages from Microsoft Azure Event Hubs, writing the contents of the Azure message to the content of the FlowFile. Note: Please be aware that this processor creates a thread pool of 4 threads for Event Hub Client. They will be extra threads other than the concurrent tasks scheduled for this processor.

Tags:

azure, microsoft, cloud, eventhub, events, streaming, streams

Properties:

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.

NameDefault ValueAllowable ValuesDescription
Event Hub NameThe name of the event hub to pull messages from
Service Bus Endpoint.servicebus.windows.net
  • Azure Servicebus endpoint for general use
  • Azure China Servicebus endpoint for China
  • Azure Germany Servicebus endpoint for Germany
  • Azure US Government Servicebus endpoint for US Government
To support namespaces not in the default windows.net domain.
Event Hub NamespaceThe namespace that the event hub is assigned to. This is generally equal to <Event Hubs Name>-ns
Shared Access Policy NameThe name of the shared access policy. This policy must have Listen claims.
Shared Access Policy Primary KeyThe primary key of the shared access policy
Sensitive Property: true
Use Azure Managed Identityfalse
  • true
  • false
Choose whether or not to use the managed identity of Azure VM/VMSS
Number of Event Hub PartitionsThe number of partitions that the event hub has. Only this number of partitions will be used, so it is important to ensure that if the number of partitions changes that this value be updated. Otherwise, some messages may not be consumed.
Consumer Group$DefaultThe name of the consumer group to use when pulling events
Message Enqueue TimeA timestamp (ISO-8601 Instant) formatted as YYYY-MM-DDThhmmss.sssZ (2016-01-01T01:01:01.000Z) from which messages should have been enqueued in the Event Hub to start reading from
Partition Receiver Fetch SizeThe number of events that a receiver should fetch from an Event Hubs partition before returning. Default(100)
Partition Receiver Timeout (millseconds)The amount of time a Partition Receiver should wait to receive the Fetch Size before returning. Default(60000)

Relationships:

NameDescription
successAny FlowFile that is successfully received from the event hub will be transferred to this Relationship.

Reads Attributes:

None specified.

Writes Attributes:

NameDescription
eventhub.enqueued.timestampThe time (in milliseconds since epoch, UTC) at which the message was enqueued in the event hub
eventhub.offsetThe offset into the partition at which the message was stored
eventhub.sequenceThe Azure sequence number associated with the message
eventhub.nameThe name of the event hub from which the message was pulled
eventhub.partitionThe name of the event hub partition from which the message was pulled
eventhub.property.*The application properties of this message. IE: 'application' would be 'eventhub.property.application'

State management:

This component does not store state.

Restricted:

This component is not restricted.

Input requirement:

This component does not allow an incoming relationship.

System Resource Considerations:

None specified.