Retrieves the messages from an Azure Queue Storage. The retrieved messages will be deleted from the queue by default. If the requirement is to consume messages without deleting them, set 'Auto Delete Messages' to 'false'. Note: There might be chances of receiving duplicates in situations like when a message is received but was unable to be deleted from the queue due to some unexpected situations.
azure, queue, microsoft, storage, dequeue, cloud
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, whether a property supports the NiFi Expression Language, and whether a property is considered "sensitive", meaning that its value will be encrypted. Before entering a value in a sensitive property, ensure that the nifi.properties file has an entry for the property nifi.sensitive.props.key.
| Name | Default Value | Allowable Values | Description |
|---|---|---|---|
| Storage Credentials | Controller Service API: AzureStorageCredentialsService Implementations: AzureStorageEmulatorCredentialsControllerService AzureStorageCredentialsControllerServiceLookup AzureStorageCredentialsControllerService | The Controller Service used to obtain Azure Storage Credentials. Instead of the processor level properties, the credentials can be configured here through a common/shared controller service, which is the preferred way. The 'Lookup' version of the service can also be used to select the credentials dynamically at runtime based on a FlowFile attribute (if the processor has FlowFile input). | |
| Storage Account Name | The storage account name. There are certain risks in allowing the account name to be stored as a flowfile attribute. While it does provide for a more flexible flow by allowing the account name to be fetched dynamically from a flowfile attribute, care must be taken to restrict access to the event provenance data (e.g., by strictly controlling the policies governing provenance for this processor). In addition, the provenance repositories may be put on encrypted disk partitions. Instead of defining the Storage Account Name, Storage Account Key and SAS Token properties directly on the processor, the preferred way is to configure them through a controller service specified in the Storage Credentials property. The controller service can provide a common/shared configuration for multiple/all Azure processors. Furthermore, the credentials can also be looked up dynamically with the 'Lookup' version of the service. Sensitive Property: true Supports Expression Language: true (will be evaluated using variable registry only) | ||
| Storage Account Key | The storage account key. This is an admin-like password providing access to every container in this account. It is recommended one uses Shared Access Signature (SAS) token instead for fine-grained control with policies. There are certain risks in allowing the account key to be stored as a flowfile attribute. While it does provide for a more flexible flow by allowing the account key to be fetched dynamically from a flowfile attribute, care must be taken to restrict access to the event provenance data (e.g., by strictly controlling the policies governing provenance for this processor). In addition, the provenance repositories may be put on encrypted disk partitions. Sensitive Property: true Supports Expression Language: true (will be evaluated using variable registry only) | ||
| SAS Token | Shared Access Signature token, including the leading '?'. Specify either SAS token (recommended) or Account Key. There are certain risks in allowing the SAS token to be stored as a flowfile attribute. While it does provide for a more flexible flow by allowing the SAS token to be fetched dynamically from a flowfile attribute, care must be taken to restrict access to the event provenance data (e.g., by strictly controlling the policies governing provenance for this processor). In addition, the provenance repositories may be put on encrypted disk partitions. Sensitive Property: true Supports Expression Language: true (will be evaluated using variable registry only) | ||
| Common Storage Account Endpoint Suffix | Storage accounts in public Azure always use a common FQDN suffix. Override this endpoint suffix with a different suffix in certain circumstances (like Azure Stack or non-public Azure regions). The preferred way is to configure them through a controller service specified in the Storage Credentials property. The controller service can provide a common/shared configuration for multiple/all Azure processors. Furthermore, the credentials can also be looked up dynamically with the 'Lookup' version of the service. Supports Expression Language: true (will be evaluated using variable registry only) | ||
| Queue Name | Name of the Azure Storage Queue Supports Expression Language: true (will be evaluated using variable registry only) | ||
| Auto Delete Messages | true |
| Specifies whether the received message is to be automatically deleted from the queue. |
| Batch Size | 32 | The number of messages to be retrieved from the queue. | |
| Visibility Timeout | 30 secs | The duration during which the retrieved message should be invisible to other consumers. | |
| Proxy Configuration Service | Controller Service API: ProxyConfigurationService Implementation: StandardProxyConfigurationService | Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP, SOCKS |
| Name | Description |
|---|---|
| success | All successfully processed FlowFiles are routed to this relationship |
| Name | Description |
|---|---|
| azure.queue.uri | The absolute URI of the configured Azure Queue Storage |
| azure.queue.insertionTime | The time when the message was inserted into the queue storage |
| azure.queue.expirationTime | The time when the message will expire from the queue storage |
| azure.queue.messageId | The ID of the retrieved message |
| azure.queue.popReceipt | The pop receipt of the retrieved message |