Apache NiFi MiNiFi is an Apache NiFi project, designed to collect data at its source. MiNiFi was developed with the following objectives in mind:
MiNiFi Java Agent is supported on the following operating systems:
You can download the MiNiFi Java Agent and the MiNiFi Toolkit from the MiNiFi download page.
You have several options for installing and starting MiNiFi.
To install MiNiFi:
You can also install MiNiFi as a service:
bin/minifi.sh install
Note: You can also specify a custom name for your MiNiFi installation, by specifying that name during your install command. For example, to install MiNiFi as a service and named dataflow, enter:
bin/minifi.sh install dataflow
Once you have downloaded and installed MiNiFi, you need to start MiNiFi. You can start NiFi in the foreground, background, or as a service.
To launch MiNiFi in the foreground:
bin/minifi.sh run
To launch MiNiFi in the background:
bin/minifi.sh start
To launch MiNiFi as a service:
sudo service minifi start
For Windows users, navigate to the folder where MiNiFi was installed. Navigate to the /bin
subdirectory and double-click the run-minifi.bat file.
This launches MiNiFi and leaves it running in the foreground. To shut down NiFi, select the window that was launched and hold the Ctrl key while pressing C.
When you are working with a MiNiFi dataflow, you should design it, add any additional configuration your environment or use case requires, and then deploy your dataflow. MiNiFi is not designed to accommodate substantial mid-dataflow configuration.
You can use the MiNiFi Toolkit, located in your MiNiFi installation directory, and any NiFi instance to set up the dataflow you want MiNiFi to run:
config.sh transform input_file output_file
minifi/conf
.Note: You can use one template at a time, per MiNiFi instance.
Result: Once you have your config.yml file in the minifi/conf
directory, launch that instance of MiNiFi and your dataflow begins automatically.
MiNiFi is able to use following processors out of the box:
MiNiFi is able to use the StandardSSLContextService out of the box.
If you want to create a dataflow with a processor not shipped with MiNiFi, you can do so by following these steps:
lib
directory.Note: The following processors are also a part of the default distribution but require adding a NAR for a Controller Service not packaged by default. The processors are grouped by the NAR that is required.
You can secure your MiNiFi dataflow using keystore or trust store SSL protocols, however, this information is not automatically generated. You will need to generate your security configuration information yourself.
To run a MiNiFi dataflow securely:
minifi/conf
and rename config.yml.Security Properties:
keystore:
keystore type:
keystore password:
key password:
truststore:
truststore type:
truststore password:
ssl protocol: TLS
Sensitive Props:
key:
algorithm: PBEWITHMD5AND256BITAES-CBC-OPENSSL
provider: BC
You can also perform some management tasks using MiNiFi.
You can use the minifi.sh flowStatus
option to monitor a range of aspects of your MiNiFi operational and dataflow status. You can use the flowStatus
option to get information about dataflow component health and functionality, a MiNiFi instance, or system diagnostics.
FlowStatus accepts the following flags and options:
processor
health
bulletins
status
connection
health
stats
remoteProcessGroup
health
bulletins
status
authorizationIssues
inputPorts
controllerServices
health
bulletins
provenancereporting
health
bulletins
instance
health
bulletins
status
systemdiagnostics
heap
processorstats
contentrepositoryusage
flowfilerepositoryusage
garbagecollection
For example, this query gets the health, stats, and bulletins for the TailFile processor:
minifi.sh flowStatus processor:TailFile:health,stats,bulletins
Note: Currently, the script only accepts one high level option at a time.
Note: Any connections, remote process groups or processors names that contain ":", ";" or "," will cause parsing errors when querying.
For details on the flowStatus
option, see the "FlowStatus Query Option" section of the Administration Guide.
You can load a new dataflow for a MiNiFi instance to run:
minifi/conf
with the new file.You can stop MiNiFi at any time.
Stopping MiNiFi:
bin/minifi.sh stop
Stopping MiNiFi as a service:
sudo service minifi stop