File Component - Camel 1.x only

Using Camel 2.x This page is only for Camel 1.x. If you are using Camel 2.0 or newer see this link.

The File component provides access to file systems; allowing files to be processed by any other Camel Components or messages from other components can be saved to disk.

URI format

file:fileOrDirectoryName[?options]

or

file://fileOrDirectoryName[?options]

Where fileOrDirectoryName represents the underlying file name. Camel will determine if fileOrDirectoryName is a file or directory.

You can append query options to the URI in the following format, ?option=value&option=value&...

Important Information See the section "Common gotchas with folder and filenames" below.

Timestamp In Camel 1.5 or older the file consumer uses an internal timestamp for last polling. This timestamp is used to match for new or changed files: if file modified timestamp > last poll timestamp => file can be consumed. Beware that its not persistent in any way so restarting Camel will restart the lastpolltime variable and you can potentially consume the same file again. You can disable this algorithm with the new option consumer.timestamp=false or setting the consumer.alwaysConsume=true. Camel keeps track of consumed files which leads to a memory leak for large number of files in Camel 1.5.0 and older. This was replaced with a LRU cache in Camel 1.6.0. Notice: This algorithm has been marked for @deprecation and has been removed in Camel 2.0. We strongly encourage you to use a different strategy for matching new files: such as deleting or moving the file after processing, then new files is always if there exists a file in the directory to poll.

Cannot move/delete file after processing on Windows There is a potential issue on Windows platform with Camel 1.5.x. That it cannot move or delete the file after processing. You should get an exception thrown. The workaround is to convert the body to a String after consuming using convertBodyTo: eg from("file://inbox").convertBodyTo(String.class).to("file://outbox");. There should be a fix in Camel 1.6.0 to remedy this, but we would like to get feedback on this issue. So if you are on Windows please give it a go.

URI Options

Default behavior for file consumer

• By default, the file is locked for the duration of the processing.
• After the route has completed, files are moved into the .camel subdirectory, so that they appear to be deleted.
• The File Consumer will always skip any file whose name starts with a dot, such as ., .camel, .m2 or .groovy.
• Only files (not directories) are matched for valid filename, if options such as: consumer.regexPattern, excludeNamePrefix, excludeNamePostfix are used. Notice: this only works properly in Camel 1.5.0, due to issue CAMEL-920.

Default Behavior Changed in Camel 1.5.0

In Camel 1.5, the file consumer will avoid polling files that are currently in the progress of being written (see option consumer.exclusiveReadLock). However, this requires Camel to be able to rename the file for its testing. If the Camel user has not got the relevant rights on the file system, you can set this option to false to revert to the default behavior of Camel 1.4 or older.

The recursive option has changed its default value from true to false in Camel 1.5.0.

Move and Delete operations

Any move or delete operation is executed after (post command) the routing has completed. So, during processing of the Exchange the file is still located in the inbox folder.
Let's illustrate this with an example:

    from("file://inbox?m
oveNamePrefix=done/").to("bean:handleOrder");

When a file is dropped in the inbox folder, the file consumer notices this and creates a new FileExchange that is routed to the handleOrder bean. The bean then processes the File. At this point in time the File is still located in the inbox folder. After the bean completes, and thus the route is completed, the file consumer will perform the move operation and move the file to the done sub folder.

By default, Camel will move consumed files to the sub-folder .camel relative to where the file was consumed.

Available in Camel 1.6.0 or newer
We have introduced a pre move operation to move files before they are processed. This allows you to mark which files has been scanned as they are moved to this sub-folder before being processed.
The following options support pre move:

• preMoveNamePrefix
• preMoveNamePostfix
• preMoveExpression

    from("file://inbox?preMoveNamePrefix=inprogress/").to("bean:handleOrder");

You can combine the pre move and the regular move:

    from("file://inbox?preMoveNamePrefix=inprogress/&moveNamePrefix=../done/").to("bean:handleOrder");

So in this situation the file is in the inprogress folder when being processed, and after it's processed it's moved to the done folder.

Message Headers

The following message headers can be used to affect the behavior of the component:

Common gotchas with folder and filenames

When Camel is producing files (writing files) there are a few gotchas how to set a filename of your choice. By default Camel will use the message ID as the filename, and since the message ID is normally a unique generated ID, you will end up with filenames such as: ID-MACHINENAME\2443-1211718892437\1-0. Such a filename is not desired and therefore best practice is to provide the filename in the message header org.apache.camel.file.name.

The sample code below produces files using the message ID as the filename:

from("direct:report").to("file:target/reports");

To use report.txt as the filename you have to do:

from("direct:report").setHeader(FileComponent.HEADER_FILE_NAME, constant("report.txt")).to( "file:target/reports");

Canel will, by default, try to auto-create the folder if it does not exist, and this is a bad combination with the UUID filename from above. So if you have:

from("direct:report").to("file:target/reports/report.txt");

And you want Camel to store in the file report.txt and autoCreate is true, then Camel will create the folder, target/reports/report.txt/. To fix this, set autoCreate=false and create the folder, target/reports manually.

from("direct:report").to("file:target/reports/report.txt?autoCreate=false");

With auto-create disabled, Camel will store the report in report.txt as expected.

File consumer, scanning for new files gotcha

This only applies to Camel 1.x
The file consumer scans for new files by keeping an internal modified timestamp of the last consumed file. So if you copy a new file that has an older modified timestamp, Camel will not pick up this file. This can happen if you are testing and you copy the same file back to the folder that has just been consumed. To remedy this, modify the timestamp before copying the file back.

Filename Expression

In Camel 1.5 we have support for setting the filename using an expression. This can be set either using the expression option or as a string-based File Language expression in the org.apache.camel.file.name header. See the File Language for some samples.

Samples

Read from a directory and write to another directory

from("file://inputdir/?delete=true").to("file://outputdir")

Listen on a directory and create a message for each file dropped there. Copy the contents to the outputdir and delete the file in the inputdir.

Read from a directory and process the message in java

from("file://inputdir/").process(new Processor() {
  public void process(Exchange exchange) throws Exception {
    Object body = exchange.getIn().getBody();
    // do some business logic with the input body
  }
});

The body will be a File object pointing to the file that was just dropped into the inputdir directory.

Read files from a directory and send the content to a jms queue

from("file://inputdir/").convertBodyTo(String.class).to("jms:test.queue")

By default the file endpoint sends a FileMessage which contains a File as body. If you send this directly to the JMS component the JMS message will only contain the File object, not the content. By converting the File to a String the message will contain the actual file contents, which is probably what you want.

The route above using Spring DSL:

   <route>
      <from uri="file://inputdir/"/>
      <convertBodyTo type="java.lang.String"/>
      <to uri="jms:test.queue"/>
   </route>

Writing to files

Camel is, of course, also able to write files, i.e. produce files. In the sample below we receive some reports on the SEDA queue that we processes before they are written to a directory.

public void testToFile() throws Exception {
    template.sendBody("direct:reports", "This is a great report");

    // give time for the file to be written before assertions
    Thread.sleep(1000);

    // assert the file exists
    File file = new File("target/test-reports/report.txt");
    file = file.getAbsoluteFile();
    assertTrue("The file should have been written", file.exists());
}

protected JndiRegistry createRegistry() throws Exception {
    // bind our processor in the registry with the given id
    JndiRegistry reg = super.createRegistry();
    reg.bind("processReport", new ProcessReport());
    return reg;
}

protected RouteBuilder createRouteBuilder() throws Exception {
    return new RouteBuilder() {
        public void configure() throws Exception {
            // the reports from the seda queue is processed by our processor
            // before they are written to files in the target/reports directory
            from("direct:reports").processRef("processReport").to("file://target/test-reports");
        }
    };
}

private class ProcessReport implements Processor {

    public void process(Exchange exchange) throws Exception {
        String body = exchange.getIn().getBody(String.class);
        // do some business logic here

        // set the output to the file
        exchange.getOut().setBody(body);

        // set the output filename using java code logic, notice that this is done by setting
        // a special header property of the out exchange
        exchange.getOut().setHeader(FileComponent.HEADER_FILE_NAME, "report.txt");
    }

}

Using expression for filenames

In this sample we want to move consumed files to a backup folder using today's date as a sub-folder name:

from("file://inbox?expression=backup/${date:now:yyyyMMdd}/${file:name}").to("...");

See File Language for more samples.

Write to subdirectory using FileComponent.HEADER_FILE_NAME

Using a single route, it is possible to write a file to any number of subdirectories. If you have a route set up as follows:

  <route>
    <from uri="bean:myBean"/>
    <to uri="file:/rootDirectory"/>
  </route> 

You can have myBean set the FileComponent.HEADER_FILE_NAME to values such as:

FileComponent.HEADER_FILE_NAME = hello.txt => /rootDirectory/hello.txt
FileComponent.HEADER_FILE_NAME = foo/bye.txt => /rootDirectory/foo/bye.txt 

This allows you to have a single route to write files to multiple destinations.

See Also

• Configuring Camel
• Component
• Endpoint
• Getting Started

• File Language
• FTP