Apache Camel + Smooks

22 Feb 2010 - Tom Fennelly

I did a little playing with Camel over the weekend… had a go at integrating Smooks.

I think those using Camel may find it interesting as another option. The integration was simple enough and can be seen in this svn workspace (including tests etc). With better knowledge of Camel, I’m sure there’s a better way of integrating it, but at least it’s a start.

At the moment, you can map String or Java results into the out-message of the Camel Exchange, so you can do templating or java binding (not exposing validation results yet) of XML and non-XML data. So you can do lots of the normal Smooks type processing.

The aspect I think most Camel users might find interesting is the potential wrt splitting and routing of huge messages (XML and non-XML). Basically… being able to split and route huge messages as you stream them i.e. not the two step method that seems to be there at the moment – 1) split out all messages 2) route all messages.

Lets look at a simple example, where we have an XML message stream containing customer account activity logs (or… could be CSV formatted). We want to route each log event to JMS queues based on nationality.

The RouteBuilder config (can also configure the SmooksProcessor via Java ala DSL) …

public class MyRouteBuilder extends RouteBuilder {

    public void configure() throws Exception {
        from("file:src/data/logs?noop=true").
            process(new SmooksProcessor("/configs/log-split.xml", this));

        from("jms:queue:ireland").process(new LogProcessor("ie"));
        from("jms:queue:greatbritain").process(new LogProcessor("gb"));
    }
}

And then the Smooks “log-split.xml” config…

<smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
    xmlns:jb="http://www.milyn.org/xsd/smooks/javabean-1.3.xsd"
    xmlns:camel="http://www.milyn.org/xsd/smooks/camel-1.4.xsd">

    <!-- Create a new ActivityLogEvent for each log entry... -->
    <jb:bean beanId="logEvent" class="com.acme.activity.ActivityLogEvent"
             createOnElement="activity/event">
        <jb:value property="transactionId" data="event/@txId" />
        <jb:value property="customerId" data="event/customer/@customerId" />
        <jb:value property="nationality" data="event/customer/countryCode" />
    <!-- etc... -->
    </jb:bean>

    <!-- Route GB based customer activity... -->
    <camel:router beanId="logEvent" toEndpoint="jms:queue:greatbritain"
                  routeOnElement="activity/event">
        <condition>logEvent.nationality == 'GB'</condition>
    </camel:router>

    <!-- Route Irish based customer activity... -->
    <camel:router beanId="logEvent" toEndpoint="jms:queue:ireland"
                  routeOnElement="activity/event">
        <condition>logEvent.nationality == 'IE'</condition>
    </camel:router>

</smooks-resource-list>

The <jb:bean> config creates an ActivityLogEvent bean instance for every activity log. The following <camel:router> configs conditionally route the ActivityLogEvent bean instances to the specified JMS endpoints based on the country-code on the event log. Of course, using Smooks Templating, you could apply a template to the ActivityLogEvent to produce some other data format and route that instead (a simple template generated String bean bound into the Smooks BeanContext).

The size of the event log is irrelevant here – could be GBs. Only one ActivityLogEvent instance is in memory at any given time (a new instance created for each event). The routing is performed as the event log is being streamed i.e. not batched up for routing in a later step. Assumption here is that there’s a way of telling the Camel File component to forward the file as a stream/reader.

The current implementation is available in SVN here. Since I’m not overly familiar with Camel, I’m sure there are things we could do better/different. Please let us know!!!