The FTP Binding

Owned by Anonymous
Last updated: Oct 27, 2012 by AnonymousVersion comment
6 min read

Many enterprise architectures rely on FTP to integrate loosely-coupled systems. Fabric3 provides the ability to receive inbound PUTs and to perform outbound PUTs to external FTP servers via a streaming infrastructure. This enables the transfer of large data sets without having to write files to disk. Inbound PUTs are sent to services exposed over the Fabric3 FTP binding while outbound PUTs are sent using a reference configured with the FTP binding. As will be detailed below, in addition to allowing streaming data transfers, this FTP binding simplifies application code by removing the need to interact directly with an FTP protocol API.

Implementing an FTP Service

To implement a servce that receives incoming FTP PUTs, define a Java interface with a single method that takes a String file name and InputStream parameters as shown in the following example:

public interface FtpDataTransferService {

   void transferData(String fileName, InputStream data) throws Exception;
}

The following is the service implementation:

public class FtpDataTransferServiceImpl implements FtpDataTransferService {

   public void transferData(String fileName, InputStream data) throws Exception {
      // process the stream
   }
}

One common use case for receiving FTP files is to place a Fabric3 runtime in a DMZ where it receives inbound puts and forwards the stream via another protocol (e.g. web services) to another server in a trusted zone. This configuration avoids directly exposing servers in a domain to inbound traffic while minimizing performance overhead as the data is streamed from the origin server through the DMZ runtime to the trusted zone. Further, these operation can be configured to be transactional.

Implementing such a fowarding service is straightforward. The previous example can be extended to forward the InputStream to another service via a reference bound to another protocol:

public class FtpDataTransferServiceImpl implements FtpDataTransferService {

   @Reference
   protected InternalService internalService;


   public void transferData(String fileName, InputStream data) throws Exception {
      // forward over a different binding
      internalService.write(data);
   }
}

Handling Different Data Types

The FTP binding supports both ASCII and binary data. To determine the type of an incoming stream, Fabric3 provides the F3RequestContext API. The F3RequestContext is a specialized form of the SCA RequestContext API and is accessed through injection using the @Context annotation. The following is an example of how to use the API to determine the type of incoming data:

public class FtpDataTransferServiceImpl implements FtpDataTransferService {

   @Context
   protected F3RequestContext context;
   public void transferData(String fileName, InputStream data) throws Exception {
      String type = context.getHeader({String.class, "f3.contentType");
      if ("BINARY".equals(type)) \{
         handleBinary(data);
      } else {
         handleText(fileName, data);
      }
   }
}

The header options for f3.contentType are TEXT or BINARY. In addition to the SCA API jar (fabric3-sca), the Fabric3 API jar (fabric3-api) is required to be on the classpath to use the F3RequestContext API. This jar is automatically made available to all deployed contributions.

Provisioning Services

A service is bound over FTP by specifying a relative URI at which it will be made available to clients. The relative URI maps to the directory FTP clients PUT data to. For example, the following maps the FtpDataTransferService to the "transfer" directory:

<component name="FtpService">
   <implementation.java class="...."/>
   <service name="FtpDataTransferService">
      <f3:binding.ftp uri="transfer"/>
   </service>
</component>

After logging in at the FTP base address for the domain (configuring the Fabric3 streaming FTP server is described below), clients can send data to the service by changing the working directory to transfer and issuing a PUT command. The data will then be streamed to the component.

A service can also be mapped to the root FTP directory using the '/' token:

<component name="FtpService">
   <implementation.java class=..."/>
   <service name="FtpDataTransferService">
      <f3:binding.ftp uri="/"/>
   </service>
</component>

If a service is mapped to the root FTP directory, clients do not need to change working directories to send data after connecting to the domain FTP server.h6. Configuring Security
FTP-bound services can be configured to require authentication-based security using a policy definition. The following policy definition shows how to secure a service using an authentication policy:

<?xml version="1.0" encoding="ASCII"?>
<definitions xmlns=" [http://docs.oasis-open.org/ns/opencsa/sca/200903]" xmlns:sca=" [http://docs.oasis-open.org/ns/opencsa/sca/200903]"  
   targetNamespace="urn:foo.com:policy" xmlns:f3="urn:fabric3.org">

   <policySet name="authenticationPolicy" provides="sca:authentication.message" appliesTo="@name='ftpDataTransferService'">
      <f3:security user="user" password="password"/>
   </policySet>
</definitions>

The above policy requires FTP clients to login prior to sending data to the service.

Connecting to External FTP Servers

The FTP binding can also be used to connect to remote FTP servers. The following demonstrates how to bind a reference to a remote FTP server using authentication and PASSIVE data transfer:

<component name="DataTransferClient">
   <implementation.java class="..."/>
   <reference name="ftpDataTransferService">
      <f3:binding.ftp uri="ftp.baz.com:2000" requires="sca:authentication.message" mode="PASSIVE"/>
   </reference>
</component>

The above authentication intent is matched to the policy definition below:

<?xml version="1.0" encoding="ASCII"?>
<definitions xmlns=" [http://docs.oasis-open.org/ns/opencsa/sca/200903]" xmlns:sca=" [http://docs.oasis-open.org/ns/opencsa/sca/200903]"
   targetNamespace="urn:fabric3.org" xmlns:f3="urn:fabric3.org">
   <policySet name="authenticationPolicy" provides="sca:authentication.message" appliesTo="../@name='DataTransferClient'">
      <f3:security user="foo" password="password"/>
   </policySet>
</definitions>

Setting the Data Type

The F3RequestContext API can be used to set the outgoing data type transfer to ASCII or BINARY via request headers. The header name and values are specified in org.fabric3.api.ftp.FtpConstants (contained in the fabric3-api jar). For example:

public class FtpClientImpl implements FtpClient {

   @Context
   protected F3RequestContext context;
   
   @Refeence&nbsp;
   protected FtpService service;

   public void transferData(InputStream data) {
      context.setHeader(ftpConstants.HEADER_CONTENT_TYPE, FtpConstants.BINARY_TYPE);
      // to set ASCII use:  
      context.setHeader(ftpConstants.HEADER_CONTENT_TYPE, FtpConstants.ASCII_TYPE);
      // send data
      service.send(data);
   }
}

Note the header must be set on each invocation.

Sending proprietary FTP commands before a PUT (STOR) operation

Some FTP servers require custom command sequences prior to a PUT operation. Sequences of commands can be configured on the reference binding using the <commands> element. These sequences will be executed prior to the PUT operation:

<component name="DataTransferClient">
   <implementaiton.javaclass="..."/>
   <reference name="ftpDataTransferService">
      <f3:binding.ftp uri="ftp.baz.com:2000" requires="sca:authentication.message" mode="PASSIVE">
         <commands>
            <command>custom command1</command>
            <command>custom command2</command>
         </commands>
      </f3:binding.ftp>
   </reference>
</component>

Setting up the FTP Server

The FTP profile installs an embedded FTP server. This server is used to listen for incoming requests and dispatch them to bound services. The FTP server has a number of configuration options:

  • commandPort - The port to accept FTP commands on. The FTP server is set to listen on port 2000 by default.
  • minPassivePort and maxPassivePort - The lowest and highest port number for passive connections. The default passive port range for PUT operations is set to 6000-7000.
  • listenAddress - The machine address the server should bind to. Used for multi-homed machines. The default is set to the value returned by InetAddress.getLocalHost().
  • idleTimeout - The timeout in milliseconds to use for socket connections. The default is one minute.
  • users - Contains users and password information.

FTP defaults can be changed by editing the runtime system configuration. In the standalone runtime, the system configuration file (systemConfig.xml) is located in the profile subdirectories under /config. In the Maven and Webapp runtimes, it is part of the plugin configuration. The following provides an example of how to modify the default settings:

<ftp.server>
   <commandPort>2000</commandPort>
   <minPassivePort>6000</minPassivePort>
   <maxPassivePort>7000</maxPassivePort>
   <users>
      <user>password</user>
   </users>
</ftp.server>

In addition to the above configuration settings, user accounts (name and password) can be specified as show below:

<component name="FtpServerComponent">
   <implementation.composite name="sample:FtpServerExtension"/>
      <property name="config">
         <config xmlns="">
            <commandPort>2000</commandPort>
            <minPassivePort>6000</minPassivePort>
            <maxPassivePort>7000</maxPassivePort>
            <users><user>password</user></users>
        </config>
      </property>
</component>

Supported Commands

The following commands are supported by the FTP server:

  • USER
  • PASS
  • PASV
  • PASS
  • STOR
  • QUIT
  • SYST
  • CWD
  • LIST
  • TYPE

Custom commands

The FTP server may be extended to add custom commands. Custom commands must implement the org.fabric3.ftp.server.protocol.RequestHandler inteface and be configured as a system component in an extension contribution. For examples, see the fabric3-ftp-server module in the source repository.