File transfer has been a common and serviceable way to exchange information among computer systems since the dawn of digital time. Even today, despite the growing use of transactional integration technologies like service-oriented architecture (SOA) and enterprise service bus (ESB), file transfer remains not only viable but also dominant across a range of integration use cases.
According to Hohpe and Woolf in their seminal book, “Enterprise Integration Patterns”, organizations generally choose to integrate applications with any of three basic patterns, or a combination of these:
- File transfer—Applications send and receive files with standards-based protocols. They may also convert the files between different formats.
- Shared databases—Applications store the data to be shared with other applications in a common database.
- SOA/messaging—Each application programmatically connects to a common messaging bus, and uses a shared set of messages to exchange data.
File transfer is usually the least complex (and thus least expensive) of these options, so it is often the best choice if transactional or “near zero” latency is not an absolute requirement. In a recent conversation with an IDC analyst, I heard that 60% of all transactional data continues to be transferred in batch files… only 40% are small transactional files. And the primary reason for choice of batch versus transactional data exchange is cost!
So when is it recommended to leverage a managed file transfer solution for integration versus SOA or another approach?
- When data interchange is only required periodically (e.g., batch updates of financial records between finance systems supporting different divisions/locations globally). In such cases a real-time messaging infrastructure would be overkill.
- Where secure transport across public networks is important. Files are efficiently transported across the Internet using secure FTP/HTTP protocols, whereas asynchronous messaging can make it trickier to sort out integration issues, especially when working with business partners.
- When you can rely on standard file formats common to the applications you need to integrate; e.g., COBOL-, text- or XML-based files.
- When transferring unstructured data files, such as spreadsheets, audio, and video.
- When integration includes both people and systems as part of a business process.
The above use cases illustrate some of the advantages of file-based integrations for business users: file transfer reduces complexity (read cost), and potentially reduces the risk of transmission failure and therefore helps optimize the reliability of key processes.
The power and flexibility of today’s leading managed file transfer solutions enable them to ensure predictable delivery and meet data security and regulatory compliance requirements, while retaining the ease-of-use and (comparatively) straightforward integration with disparate systems that made them popular in the first place.
Indeed, managed file transfer and messaging-based architectures are converging—putting Managed File Transfer-as-a-Service (MFTaaS) as a key service component for multi-enterprise integration. More on that in Part 2 of this post.