Detailed Release Notes

PilotFish Software Release Notes – New Features & Improvements


Release 19R1 – Mar 2019


  • Updated Java Runtime To OpenJDK 11
    • To adjust to changes in Oracle JDK public update policies and cadence the bundled JRE has been changed to OpenJDK 11. We have a dedicated knowledge base article covering the change available here.
  • New Google Cloud Storage Support
    • Google Cloud Platform has established itself as a top-tier cloud provider option, offering customers compelling cost and performance features. In response to increased interest, we’ve added a dedicated Listener and Transport pair for Google Cloud Storage allowing for easy retrieval and storage to Google Compute Platforms’s Cloud Storage.
  • New Redis Support
    • Redis is the most popular high-performance key/value databases on the market today. In 19R1 we have added robust support for integrating with Redis using our Redis-XL declarative syntax. Similar in approach to the widely used API-XL and SQLXML, Redis-XL supports commands on Redis data structures such as strings, hashes, sets, and hyperloglogs.
  • New Hashing Processor
    • Augmenting our long-standing support for MD5 and SHA1 hashing, we’ve added a more general Hashing Processor in order to expose the full range of hashing and message authentication code (MAC) facilities available in the underlying security provider(s). This provides greatly expanded support for algorithms such as SHA2, SHA3, TIGER, WHIRLPOOL and many more.
  • New FHIR 4 Support
    • HL7.org released the FHIR R4 at the end of December 2018 which included thousands of change proposals and substantive changes. To accommodate these changes we’ve updated the PilotFish FHIR format reader and reference schemas to include this latest version along with support for past versions.
  • New JSON Web Token (JWT) Generator Processor
    • In previous releases, we added support for OAuth including support for JSON Web Token (JWT) token validation. In this release, we are taking this a step further by providing the ability to generate JWT tokens as well.
  • Improved EDI Support
    • There have been several improvements to our already industry-leading EDI handling including:
      • Addition of a new EDI Identification processor for quick extraction of key EDI details to make routing and processing decisions.
      • Increased validation for imperfect messages (misuse of sub-element delimiters, table data matching, etc)
      • Performance improvements to the core EDI Transformer
  • Improved API XL Support
    • API XL is our advanced RESTful API interaction functionality, providing the ability to construct robust remote API request/response data flows. Additional functionality has been added after continued real-world usage and feedback including:
      • Addition of PATCH, OPTIONS, HEAD, and TRACE HTTP methods
      • Support for disabling of hostname and certificate validation (particularly useful during the testing phase)
      • Added rawURL attribute for cases where the remote system needs URL strings that are non-standards compliant
      • Added ability to specify BinaryParts to MIME-MultiPart messages
  • Improved ACORD Format Support
    • Support has been added for ACORD TXLife v2.41, ACORD (P&C) XML v2.5, and AL3 v2018-2 standards. In addition, the AL3 Transformer has been improved to support more control over the XML generated including the ability to include the description in attributes and indexing for easier subsequent processing.
  • Improved JMS Listener and Transport
    • A new 2.0 version of the JMS Listener and JMS Transport have been developed and added to the software. This new version is much more performant than its predecessor, however, the original JMS Listener and Transport (now referred to as 1.0) have been retained in the products for backward compatibility.
  • Improved SQLXML Functionality
    • SQLXML and its related modules are responsible for communicating with relational databases and are by far one of the most heavily used modules in the software. We’ve introduced a handful of improvements to assist with compatibility (new toggle to have JDBC map value types), better null value handling and performance improvements.
  • Improved User Experience
    • In addition to the underlying technology improvements, many overall user experience improvements were also incorporated, including:
      • Native OS window decorations for better multi-monitor support
      • Ability to enable or disable interfaces from the eiConsole File Management UI
      • In eiConsole Testing Mode – automatically attempt to show XML output in pretty-print format
      • In eiConsole’s Local eiPlatform Emulator – Darkened DEBUG color and prefixed all lines with log level
      • eiPlatform Windows installer improvements including specifying Windows service start type and port to listen on
      • Surface more details of underlying XSLT engine exceptions in error messages and stack traces
  • Improved Directory / File Transport
    • A new File Name Conflict Pattern option has been added to improve I/O performance when there are thousands of same-named files are being written and ‘Create New’ naming conflict has been selected.
  • Improved Support for IIS FTP Server
    • Recent versions of IIS FTP Server are sensitive to PBSZ settings when connecting via secure FTPS. In 19R1 we provide a new option to explicitly set this value along with a default that will work in most cases.
  • Improved MongoDB Support
    • Support for connecting to MongoDB servers via SSL has been added.
  • Updated 3rd-Party Dependencies
    • Several supporting libraries have been updated to include upstream fixes.

Release 18R2 – Aug 2018


  • New Apache Kafka Listener and Transport
    • Apache Kafka is a distributed streaming system which is commonly used as a fault-tolerant, high-throughput/low-latency enterprise messaging system
    • We’ve added both a Kafka Listener and Transport that allow consuming and publishing messages on Kafka architectures
  • New AS2 Listener and Transport
    • AS2 is a secure point-to-point transmission protocol often used for transporting EDI Messages
  • New UDP Listener and Transport
    • UDP is a core Internet protocol which uses stateless datagrams to transmit data.
    • Given is minimalistic guarantees, UDP has a low amount of overhead and latency making it better suited for some use-cases
  • New Splunk® Support
    • Splunk is an output aggregator for enterprises, it allows for analysis and insight from machine-generated data and logs
    • We’ve added several modules to make integrating with Splunk simple and straightforward:
      • Splunk Search Listener – Execute Splunk search queries and process the output
      • Splunk Transport – Post transaction data as event data to Splunk
      • Splunk Processor – Send events to Splunk at specified stages of interface progression
      • Splunk Log Appender – A Log4J appender configuration provided in logging configuration file
  • New Microsoft Azure Storage Processor
    • We’ve expanded our cloud provider support and now include a Microsoft Azure Storage Processor to work with files and blobs storage on Azure
  • Added Regression System
    • Developed behind the scenes for months, we’re releasing our Regression System which allows for recording of data flows and replay on future releases to reduce regressions. The GUI has been integrated into our eiDashboard product to allow for convenient operation.
  • Added Capability to set Metadata on Sources and Targets along with the ability to search for them
    • The ability to add relevant metadata to formats and routes has been in the product for some time, in 18R2 we are expanding this capability in two ways:
      • Adding the ability to enter metadata on Sources and Targets
      • Adding the ability to search/filter on this metadata
  • Added support for additional Encryption Algorithms for Encryption Processors
    • AES-192, AES-256 and TwoFish as selectable algorithms (Note: Requires full-security installer, contact support@pilotfishtechnology.com if you need more information)
  • Added OGNL Member Access Security Support
    • OGNL is a powerful expression language that can be used for specifying dynamic configuration values
    • We’ve added the ability to enable or restrict specific packages, classes and/or methods via an ognlSettings.xml file for tailored security settings
  • Added XSLT 2.0/3.0/3.1 items to Data Mapper palette
    • As a follow up to the inclusion of Saxon PE in 18R1, we’ve gone further to add XSLT 2.0-3.1 functions and constructs to the Data Mapper tools palette
  • Added SSL Configurations to better support self-signed certificates and hostname validation options
    • Often while building interfaces in a dev environment, HTTPS/SSL endpoints will use non-commercial, self-signed certificates. To remove the need inject these temporary certificates into a custom trustStore we’ve added configuration options that can relax the SSL connection validations.
  • Added the ability to run Transformation Modules as Processors
    • When designing an interface, occasionally there will be a need to run a native transformation module mid-stream, for example, JSON to XML or vice versa. The traditional approach to do this would be to add a Call Route Processor to a separate route with a dedicated format. However,  as the number of interfaces scale, these types of single-purpose routes can distract from the overall flow.
    • In order to simplify interface design and construction, we’ve added the ability to call transformation modules directly as processors.
  • Added the ability to use Route Callouts within XSLT for dynamic value evaluation
    • Traditional code value lookups in XSLT are done with hard-coded data mapping templates, i.e., code ‘2’ signifies ‘Personal Email Address’
    • However, in the real world, keys and values are often dynamic or updated from time to time. This non-fixed nature requires additional effort to keep code value mappings up to date.
    • To address this, we’ve added the ability to call routes in order to be able to determine values dynamically.
    • By using routes in this way, we can take full advantage of the eiPlatform’s connectivity options and capabilities to look up values from databases, web services or any other endpoint the eiPlatform supports.
  • Improved EDI Support, Functionality and Performance
    • The EDI Transformer is now more capable of parsing EDI that contains invalid or out-of-place segments
    • We’ve improved version recognition for older 4010 EDI and X12 minor releases
    • Errors that occur while transforming EDI to XML now provide better segment and element context to help debugging
    • Numerous enhancements to parsing performance, especially when transforming EDI to XML and when X12 table data is not available
  • Improved XML to JSON Conversion Performance
    • After carefully reviewing how PilotFish converts XML contents to JSON, we were able to reduce the number of computationally-expensive operations further to improve conversion performance
  • Improved NCPDP Support
    • NCPDP schemas could be previously read into PilotFish’s Data Mapper using the generic XSD Reader, however, in practice there was some confusion over which file needed to be read in. To address this we’ve added a dedicated format reader for NCPDP that will check the file selected, and if not the correct one, the software will attempt to identify the right one to use.
  • Improved MongoDB Support
    • We’ve updated our support for MongoDB with a couple of changes:
      • Updated the Mongo libraries to support the latest MongoDB release
      • Added the ability to connect to MongoDB instances that do not require authentication
      • Added support for empty filters
  • Improved JavaScript Processor Functionality
    • Several improvements to the JavaScript Processor have been made in 18R2:
      • JavaScript files can now be stored in the route directory, allowing them to be more easily deployed with other configurations
      • More control over injecting the contents of Transaction Data into the script
      • Ability to read and write Transaction Attributes in the JavaScript file
      • A built-in code editor for on-the-fly editing of the JavaScript content
  • Improved eiConsole Look and Feel
    • The eiConsole has been updated to a newer Look and Feel that is more responsive and memory-efficient
    • Visual differences are minimal as efforts were undertaken to give a consistent experience
  • Improved eiConsole Menu Bar Layout
    • Due in part to its long history, the eiConsole’s menu bar had accumulated many sub-items under just a couple top-level menus.
    • With 18R2 we’ve arranged menu items more consistent with common expectations by utilizing sub-menus and better grouping related items
  • Improved Syntax Highlighting in Stage Output Viewer of Testing and Debug Modes
    • Drop-down to add the ability to toggle between no syntax, XML and JSON
    • XML and JSON now have collapsible code segments
  • Improved support for Character Encodings in XPath Value Retrieval Processor
    • Functionality and configuration options have been added to the XPath Value Retrieval Processor that will allow for:
      • Detection and use of declared character encoding in XML contents
      • Override character encoding declared in XML
  • Improved Logging Performance and Capabilities
    • One of the underlying changes to eiPlatform is the switch from Log4j to Log4j2
    • Log4j2 provides much better logging performance and enables more functionality
  • Improved XPath Processors by switching underlying engine
    • The XPath processors now use Saxon which provides XPath 2.0 & 3.1 support and enhances performance
  • Improved Registration Workflow for White-Labeled Installers
    • We’ve refined our registration process for white-labeled installers by reducing the steps necessary for users to go through manually
  • Improved Directory Transport performance for High File Volumes with Dynamic Naming
    • When Directory Transport is used with the ‘If File Exists’ setting, it is set to ‘Create New’.  It attempts to find the first non-conflicting filename by incrementing a counter
    • On interfaces with a very large number of files being created and dynamically named, it was possible to incur delays due to the conflict checking
    • As of 18R2, a new ‘File Name Conflict Pattern’ option was added in the Advanced tab which uses SimpleDateFormat patterns and defaults to ‘yyyyMMddHHmmss’ which should minimize the amount of conflict resolution operations necessary
  • Improved XSLT Transformation Performance
    • We’ve been able to improve XSLT transformation performance by utilizing more of the XSLT engine’s specific functionality
  • Improved Basic Statistics Functionality
    • Basic statistics are persisted to respond to REST API calls, storage of these statistics has been changed to a lower-overhead format
  • Updated Socrata Open Data Transport for API changes
    • The Socrata Open Data Transport has been updated to work with the current API
  • Updated ACORD Life And Annuity to Version 2.40.00
    • The ACORD Life and Annuity artifacts have been updated to the latest version (2.40.00) for the eiConsole for ACORD (available from ACORD.org)
  • Updated 3rd-Party Dependencies
    • Several supporting libraries have been updated to include upstream fixes

This is a unique website which will require a more modern browser to work! Please upgrade today!