UltraESB Configuration

Transports

Transport Listener

● TransportListeners – accept / create messages and dispatch to proxy

– Can be event driven or Polling● HTTP/S, File, SFTP, JMS..

– Could be transactional – e.g. JMS

– Can be blocking or non-blocking

– The payload of the received message is in the MessageFormat picked by the TL

– Static, per ESB node – no changes at runtime

Transport Sender

● Sends an outgoing message using the selected transport

● Selected by the ESB based on the URL of the endpoint address

● Same transport can have multiple sender configurations– e.g. JMS, HTTP (with/without proxy) etc

– {http-proxy}http://host:port/path

● Can be blocking or non-blocking– Check for errors after sendToEndpoint() call for blocking

HTTP/S Transport

● Non-Blocking, uses Zero-Copy via RAM disk● RequestFilters

– authentication, service resources (WSDL, WADL, XSD), or custom..

● Supports HTTP Basic and Digest auth and Amazon S3 (outgoing)– Authentication can be against any Spring Security backend such as LDAP,

DB etc.

● Two way SSL and client certificate DN and attributes could be used during mediation

● Can assign custom error handlers (e.g. 404)● Writes error log● Can tune maximum payload size, connections etc

● Sender can use a proxy server and unzip responses, or keep compressed

● SSL remote cert/hostname validation can be turned off● Troubleshooting

– Connection Debug (log4j.properties, UTerm/UConsole)

– SSL debug (Only via system property in wrapper.conf/ultraesb.sh)

● Response validators can detect HTML error pages etc● Be careful with HW Lbs that might terminate connections

– Make the ESB socket timeout less than the remote party timeout

● Requires tuning for optimal performance● Connection and Socket timeouts

JMS Transport

● Configure via Spring with additional JARs● SSL with MQ may require IBM JCE and JSSE

providers● Dynamically scales as per Spring JMS

semantics

Polling Transports (Listener)

● File / Email / Timer● Configurations

– Threads (default 4)

– ultra.polling.concurrent● Single threaded if false

– (Note: IMAP may support concurrent downloads)

– ultra.polling.concurrent_execution● Limit for concurrent execution

– Default MAX_INT, Setting to 1 => single threaded

● CRON or start delay / polling interval / count

File Transport

● Supports file, s/ftp, ftps● Poll for files in local or remote file systems

– File URL or path pattern

– Specify operation after processing● Delete / Move – can be troublesome on Windows*● Move with time stamp appended to file name

– Based on CRON or delay

– Fetch files that are at lease <n> old

– When clustering is enabled, a node will acquire a cluster lock before downloading each file, to prevent the same file being downloaded concurrently

● When clustering is not available, defaults to file locking (Note: Windows*)

● Upload files– Can specify an optional timestamp to apply to the

name

– FastSFTPConnector available for always-connected SFTP

Can use certificate based authentication for SFTP

Email Transport

● Poll POP3/IMAP accounts, or send via SMTP● Can move email to folders after processing, or leave them on server● Ability to process multiple emails concurrently● Preserve headers or Remove headers

– Based on comma separated list● e.g. “To, Subject, From”

● Main part identification – list of content types in priority, other MIME parts become attachments

● mail.pop3|imap|smtp.* or other SSL properties as usual● Endpoint properties can specify Email headers (e.g. Subject)● Turn on debugging for detailed conversation

Custom TCP / New Transports

● TCP/S transport is an abstract framework– MLLP/S is a concrete example for HL7

– Can cater to custom TCP based protocols

● Proprietary transports may require custom implementations– e.g. based on third party libraries etc

– LogB, FIX etc

Other - Transports

● Support for WebSockets in the pipeline● Support for SWIFT, FIX, MLLP available● AMQP support is available

– No production user yet

● AS2 protocol support– Extensive, e.g. http://as2gateway.org

UltraESB is a Spring application

● In reality, UltraESB is a Spring application● The standalone deployment is always

recommended● The [almost never used] Web container

deployment model too triggers Spring initialization– This should ONLY be used where JTA XA is

required over a JEE server such as Jboss

Root Configuration● Static Configuration

– Root configuration file is conf/ultra-root.xml

– Defines the key Spring beans ● Should be used to tune the UltraESB specifics

– Clustering, HA, JMX, FileCache, Thread Pools etc

– Static includes● conf/ultra-custom.xml

– For users to define custom Spring beans– Define one or more

● monitoring/ultra-metrics.xml (Monitoring aspects)

– Any changes to static configuration requires a restart

● Dynamic includes– Deployment units from conf/deployments

– DUs can be added/updated/removed during use

conf/ultra-root.xml

● File-cache– Used by the HTTP/S transports, and transformations as

the default internal message format

– PooledMessageFileCache● Originally used memory mapped files, but now deprecated,

and disk based.● Fine for development time, but do not run benchmarks● Defaults to “tmp” directory● 10 files for development is fine (grows if required)

– RAMDiskFileCache (Next)

RAMDiskFileCache

● RAM disk must be created at the OS level– sudo mount -t tmpfs -o size=2048M tmpfs /tmp/ram/

– edit /etc/fstab and add line for persistence● tmpfs /tmp/ram tmpfs defaults,size=2048M

– Specify path, number of initial files, warn/stop thresholds, max files on RAM disk, and overflow

– Reaching stop threshold implies a file cache that's too small, or a file leak*

– Size is based on concurrency and Message size● Overflow to a disk {java.io.tmp} is possible

Setting System Properties

● Sometimes, easier to define/manage within config– e.g. MQ connectivity over SSL

<bean id="system-properties" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">

<property name="targetObject" value="#{@systemProperties}" />

<property name="targetMethod" value="putAll" />

<property name="arguments">

<util:properties>

<prop key="javax.net.ssl.keyStore">conf/keys/esb.jks</prop>

<prop key="javax.net.ssl.keyStorePassword">password</prop>

<prop key="javax.net.ssl.trustStore">conf/keys/esb.jks</prop>

<prop key="javax.net.ssl.trustStorePassword">password</prop>

</util:properties>

</property>

</bean>

● [Note: System properties can be also set though the conf/wrapper.conf; or via the bin/ultraesb.sh during development]

Use and Encryption of Properties

● Externalize properties with a Spring PropertyPlaceholder

● To encrypt properties such as passwords– Use simple / non-secure password

– Use secure password, with master password as environment variable

– Use bin/encrypt.sh to encrypt values

Example of secured properties<bean id="secure-property-configurer" class="org.jasypt.spring.properties.EncryptablePropertyPlaceholderConfigurer">

<constructor-arg>

<bean class="org.jasypt.encryption.pbe.StandardPBEStringEncryptor">

<property name="config">

<!-- Use this to specify the Jasypt password as an Environment variable (e.g. APP_PASSWORD=jasypt)-->

<bean class="org.jasypt.encryption.pbe.config.EnvironmentStringPBEConfig">

<property name="algorithm" value="PBEWithMD5AndDES"/>

<property name="passwordEnvName" value="APP_PASSWORD"/>

<property name="providerClassName" value="org.bouncycastle.jce.provider.BouncyCastleProvider"/>

<property name="providerName" value="BC"/>

</bean>

</property>

</bean>

</constructor-arg>

<property name="locations">

<list>

<value>file:conf/sgx-config.properties</value>

</list>

</property>

</bean>

Custom JCE/JSSE Providers

● Use Spring to define– e.g. for MQ connectivity via strong SSL

<bean id="jce-providers-1" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">

<property name="targetClass" value="java.security.Security"/>

<property name="staticMethod" value="java.security.Security.insertProviderAt"/>

<property name="arguments">

<list>

<bean class="com.ibm.jsse2.IBMJSSEProvider2"/>

<value>1</value>

</list>

</property>

</bean>

Environment

● Configure a few environment specifics– OnTheFlyCompilation

– binaryClassReloading

– deploymentUnitsEnabled

– UnitTestMode

● Predefined environments– e.g. dev, unit_test, sample, test, stage, prod

Miscellaneous

● Clustering, HA and FO– Separate set of slides

● Metrics / Monitoring– monitoring/ultra-metrics.xml

– Separate set of slides

● Cache Manager – Configure underlying EHCache

● Zabbix Agent– Expose JMX metrics over Zabbix native protocol

– Specify host/IP and port

JMX Connector

● JMX is used for management and monitoring● Local vs Remote JMX

– Local connectivity is not always stable

– Remote JMX connectivity is preferred● Secured via password in property file● Secured via LDAP / AD

● UConsole / UTerm / Zabbix requires JMX

ultra-custom.xml

● Use to define custom Spring beans and transports– e.g. Easier to keep JMS specifics separated

● Define transaction managers, connection pools● Define any common artifacts

– e.g. services, endpoints, error handlers

– e.g. health check service, default error handler

● Define AuditInterceptors etc

Database Connectivity

● Use Connection Pools– e.g. bonecp

● Define local transaction manager from Spring– <bean id="esresb-transaction-manager"

class="org.springframework.jdbc.datasource.DataSourceTransactionManager">

– <property name="dataSource" ref="esresb-pool"/>

– </bean>

<bean id="esresb-pool" class="com.jolbox.bonecp.BoneCPDataSource" destroy-method="close"> <property name="poolName" value="esresb" /> <property name="driverClass" value="oracle.jdbc.OracleDriver" /> <property name="jdbcUrl" value="${oracle.jdbc-url}" /> <property name="username" value="${oracle.jdbc-username}"/> <property name="password" value="${oracle.jdbc-password}"/> <property name="maxConnectionsPerPartition" value="20"/> <property name="minConnectionsPerPartition" value="2"/> <property name="partitionCount" value="2"/> <property name="statementsCacheSize" value="10"/> <property name="acquireRetryAttempts" value="3"/> <property name="acquireRetryDelayInMs" value="5000"/> <property name="acquireIncrement" value="2"/> <property name="driverProperties"> <util:props> <prop key="SetBigStringTryClob">true</prop> </util:props> </property> <property name="statisticsEnabled" value="true"/> <property name="disableJMX" value="false"/> <property name="connectionTestStatement" value="select 1 from dual"/> <property name="idleConnectionTestPeriodInSeconds" value="300"/> <property name="idleMaxAgeInSeconds" value="600"/> <property name="maxConnectionAgeInSeconds" value="3600"/> <property name="lazyInit" value="false"/> <property name="connectionTimeoutInMs" value="5000"/> <!--<property name="logStatementsEnabled" value="true"/>--> </bean>

Define custom AuditSupport<bean id="auditSupport" class="org.adroitlogic.ultraesb.core.helper.audit.AuditSupportImpl">

<constructor-arg name="dataSource" ref="esresb-pool"/>

<property name="auditRecordMapper">

<bean class="org.adroitlogic.ultraesb.core.helper.audit.AuditRecordMapper">

<constructor-arg name="tableName" value="AUDIT_LOG"/>

<constructor-arg name="saveUnmappedWithDefault" value="false"/>

<constructor-arg name="dbType" value="ORACLE"/>

<property name="cluster" value="CLUSTER"/>

<property name="source" value="SOURCE"/>

<property name="messageId" value="MESSAGE_ID"/>

<property name="server" value="SERVER"/>

<property name="serviceId" value="SERVICE_ID"/>

<property name="subject" value="MESSAGE"/>

<property name="tag" value="TYPE"/>

<property name="payload" value="PAYLOAD"/>

<property name="headers" value="HEADERS"/>

<property name="timestamp" value="TIMESTAMP"/>

<property name="transport" value="TRANSPORT"/>

<property name="propertyMapping">

<map>

<entry key="subj" value="SUBJECT"/>

<entry key="sev" value="SEVERITY"/>

</map>

</property>

</bean>

</property>

</bean>

Security

● HTTP Authentication● SSL Configurartion

– 2-way SSL

● WS-Security● XACML

HTTP Authentication

● Enforcement, based on Request filters– BasicAuthenticationFilter

– DigestProcessingFilter

– e.g.Sample 110

● Validated against standard Spring Security authentication providers– Easier configuration

● Mediation– getHTTPSupport().getAuthenticatedUsername(msg)

– mediation.getHTTPSupport().getAuthenticatedRolesString(msg)

● Outgoing can be pre-emptive● Configured at

– Endpoint

<u:inDestination>

<u:address>http://localhost:8281/service/rest-mock</u:address>

<u:property name="ultra.http.auth_username" value="asankha"/>

<u:property name="ultra.http.auth_password" value="adroitlogic"/>

<u:property name="ultra.http.auth_scheme" value="basic"/>

</u:inDestination>

– Mediation● getHTTPSupport().addPreemptiveBasicAuthentication()● mediation.getHTTPSupport().setAuthenticationCredentials()

SSL Configuration

● Support TLS● Transport Listener and Sender accepts key stores

– Trust store – defaults to Java

– Identity store● Must be configured

● Can turn off remote certificate validation for testing / debugging– NonProductionNoRemoteCertValidation

● Remote hostname verification– HostnameVerifier: Default, Strict, DefaultAndLocalHost

● Patch security policy files for high strength security support● Use SSL debugging to detect serious issues

– Note: Captured on standard output/error and not in the log file

SSL Client Authentication

● Listener– SslVerifyClient: none, optional, required

– Read in Mediation● HttpConstants.SSL_CLIENT_DN● HttpConstants.SSL_CLIENT_CERTS

● Sender will send automatically if reqested– Identity store must be configured

WS-Security

● WSSecurityManager - Spring bean– Does not implement all of WS-Security, especially WS-Policy support

– timestamp, sign, encrypt ● plus combinations

– UsernameToken authentication● Plain, with Digest, with Timestamp

– Incoming verified against Spring Security authentication provider

● Verify received messages● Based on configured keystores● Read during mediation

– MessageSecurityContext.USER_NAME | USER_ROLES | USER_CERTS

● Consider cost of message level security vs SSL

XACML

● For fine grained authorization● Based on JBoss XACML● Could be overkill?

– No production customers

● See sample #216 if interested