DE4A Connector Installation and configuration guide

All details concerning the configuration and commissioning of the DE4A Connector. Minimal technical knowledge is required to follow the guide. It includes relevant aspects for a correct configuration, but may vary depending on web servers, application servers or environments under which the component is configured.

Installation

A prerequisite to build the Connector is to have at least Java 11 and Apache Maven 3.6 or later installed.

You should be able to compile entire packages from the parent POM file:

mvn clean install

It is also possible to compile each package separately by browsing to the folder and running the command above.

Package

The compilation process will be packaging the project into a .war file located on /target/ path, which should be deployable on any applications server. If you compile the parent pom, the IDK and Connector target paths will be created with their corresponding war files.

de4a-commons

de4a-commons project is now on maven central OSS Sonatype repository

Toop version v2.1.2-SNAPSHOT

Due to the lastest changes on de4a-commons Toop-connector-ng version should be 2.1.2-SNAPSHOT, so you may need to add following repo server on your maven settings

https://oss.sonatype.org/content/repositories/snapshots/

Connector configuration guide

For a correct configuration of the Connector, three main property files must be considered:

application.properties: main system configuration
phase4.properties: AS4 gateway configurations
log4j2.xml: logging configuration

Bellow, a working example of the application.properties file:

# Database properties
database.datasourceConf.url=jdbc:h2:mem:testdb
database.datasourceConf.driverClassName=org.h2.Driver
database.datasourceConf.username=sa
database.datasourceConf.password=password
database.datasourceConf.initializationMode=always
database.datasourceConf.jpaHibernate.dialectPlatform=org.hibernate.dialect.H2Dialect
database.datasourceConf.jpaHibernate.ddlauto=create-drop
database.datasourceConf.jpaHibernate.generateddl=true
database.datasourceConf.jpaHibernate.namingStrategy=org.hibernate.cfg.ImprovedNamingStrategy
database.datasourceConf.jpaHibernate.showSql=true
database.datasourceConf.jpaHibernate.formatSql=true

# H2 in-memory database console port (default 21080)
h2.console.port=21080

# i18n properties
spring.messages.basename=messages/messages
spring.messages.default_locale=en

# Spring allowing override beans
spring.main.allow-bean-definition-overriding=true

# Charset encoding
server.servlet.encoding.charset=UTF-8
server.servlet.encoding.enabled=true
server.servlet.encoding.force=true

# SSL context enabled (true|false)
ssl.context.enabled=false

# SSL configuration (optional when ssl.context.enabled is false, otherwise, it must be configured)
#ssl.keystore.type=
#ssl.keystore.path=
#ssl.keystore.password=
#ssl.truststore.path=
#ssl.truststore.password=

# Global flags for initializer
global.debug = true
global.production = false

# Instance name for logging
global.instancename = dev-from-ide

# DE4A Kafka settings
de4a.kafka.enabled=true
# Enables the standard logging separately of the Kafka messages. It is necessary for print metrics messages - (default: true)
de4a.kafka.logging.enabled=true
# Enables Kafka connection via HTTP (Only enable HTTP mode if outbound TCP connections are blocked from your internal network)
de4a.kafka.http.enabled=false

# Kafka server address (Eg.: de4a-dev-kafka.egovlab.eu:9092)
de4a.kafka.url=de4a-dev-kafka.egovlab.eu:9092
# Uncomment the following property and remove the above one if HTTP mode is enabled
# de4a.kafka.url=https://de4a-dev-kafka.egovlab.eu

# Establish a topic on kafka tracker - Pattern: de4a-<country-code>-<partner-name> - Eg.: de4a-se-egovlab - (default: de4a-connector)
de4a.kafka.topic=de4a-connector

# Logging metrics messages prefix - Default: DE4A METRICS
log.metrics.prefix=DE4A METRICS

# toop legacy kafka properties (Do not touch)
toop.tracker.enabled = false

# DSD base URL (Do not modify)
toop.dsd.service.baseurl = http://dsd.dev.exchange.toop.eu

# What AS4 implementation to use?
toop.mem.implementation = phase4

# Our AS4 sending AP endpoint (holodeck)
#toop.mem.as4.endpoint = http://localhost:8083/tc-webapp/as4

# Domibus server endpoint
# domibus.endpoint=

# SMP Client configuration stuff - Do not modify (default values)
smpclient.truststore.type = JKS
smpclient.truststore.path = truststore/de4a-truststore-test-smp-pw-de4a.jks
smpclient.truststore.password = de4a

# Spring As4 gateway implementation bean(provided: phase4GatewayClient and domibusGatewayClient).Implements eu.toop.as4.client.As4GatewayInterface
as4.gateway.implementation.bean=phase4GatewayClient

# External endpoints
# SMP endpoint Eg.: https://de4a-smp.egovlab.eu/
smp.endpoint=
# IDK endpoint Eg.: https://de4a-dev-idk.egovlab.eu/
idk.endpoint=

# IM response timeout
as4.timeout.miliseconds=30000

# Properties to create the http client connection through a proxy (optional)
#http.proxy.enabled=
#http.proxy.address=
#http.proxy.port=
#http.proxy.non-proxy=
#http.proxyUsername=
#http.proxyPassword=

# Required renamed proxy configuration for BDXRClient (if is needed, only uncomment)
#http.proxyHost=${http.proxy.address}
#http.proxyPort=${http.proxy.port}
#http.nonProxyHosts=${http.proxy.non-proxy}

From now on, we will explain the main and most critical configuration.

Database properties

Regarding database configuration and structure, the component creates an in-memory database through an H2 DB Engine, which will be created and deleted on each execution.

Datasource parameters

spring.datasource.url=jdbc:h2:mem:testdb
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=password
spring.jpa.database-platform=org.hibernate.dialect.H2Dialect
spring.datasource.initialization-mode=always
spring.jpa.hibernate.ddl-auto=create-drop
spring.jpa.generate-ddl=true

On the previous properties you also can specify any driver o connection configuration in order to stablish the connection to any database engine.

H2 in-memory database console

In order to access and manage information stored, until an external environment is created, the Connector expose H2 server engine console on the port defined by the property:

h2.console.port=21080

By default, even if you do not define the property (non-empty), port will be 21080, so you will be able to access through the following direction pattern:

http://<host-endpoint>:<portH2Console>

SSL Context (not for AS4) application.properties

You can configure secure HTTP connections from the Connector by setting the following property to true:

# SSL context enabled (true|false)
ssl.context.enabled=true

In this case you should properly configure the following properties in order to create an SSL context for HTTP communications:

SSL configuration (optional when ssl.context.enables is false)
ssl.keystore.type= #(JKS|PKCS12)
ssl.keystore.path= #(Path to keystore where signing private key are included)
ssl.keystore.password= #(Private key password)
ssl.truststore.path= #(JKS truststore)
ssl.truststore.password= #(Truststore password)

In the case that you disabled the SSL context property, you should configure the corresponding JVM parameters to specify the truststore, keystore, etc. or the further actions depending of your environment configuration.

Kafka configuration `application.properties`

To send log messages to a Kafka server, configure the following parameters:

# DE4A Kafka settings
de4a.kafka.enabled=true
# Enables the standard logging separately of the Kafka messages. It is necessary for print metrics messages - (default: true)
de4a.kafka.logging.enabled=true
# Enables Kafka connection via HTTP (Only enable HTTP mode if outbound TCP connections are blocked from your internal network)
de4a.kafka.http.enabled=false

# Kafka server address (Eg.: de4a-dev-kafka.egovlab.eu:9092)
de4a.kafka.url=de4a-dev-kafka.egovlab.eu:9092
# Uncomment the following property and remove the above one if HTTP mode is enabled
# de4a.kafka.url=https://de4a-dev-kafka.egovlab.eu

# toop legacy kafka properties (Do not touch)
toop.tracker.enabled = false

IMPORTANT - If your server has no access to external domains, the HTTP Kafka and proxy configuration should be enabled. To enable HTTP Kafka log producer, you only need to set the property to true de4a.kafka.http.enabled=true - Also configure the proper endpoint in order to use HTTP connections.

An important setting is to set the de4a.kafka.topic to something that distinguishes your service from all the others. A good naming convention for your topic would be de4a-<two letter country code>-<pilot/partner acronym>.

It is important to mention the property de4a.kafka.logging.enabled, used to enable the file log printing for each Kafka message sent, that property could be enabled even when the de4a.kafka.enabled=false, just for write the log at the different appenders configured in the log4j2 configuration file.

SMP properties `application.properties`

To establish which SMP server will provide the Connector with metadata services, the following properties must be used:

# SMP Client configuration stuff - Do not touch (default values)
smpclient.truststore.type = JKS
smpclient.truststore.path = truststore/de4a-truststore-test-smp-pw-de4a.jks
smpclient.truststore.password = de4a
..........

# External endpoints
smp.endpoint=
You can define there your SMP endpoint and truststore which will be used to validate the signature of the responses.

Do not modify, all consortium SMPs should be validated with the default truststore.

AS4 – phase4 configuration

# What AS4 implementation to use?
toop.mem.implementation = phase4
..........

# Spring As4 gateway implementation bean(provided: phase4GatewayClient and domibusGatewayClient).Implements eu.toop.as4.client.As4GatewayInterface
as4.gateway.implementation.bean=phase4GatewayClient
..........

# Domibus server endpoint
# domibus.endpoint=

The Connector includes two different gateways acting as interfaces to the AS4 outgoing messages, but on one hand the phase4 gateway uses the internal libraries phase4/TOOP to process the outgoing/incoming messages, meanwhile, the Domibus gateway needs to send the messages to an external platform which will take over the exchange role. In this case, it just needs to configure the following parameters to runs the Connector over the phase4 gateway.

What AS4 implementation to use?

toop.mem.implementation = phase4 ..........

Spring As4 gateway implementation bean(provided: phase4GatewayClient and domibusGatewayClient).Implements eu.toop.as4.client.As4GatewayInterface

as4.gateway.implementation.bean=phase4GatewayClient

Also, all the concerns about phase4/TOOP versions are handled by the Connector itself, which is built using the corresponding versions regarding the compatibility and functionality.

AS4 – Domibus configuration

When configuring the Connector to run with the Domibus gateway it is important to align the party’s identifiers both in the Connector and in the Domibus PMode configuration [8] to allow the proper functioning.

application.properties

..........

# Spring As4 gateway implementation bean (provided: phase4GatewayClient and domibusGatewayClient).Implements eu.toop.as4.client.As4GatewayInterface
as4.gateway.implementation.bean=domibusGatewayClient
………

# Domibus properties
domibus.endpoint=https://<domibus-host>/domibus/services/backend

phase4.properties

# (string) - the from party ID to be used for outgoing messages. Previous versions need to use toop.mem.as4.tc.partyid - starting from RC3 this property is still used as a fallback)
phase4.send.fromparty.id=egdovlab

# (string) to party ID to be used for outgoing messages. Configure it in case of using Domibus without dynamic participant discovery or in the phase4 side in a mixed AS4 implementations phase4<->Domibus
phase4.send.toparty.id=sgad

# (string) - the AS4 To/PartyId/@type value. E.g. urn:oasis:names:tc:ebcore:partyid-type:unregistered
phase4.send.toparty.id.type=urn:oasis:names:tc:ebcore:partyid-type:unregistered

# (string) - the AS4 From/PartyId/@type value. E.g. urn:oasis:names:tc:ebcore:partyid-type:unregistered
phase4.send.fromparty.id.type=urn:oasis:names:tc:ebcore:partyid-type:unregistered

In addition to the common configuration of phase4, the mentioned above is especially important to match with the configuration settled on the Domibus instance.

About the compatibility between the Connector and Domibus, it has been tested versions from 4.1.6 to the current (4.2.2).

Proxy properties

Some environments may require to perform proxy connections due to security policies or environment limitations. That is why the Connector allows to establish HTTP connections via proxy.

# Properties to create the HTTP client connection through a proxy (optional)
#http.proxy.enabled=
#http.proxy.address=
#http.proxy.port=
#http.proxy.non-proxy= ("|" delimiter)
#http.proxyUsername=
#http.proxyPassword=

# Required renamed proxy configuration for BDXRClient (if needed, only uncomment)
#http.proxyHost=${http.proxy.address}
#http.proxyPort=${http.proxy.port}
#http.nonProxyHosts=${http.proxy.non-proxy}

To disable proxy configuration, you can either comment the properties or set up enabled property to false: http.proxy.enabled=false

PLEASE NOTE that in case that you enabled the property (http.proxy.enabled=true) above you should uncomment and set up the rest of them. Also uncomment properties regarding to BDXRClient.

phase4 properties

Parameters used by the built-in Phase4 module of the Connector. Set up the properties above following the commented indications. Some of them are filled in to clarify the content -- Important to consider if each property is optional or not (check out the in-line comments).

# (string) - the absolute path to a local directory to store data
phase4.datapath=

# (boolean) - enable or disable HTTP debugging for AS4 transmissions. The default value is false.
phase4.debug.http=

# (boolean) - enable or disable debug logging for incoming AS4 transmissions. The default value is false.
phase4.debug.incoming=

# (string) - an optional absolute directory path where the incoming AS4 messages should be dumped to. Disabled by default.
phase4.dump.incoming.path=

# (string) - an optional absolute directory path where the outgoing AS4 messages should be dumped to. Disabled by default.
phase4.dump.outgoing.path=

# (string) - the from party ID to be used for outgoing messages. Previous versions need to use toop.mem.as4.tc.partyid - starting from RC3 this property is still used as a fallback)
phase4.send.fromparty.id=

# (string) to party ID to be used for outgoing messages. Configure it in case of using Domibus without dynamic participant discovery or in the phase4 side in a mixed AS4 implementations phase4<->Domibus
phase4.send.toparty.id=

# (string) - the AS4 To/PartyId/@type value. E.g. urn:oasis:names:tc:ebcore:partyid-type:unregistered
phase4.send.toparty.id.type=urn:oasis:names:tc:ebcore:partyid-type:unregistered

# (string) - the AS4 From/PartyId/@type value. E.g. urn:oasis:names:tc:ebcore:partyid-type:unregistered
phase4.send.fromparty.id.type=urn:oasis:names:tc:ebcore:partyid-type:unregistered

# (string) - an optional folder, where sent responses should be stored. If this property is not provided, they are not stored
phase4.send.response.folder=

# (string) - the type of the keystore (either "JKS" or "PKCS12" - case insensitive) - defaults to JKS.
phase4.keystore.type=

# (string) - the path to the keystore (can be classpath relative or an absolute file)
phase4.keystore.path=

# (string) - the password to access the keystore
phase4.keystore.password=

# (string) - the alias of the key in the keystore (may be case sensitive)
phase4.keystore.key-alias=

# (string) - the password to access the key in the keystore
phase4.keystore.key-password=

# (string) - the type of the truststore (either "JKS" or "PKCS12" - case insensitive) - defaults to JKS.
phase4.truststore.type=

# (string) - the path to the truststore (can be classpath relative or an absolute file)
phase4.truststore.path=

# (string) - the password to access the truststore
phase4.truststore.password=

Logging configuration `log4j2.xml`

The configuration file bellow maintains the logging configuration where you can set the level of each appender, set up log file path, or even include more appenders or configuration. Important - to correctly configure the path of log file. By default it is a relative path to catalina.base (Tomcat server): ${sys:catalina.base}/logs/connector.log

<RollingFile name="rollingFile"
    fileName="${sys:catalina.base}/logs/connector.log"
    filePattern="${sys:catalina.base}/logs/history/connector.%d{dd-MMM}.log.gz"
    ignoreExceptions="false">
    <PatternLayout>
        <Pattern>>[%date{ISO8601}][%-5level] %msg -- %location [%thread]%n</Pattern>
    </PatternLayout>
    <Policies>
        <OnStartupTriggeringPolicy />
        <SizeBasedTriggeringPolicy size="30 MB" />
    </Policies>
    <DefaultRolloverStrategy max="5" />
</RollingFile>
<RollingFile name="metricsFile"
    fileName="${sys:catalina.base}/logs/metrics-connector.log"
    filePattern="${sys:catalina.base}/logs/history/metrics-connector.%d{dd-MMM}.log.gz"
    ignoreExceptions="false">
    <PatternLayout>
        <Pattern>>$${ctx:metrics:-}[%date{ISO8601}][%-5level][$${ctx:logcode:-}][DE4A-CONNECTOR][$${ctx:origin:-}][$${ctx:destiny:-}] %msg -- %location [%thread]%n</Pattern>
    </PatternLayout>
    <Policies>
        <OnStartupTriggeringPolicy />
        <SizeBasedTriggeringPolicy size="30 MB" />
    </Policies>
    <DefaultRolloverStrategy max="5" />
</RollingFile>

Two different log files are been generated, rollingFile include more technical and debuggin log lines, whereas metricsFile is composed by metrics messages specificied mostly at list of log messages. Also, in the application.properties there is another property related with the logging.

Logging metrics messages prefix - Default: DE4A METRICS

log.metrics.prefix=DE4A METRICS

It is used to include a prefix on each logging line written by the Kafka logging that could be useful to parse and filter the lines with metrics information.

Starting up the Connector

Once you have all configuration parameters well configured (if not, check the logs to find out the problem), it is time to deploy the component into an applications server. Once you have deployed the war file or the docker image, there are several checks to ensure that the deployment was successful:

Open Swagger UI browsing: http://host:port/swagger-ui/

E.g.: UM Connector

DE4A Connector index page will be at root path: http://host:port/

E.g.: UM Connector

The Connector will be able to process requests through the following interfaces:

/requestTransferEvidenceIM

/requestTransferEvidenceUSI

/lookupRoutingInformation

/requestTransferEvidenceUSIDT

DE4A Connector Installation and configuration guide

Contents

Installation

Package

Connector configuration guide

Database properties

SSL Context (not for AS4) application.properties

Kafka configuration `application.properties`

SMP properties `application.properties`

AS4 – phase4 configuration

AS4 – Domibus configuration

Proxy properties

phase4 properties

Logging configuration `log4j2.xml`

Starting up the Connector

Navigation menu

Search

DE4A Connector Installation and configuration guide

Installation

Package

Connector configuration guide

Database properties

SSL Context (not for AS4) application.properties

Kafka configuration application.properties

SMP properties application.properties

AS4 – phase4 configuration

AS4 – Domibus configuration

Proxy properties

phase4 properties

Logging configuration log4j2.xml

Starting up the Connector

Navigation menu

Search

Kafka configuration `application.properties`

SMP properties `application.properties`

Logging configuration `log4j2.xml`