DE4A Connector it2 - Revision history

Mpttfp eduardo.moreno: /* Communications */

2023-03-27T13:06:40Z

Communications

Mpttfp eduardo.moreno: /* Utilities libraries */

2023-03-27T12:04:39Z

Utilities libraries

Mpttfp eduardo.moreno: /* Third party libraries */

2023-03-27T11:52:26Z

Third party libraries

Mpttfp eduardo.moreno: /* Utilities libraries */

2023-03-27T11:50:39Z

Utilities libraries

Javier.ferrero: Quality review

2023-01-04T17:18:15Z

Quality review

Javier.ferrero: Quality review

2022-12-16T19:26:48Z

Quality review

Javier.ferrero: /* AS4 implementations */

2022-12-16T18:13:46Z

AS4 implementations

Javier.ferrero: Quality review

2022-11-15T09:24:15Z

Quality review

Javier.ferrero: Minor fixes

2022-11-14T16:51:34Z

Minor fixes

Mpttfp eduardo.moreno at 11:03, 10 November 2022

2022-11-10T11:03:39Z

@@ Line 90: / Line 90: @@
 The main elements for such communication securitisations are:
-*'''Private key''': private part of private-public key pair used in the TLS handshake between your server and the connecting clients. This private key must be configured in the Connector properties in order to create the secure context as well in the web server, in the case of reverse proxy or any similar structure. The keys configured on each component are generated under the DE4A management.
+*'''Private key''': private part of private-public key pair is used in the message exchange between your server and the connecting clients. This private key must be configured in the Connector properties in order to create the secure context as well in the web server, in the case of reverse proxy or any similar structure. The keys configured on each component are generated under the DE4A management.
 *'''Truststore''': used for the storage of certificates from the trusted Certificate Authority (CA), which is used in the verification of the certificate provided by the server in an SSL connection. The Connector provides a default truststore which allows to trust on certificates generated under the main DE4A CA.

@@ Line 166: / Line 166: @@
 *'''Commercial libraries'''
-To perform common and non-business operations on web and data exchange projects, the De4a Connector does not use any commercial libraries as it is published as an Open Source project, all the used libraries are embedded in the Java development kit.
+To perform common and non-business operations on web and data exchange projects, the DE4A Connector does not use any commercial libraries as it is published as an Open Source project, all the used libraries are embedded in the Java development kit.
 *'''In-house solutions'''
 The Connector project includes some utilities that allow the data processing and internal tools to perform all the Connector tasks. Those utilities are within the Connector project as a module called ''de4a-commons.''
 [[Category:Wip]]

@@ Line 134: / Line 134: @@
 ===Third party libraries===
-As part of the bunch of libraries used by the Connector there are some of the related with the core features and which represent a starting point for the functionalities provided by the Connector.
+As part of the libraries used by the Connector some are related with the core features and represent a starting point for the functionalities provided by the Connector.
 ==== TOOP Connector====

@@ Line 166: / Line 166: @@
 *'''Commercial libraries'''
-To perform common and non-business operations on web and data exchange projects, the Connector uses different commercial solutions, also all the libraries embedded in the Java development kit.
+To perform common and non-business operations on web and data exchange projects, the De4a Connector does not use any commercial libraries as it is published as an Open Source project, all the used libraries are embedded in the Java development kit.
 *'''In-house solutions'''
 The Connector project includes some utilities that allow the data processing and internal tools to perform all the Connector tasks. Those utilities are within the Connector project as a module called ''de4a-commons.''
 [[Category:Wip]]

@@ Line 11: / Line 11: @@
 === Routing information lookup ===
-The DE4A Connector is responsible for obtaining the Data Provider information from the Central IAL. The Central IAL collects information from all the SMPs deployed at each partner infrastucture and offers it in a centralized way through the REST API <code>/lookupRoutingInformation</code>. When queried about a Canonical Item (Evidence or Event Catalogue) and a country, it returns the identifier of the Data Owner to request that Canonical Item from in that country. Therefore, through this functionality, the DE4A Connector implements the [[DE4A Issuing Authority Locator (IAL)|IAL]] semantic tool.
+The DE4A Connector is responsible for obtaining the Data Provider information from the Central IAL. The Central IAL collects information from all the SMPs deployed at each partner infrastucture and offers it in a centralized way through the REST API <code>/service/ial/</code>. When queried about a Canonical Item (Evidence or Event Catalogue) and a country, it returns the identifier of the Data Owner to request that Canonical Item from in that country. Therefore, through this functionality, the DE4A Connector implements the [[DE4A Issuing Authority Locator (IAL)|IAL]] semantic tool.
 === Dynamic discovery of services ===
@@ Line 72: / Line 72: @@
 This feature is an advantage from a technical and business point of view, as the Connector performs the message exchange transparently to the other components.
-It should be noted that the messages sent to the Kafka server are hardcoded, it is not a parameterizable feature, so any enhancements must be hardcoded again and deployed. It has been developed in such a way because the necessity of a Kafka server is not expected outside the DE4A project. In a real scenario, an alternative way of collecting logs should be implemented.
+The messages sent to the Kafka server are based on the set of [[DE4A Logs and error messages it2|log and error messages]] defined for DE4A.
 ===Data management===

@@ Line 11: / Line 11: @@
 === Routing information lookup ===
-The DE4A Connector is responsible for obtaining the Data Provider information from the [[DE4A Issuing Authority Locator (IAL)|Central IAL]]. The Central IAL collects information from all the SMPs deployed at each partner infrastucture and offers it in a centralized way through the REST API <code>/lookupRoutingInformation</code>. When queried about a Canonical Item (Evidence or Event Catalogue) and a country, it returns the identifier of the Data Owner to request that Canonical Item from in that country. Therefore, through this functionality, the DE4A Connector implements the [[DE4A Issuing Authority Locator (IAL)|IAL]] semantic tool.
+The DE4A Connector is responsible for obtaining the Data Provider information from the Central IAL. The Central IAL collects information from all the SMPs deployed at each partner infrastucture and offers it in a centralized way through the REST API <code>/lookupRoutingInformation</code>. When queried about a Canonical Item (Evidence or Event Catalogue) and a country, it returns the identifier of the Data Owner to request that Canonical Item from in that country. Therefore, through this functionality, the DE4A Connector implements the [[DE4A Issuing Authority Locator (IAL)|IAL]] semantic tool.
 === Dynamic discovery of services ===
@@ Line 53: / Line 53: @@
 The message exchange is built on the [[Usage of third party specifications and components|AS4]] protocol, which is an open standard for secure, payload-agnostic, business-to-business document exchange via web services. Secure document exchange is governed by WS-Security aspects, including XML Encryption and XML Digital Signatures.
-==== phase4 ====
+The DE4A Connector implements phase4 as an AS4 gateway to perform the AS4 message exchange. Its main features are:
-The DE4A Connector implements phase4 as a gateway to perform the AS4 message exchange.
+* It is a generic AS4 solution.
-* '''Main features'''
+* It can be integrated in basically any Servlet-based application server.
-** It is a generic AS4 solution.
+* It can easily be used with Peppol.
-** It can be integrated in basically any Servlet-based application server.
+* It can be used with other Dynamic Discovery solutions.
-** It can easily be used with Peppol.
+* It supports a high degree of customization.
-** It can be used with other Dynamic Discovery solutions.
+* It is not limited to the CEF/e-SENS profile.
 All necessary configuration parameters are provided by the properties file of the DE4A Connector.
@@ Line 126: / Line 123: @@
 ===Data management===
-The Connector stores and manages certain information such as DE endpoints (not to be confused with SMP endpoints), DO endpoints, certain request records for asynchronous response matching, etc. All this data and where to find it is described in the following subsections.
+DE4A Connector can take the DE4A roles of Data Requestor and Data Transferor. This means that a Connector might provide service to multiple DEs and DOs at the same time. Therefore, the DE4A Connector needs to store the base endpoint (without path name) of each DE and DO it is providing service. Each record is identified by the participant ID of the DEs and DOs, so that when a message arrives to the DE4A Connector, whose recipient is identified by the participant ID, the DE4A Connector can forward the message to the corresponding endpoint service. This information is configured in the '''de-do.json''' file. More information on this is available at [[Connector iteration 2 installation and configuration guide]].
 ==Security==
 The message exchange performed by the Connector has been conceived to ensure different security aspects as:

@@ Line 50: / Line 50: @@
 No configuration is needed to differentiate the roles, it only depends on the usage, i.e., the behaviour will be according to the messages sent. Moreover, in a functional scenario, the SMP data will determine which Connector is playing the DR or DT role.
-=== AS4 implementations ===
+=== AS4 implementations supported ===
 The message exchange is built on the [[Usage of third party specifications and components|AS4]] protocol, which is an open standard for secure, payload-agnostic, business-to-business document exchange via web services. Secure document exchange is governed by WS-Security aspects, including XML Encryption and XML Digital Signatures.

@@ Line 11: / Line 11: @@
 === Routing information lookup ===
-The Connector is responsible for obtaining the Data Provider information from the Central IAL. It exposes a REST API <code>/lookupRoutingInformation</code> to get information about the Data Owners that provide a specific Canonical Evidence Type and further related information. That information is used to construct a request message to be sent through the Connector. Thus, the Data Evaluator is the only consumer of the mentioned API method.
+The DE4A Connector is responsible for obtaining the Data Provider information from the [[DE4A Issuing Authority Locator (IAL)|Central IAL]]. The Central IAL collects information from all the SMPs deployed at each partner infrastucture and offers it in a centralized way through the REST API <code>/lookupRoutingInformation</code>. When queried about a Canonical Item (Evidence or Event Catalogue) and a country, it returns the identifier of the Data Owner to request that Canonical Item from in that country. Therefore, through this functionality, the DE4A Connector implements the [[DE4A Issuing Authority Locator (IAL)|IAL]] semantic tool.
-=== Dynamic discovery of Services ===
+=== Dynamic discovery of services ===
-In order for the Connector to be able to send a message to the corresponding endpoint, the [[Usage of third party specifications and components|eDelivery]] dynamic discovery mode of operation is used. This operation mode is based on the use of the [[DE4A Issuing Authority Locator (IAL)|IAL]] and [[Usage of third party specifications and components|SML/DNS]] and SMP components of the eDelivery infrastructure.
+In order for the Connector to be able to send a message to the corresponding endpoint, the [[Usage of third party specifications and components|eDelivery]] dynamic discovery mode of operation is used. This operation mode is based on the use of the components of the eDelivery infrastructure: [[Usage of third party specifications and components|SML/DNS]] and SMPs. Through this functionality, the DE4A Connector implements the [[DE4A Evidence Service Locator (ESL)|ESL]] semantic tool.
 The main elements stored in the SML/DNS and SMPs for this purpose are the following:
-* ''ParticipantIdentifier'': The Data Owner/Data Evaluator identifier who is publishing its [[Usage of third party specifications and components|AS4]] communication point (of the Connector linked to it).
+* ''ParticipantIdentifier'': The Data Owner/Data Evaluator identifier who is publishing its [[Usage of third party specifications and components|AS4]] communication point (or the Connector linked to it).
-* ''DocumentTypeIdentifier'': Canonical evidence type.
+* ''DocumentTypeIdentifier'': Canonical Evidence Type or Canonical Event Catalogue.
-* ''ProcessIdentifier'': Orchestration type (request/response).
+* ''ProcessIdentifier'': Orchestration type: request, response or notification.
 * ''AS4'' endpoint: AS4 service endpoint URL.
 * ''Certificate'': The X.509 certificate of the AS4 server, used to encrypt the transmitted data for this specific participant.
@@ Line 33: / Line 31: @@
 * Response signature validation is mandatory.
 * The communication between SMP and SML requires the usage of a client certificate.
-The service metadata lookup will be performed as a step prior to the [[Usage of third party specifications and components|AS4]] message exchange. Therefore, the participant IDs and other related information must be known by the Connector in advance.
+The service metadata lookup will be performed as a step prior to the [[Usage of third party specifications and components|AS4]] message exchange. Therefore, the participant IDs and other related information must be known by the DE4A Connector in advance.
 === Supported interaction patterns ===
-Interaction patterns define the flow of data through the Connector and the intercommunication between the different components. Each pattern exchanges certain types of messages, and the incoming/outgoing information will depend on the processes occurring in the external components [3].
+Interaction patterns define the flow of data through the Connector and the intercommunication between the different components. Each pattern exchanges certain types of messages, and the incoming/outgoing information will depend on the processes occurring in the external components.
-* '''Subscription and Notification (S&N) pattern'''
+The Connector currently supports four interaction patterns: [[Intermediation Pattern|Intermediation (IM)]], [[User-supported Intermediation Pattern|User-Supported intermediation (USI)]], [[Subscription and Notification Pattern|Subscription and Notification (S&N)]] and [[Lookup Pattern|Lookup (LU)]] patterns. The [[Verifiable Credentials Pattern|Verifiable Credentials (VC)]] pattern is not supported by the DE4A Connector since that interaction pattern has benn implemented through its own components: the [[DE4A SSI Authority Agent|SSI Authority Agent]] and the [[DE4A SSI Edge Agent|SSI Edge Agent]].
 Most of the specific behaviour of each interaction pattern is independent of the Connector itself, as the Connector component is just designed to exchange messages and the main differences between the patterns take place in the external components such as the Data Evaluator and the Data Owner.
-Additionally, the it2 connector offers a '''backwards compatibility feature''' for the IM pattern of it1. It can handle old messaging structure so implementation for it1 will be fully compatible with the it2 connector. In it1 IM pattern worked as a synchronous communication, this backwards compatibility also works with synchronous communication.
+Additionally, the DE4A Connector offers a '''backwards compatibility feature''' for the former synchronous IM pattern. It can handle the old messaging structure, so the synchronous implementation is fully compatible with the current asynchronous version.
 === Connector roles ===
@@ Line 65: / Line 48: @@
 * Data Transferor (DT)
-No configuration is needed to differentiate the roles, it only depends on the usage, i.e., the behaviour will be according to the messages sent.
+No configuration is needed to differentiate the roles, it only depends on the usage, i.e., the behaviour will be according to the messages sent. Moreover, in a functional scenario, the SMP data will determine which Connector is playing the DR or DT role.
 === AS4 implementations ===
-The message exchange is built on the [[Usage of third party specifications and components|AS4]] protocol, which is an open standard for secure, payload-agnostic business-to-business document exchange via web services. Secure document exchange is governed by WS-Security aspects, including XML Encryption and XML Digital Signatures.
+The message exchange is built on the [[Usage of third party specifications and components|AS4]] protocol, which is an open standard for secure, payload-agnostic, business-to-business document exchange via web services. Secure document exchange is governed by WS-Security aspects, including XML Encryption and XML Digital Signatures.
 * '''Main features'''
 ** It is a generic AS4 solution.
@@ Line 82: / Line 63: @@
 ** It is not limited to the CEF/e-SENS profile.
-* '''Implementation'''
+All necessary configuration parameters are provided by the properties file of the DE4A Connector.
 === Error handling ===
 Since the Connector performs multiple communications between different external components and some data and structure validations are performed, the Connector needs to monitor all failure points and be able to identify them in order to build the corresponding messages and warnings to inform each external component. When an error happens, the corresponding component creates an error message with the information about the error to be sent back to the entity that sent the failed message.
 === Kafka messages ===

@@ Line 1: / Line 1: @@
-The DE4A Connector is a technical proxy that allows the final participants (DE or DO) to send requests for evidence or responses to other final participants over an [[Usage of third party specifications and components|eDelivery]] communication environment. In addition, to handle the message exchange process, the Connector is responsible for obtaining the message routing information, by exchanging information with external components such as the IDK, the [[Usage of third party specifications and components|SML/DNS]] or the [[SMP]].
+The DE4A Connector is a technical proxy that allows the final participants (DE or DO) to send requests for evidence or responses to other final participants over an [[Usage of third party specifications and components|eDelivery]] communication environment. In addition, to handle the message exchange process, the Connector is responsible for obtaining the message routing information, by exchanging information with external components such as the Central IAL, the [[Usage of third party specifications and components|SML/DNS]] or the [[SMP]].
 To do so, it provides a common interface to DEs and DOs, making the complexity of the system transparent to the final participants and integration easier.
-The Connector component provides the [[AS4 Gateway]] functionality, so it can assume both the role of Data Requestor and Data Transferor. This first approach makes the Connector a stand-alone web application that can be deployed on any suitable application server.
+The Connector component provides the [[AS4 Gateway]] functionality, so it can assume both the role of Data Requestor and Data Transferor. This approach makes the Connector a stand-alone web application that can be deployed on any suitable application server.
-The security and integrity of messages, as well as the unique identification of the participants involved, are the cornerstones of the Connector component.
+The security and integrity of messages, as well as the unique identification of the participants involved, are the cornerstones of the Connector component.[[File:DE4A network IT2 v2-Target infrastructure it2.drawio.png|alt=|center|frame|Conceptual schema of the target DE4A system for the second iteration.]]
 == Functionalities provided ==
-The main purpose of the DE4A Connector is sending and receiving of evidence requests and their responses. The message exchange process is described in the DE4A deliverables D2.4 Project Start Architecture and D5.3 Initial technical design of interfaces.
+The main purpose of the DE4A Connector is sending and receiving evidence requests and their responses. The message exchange process is described in the DE4A deliverables D2.4 Project Start Architecture and D5.3 Initial technical design of interfaces.
 === Routing information lookup ===
-The Connector is responsible for obtaining the Data Provider information from the IDK. It exposes a REST API <code>/lookupRoutingInformation</code> to get information about the Data Owners that provide a specific Canonical Evidence Type and further related information. That information is used to construct a request message to be sent through the Connector. Thus, the Data Evaluator is the only consumer of the mentioned API method.
+The Connector is responsible for obtaining the Data Provider information from the Central IAL. It exposes a REST API <code>/lookupRoutingInformation</code> to get information about the Data Owners that provide a specific Canonical Evidence Type and further related information. That information is used to construct a request message to be sent through the Connector. Thus, the Data Evaluator is the only consumer of the mentioned API method.
 === Dynamic discovery of Services ===