Difference between revisions of "Usage of third party specifications and components"
| (22 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
| − | + | The DE4A project is based on some specifications and components that allow building new software with some inherited capabilities such as security, reliability, integrity and scalability. Some of the main technologies involved are described here. | |
== eDelivery == | == eDelivery == | ||
| − | The CEF [https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/How+does+eDelivery+work eDelivery] Building Block helps users to exchange electronic data and documents with | + | The CEF [https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/How+does+eDelivery+work eDelivery] Building Block helps users to exchange electronic data and documents with each other in a reliable and trusted way. |
| − | The CEF eDelivery solution is based on a distributed model called the “4-corner model”. In this model, | + | The CEF eDelivery solution is based on a distributed model called the “4-corner model”. In this model, users' back-end systems do not exchange data directly with each other, but via [https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/Access+Point+software Access Points]. These Access Points comply with to the same [https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/Access+Point+specifications technical specifications] and are therefore able to communicate with each other. |
| − | + | In this way, users adopting the CEF eDelivery can exchange data easily and securely, even if their IT systems were developed independently. | |
===SML=== | ===SML=== | ||
| − | The Service Metadata Locator (SML) is a singleton instance in the DE4A network. It is operated by CEF, a key facility of the European Commission, and provided to the DE4A project | + | The Service Metadata Locator (SML) is a singleton instance in the DE4A network. It is operated by CEF, a key facility of the European Commission, and is provided to the DE4A project free of charge. DE4A is currently operating on the test instance of the SML, called "SMK", and was assigned the DNS zone of4a.acc.edelivery.tech.ec.europa.eu. for the project. |
| − | + | In addition, CEF provided the project with 10 SMP X.509 test certificates based on the "DE4A_TEST_SMP_CA", which is based on CEF's "Connectivity Test Component CA". Only certificates issued by the "DE4A_TEST_SMP_CA" can be registered in the DNS zone of the DE4A SML. | |
| − | + | Each SMP (see below) that wants to join the DE4A network needs to register once in the SML using the specific DNS zone and a certificate based on the "DE4A_TEST_SMP_CA". | |
| + | In the case of DE4A project, the Connector handles the interaction with the ''"service metadata resources"'', there are two function modes depending of the configuration in the Connector: | ||
| + | |||
| + | '''Establishing a certain SMP''' | ||
| + | *The Connector will request all the services information to the SMP configured. | ||
| + | '''Without SMP definition''' | ||
| + | *The Connector will request to the SML in order to get the SMP location which contains services information for the certain participant given. | ||
===SMP=== | ===SMP=== | ||
| − | The task of a Service Metadata Publisher (SMP) is to create the link between the technical identifiers and the | + | The task of a Service Metadata Publisher (SMP) is to create the link between the technical identifiers and the URL of the actual endpoint with which to exchange messages. In addition, it provides the X.509 certificates that will be used to encrypt the messages for a specific recipient. Each participant has a list of so-called "Endpoints" that are uniquely identified by a combination of a document type identifier, a process identifier and a transport profile. An SMP makes it possible for two domain parties to exchange documents with dynamic discovery and utilizing the 4-Corner Model used in eDelivery within the DE4A project. |
| − | |||
| − | |||
| + | The information has been structured as follows, based on the SMP schemas outlined: | ||
{| class="wikitable" | {| class="wikitable" | ||
| − | + | ! SMP filed name | |
| + | ! DE4A parameter | ||
| + | ! Description | ||
| + | ! Example | ||
|- | |- | ||
| − | |iso6523-actorid-upis | + | |ParticipantIdentifier |
| + | |Participant ID | ||
| + | |Unique identifier assigned to each entity involved (DE's/DO's) | ||
| + | |Scheme: iso6523-actorid-upis - Value: 9999:egov | ||
|- | |- | ||
| − | | | + | |DocumentIdentifier |
| + | |Canonical Evidence Type | ||
| + | |Type of canonical evidence | ||
| + | |Scheme: <nowiki>urn:de4a-eu:CanonicalEvidenceType</nowiki> - Value: CompanyRegistration | ||
|- | |- | ||
| − | | | + | |ProcessIdentifier |
| + | |Message Type | ||
| + | |Action type (request/response) for a message | ||
| + | |Scheme: <nowiki>urn:de4a-eu:MessageType</nowiki> - Value: request | ||
|- | |- | ||
| − | | | + | |EndpointURI |
| + | |AS4 endpoint | ||
| + | |Endpoint where the AS4 message will be sent for a certain participant | ||
| + | |https://de4a-connector.informatika.uni-mb.si/ | ||
|- | |- | ||
| − | | | + | |Certificate |
| − | | | + | |AS4 public key certificate |
| − | | | + | |Base64 X.509 certificate for the AS4 gateway |
| − | | | + | |MIIFfzCCA2egAwIBAgICEAkwDQYJKoZIhvcNAQELBQAwajELMAk...(more) |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
|} | |} | ||
| + | The requests to the SMP/SML components are performed using the [https://github.com/phax/peppol-commons peppol-commons] ''(by Phax'') libraries, based on the CEF Specifications: | ||
| + | *[https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/eDelivery+BDXL eDelivery BDXL] | ||
| + | *[https://oasis-open.org/committees/tc_home.php?wg_abbrev=bdxr OASIS BDXR] | ||
| + | ===AS4 Gateway=== | ||
| + | The message exchange is built on the AS4 protocol, which is an open standard for secure, payload-agnostic business-to-business document exchange via web services. Secure document exchange is governed by WS-Security aspects, including XML Encryption and XML Digital Signatures. There are many implementations of AS4. The Connector includes an implementation of an AS4 gateway ('''phase4''') using the following third-party libraries and components: | ||
| + | *[https://github.com/TOOP4EU/toop-connector-ng Toop Connector NG] | ||
| + | *[https://github.com/phax/phase4 phax/phase4 libraries] | ||
| + | Based on the specifications: | ||
| + | *[https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/eDelivery+AS4+-+1.14 eDelivery AS4] | ||
| + | *[https://docs.peppol.eu/edelivery/as4/specification/ PEPPOL AS4 Profile] | ||
| + | ==RegRep== | ||
| + | ebXML RegRep is a standard defining the service interfaces, protocols and information model for an integrated registry and repository. The repository stores digital content while the registry stores metadata that describes the content in the repository. | ||
| + | An ebXML Registry is capable of storing any type of electronic content such as XML documents, text documents, images, sound and video. Instances of such content are referred to as a RepositorytItems. RepositorytItems are stored in a content repository provided by the ebXML Registry. | ||
| − | + | RegRep usage at DE4A project has been limited to a minimal approach due to the low benefits compared to a "plain" message. Nevertheless, the eDelivery implementation at the Connector includes the message wrapping into a RegRep specification with basic slots organization and the message obtained is sent via AS4 as a payload. | |
==Kafka== | ==Kafka== | ||
| + | The [https://www.confluent.io/what-is-apache-kafka/?utm_medium=sem&utm_source=google&utm_campaign=ch.sem_br.nonbrand_tp.prs_tgt.kafka_mt.xct_rgn.emea_lng.eng_dv.all_con.kafka-general&utm_term=apache%20kafka&creative=&device=c&placement=&gclid=CjwKCAjw95yJBhAgEiwAmRrutDb2Nv4YGzhiVGoi0LGOmtnAC1kWCAgrggycC6EB-brGTfPJt1PughoCHT0QAvD_BwE Apache Kafka] instance used in DE4A is an open source distributed event streaming platform. It is used to receive messages from the different services used in the project. The received messages can be viewed online as they are received in the Package Tracker made available for everyone. The main purpose is to be able to quickly see if there are any problems between services and what type of problem (e.g. connection problems) to help pilot partners and developers find those issues. | ||
| + | Therefore, the main use case is distributed event logging and not digital traffic reporting where full logs are sent and stored off-site. The reasoning for this is that the Kafka instance on the eGovlab servers is fully open, meaning that anyone can read and write messages on that service. To reduce security risks, the Kafka instance on the eGovlab premises does not store messages sent for extended periods of time, i.e. messages are essentially ephemeral. | ||
| − | + | The particular configuration used for the DE4A project uses ZooKeeper as the coordinator, Kafka as the message broker and the Package Tracker as the web service configured to consume and display any messages received by the Kafka broker. The Package Tracker can only display "live" messages, i.e. any messages received in the past will never be displayed. | |
| − | + | [[Category:Wip]] | |
Latest revision as of 13:55, 22 February 2022
The DE4A project is based on some specifications and components that allow building new software with some inherited capabilities such as security, reliability, integrity and scalability. Some of the main technologies involved are described here.
eDelivery
The CEF eDelivery Building Block helps users to exchange electronic data and documents with each other in a reliable and trusted way.
The CEF eDelivery solution is based on a distributed model called the “4-corner model”. In this model, users' back-end systems do not exchange data directly with each other, but via Access Points. These Access Points comply with to the same technical specifications and are therefore able to communicate with each other.
In this way, users adopting the CEF eDelivery can exchange data easily and securely, even if their IT systems were developed independently.
SML
The Service Metadata Locator (SML) is a singleton instance in the DE4A network. It is operated by CEF, a key facility of the European Commission, and is provided to the DE4A project free of charge. DE4A is currently operating on the test instance of the SML, called "SMK", and was assigned the DNS zone of4a.acc.edelivery.tech.ec.europa.eu. for the project.
In addition, CEF provided the project with 10 SMP X.509 test certificates based on the "DE4A_TEST_SMP_CA", which is based on CEF's "Connectivity Test Component CA". Only certificates issued by the "DE4A_TEST_SMP_CA" can be registered in the DNS zone of the DE4A SML.
Each SMP (see below) that wants to join the DE4A network needs to register once in the SML using the specific DNS zone and a certificate based on the "DE4A_TEST_SMP_CA".
In the case of DE4A project, the Connector handles the interaction with the "service metadata resources", there are two function modes depending of the configuration in the Connector:
Establishing a certain SMP
- The Connector will request all the services information to the SMP configured.
Without SMP definition
- The Connector will request to the SML in order to get the SMP location which contains services information for the certain participant given.
SMP
The task of a Service Metadata Publisher (SMP) is to create the link between the technical identifiers and the URL of the actual endpoint with which to exchange messages. In addition, it provides the X.509 certificates that will be used to encrypt the messages for a specific recipient. Each participant has a list of so-called "Endpoints" that are uniquely identified by a combination of a document type identifier, a process identifier and a transport profile. An SMP makes it possible for two domain parties to exchange documents with dynamic discovery and utilizing the 4-Corner Model used in eDelivery within the DE4A project.
The information has been structured as follows, based on the SMP schemas outlined:
| SMP filed name | DE4A parameter | Description | Example |
|---|---|---|---|
| ParticipantIdentifier | Participant ID | Unique identifier assigned to each entity involved (DE's/DO's) | Scheme: iso6523-actorid-upis - Value: 9999:egov |
| DocumentIdentifier | Canonical Evidence Type | Type of canonical evidence | Scheme: urn:de4a-eu:CanonicalEvidenceType - Value: CompanyRegistration |
| ProcessIdentifier | Message Type | Action type (request/response) for a message | Scheme: urn:de4a-eu:MessageType - Value: request |
| EndpointURI | AS4 endpoint | Endpoint where the AS4 message will be sent for a certain participant | https://de4a-connector.informatika.uni-mb.si/ |
| Certificate | AS4 public key certificate | Base64 X.509 certificate for the AS4 gateway | MIIFfzCCA2egAwIBAgICEAkwDQYJKoZIhvcNAQELBQAwajELMAk...(more) |
The requests to the SMP/SML components are performed using the peppol-commons (by Phax) libraries, based on the CEF Specifications:
AS4 Gateway
The message exchange is built on the AS4 protocol, which is an open standard for secure, payload-agnostic business-to-business document exchange via web services. Secure document exchange is governed by WS-Security aspects, including XML Encryption and XML Digital Signatures. There are many implementations of AS4. The Connector includes an implementation of an AS4 gateway (phase4) using the following third-party libraries and components:
Based on the specifications:
RegRep
ebXML RegRep is a standard defining the service interfaces, protocols and information model for an integrated registry and repository. The repository stores digital content while the registry stores metadata that describes the content in the repository. An ebXML Registry is capable of storing any type of electronic content such as XML documents, text documents, images, sound and video. Instances of such content are referred to as a RepositorytItems. RepositorytItems are stored in a content repository provided by the ebXML Registry.
RegRep usage at DE4A project has been limited to a minimal approach due to the low benefits compared to a "plain" message. Nevertheless, the eDelivery implementation at the Connector includes the message wrapping into a RegRep specification with basic slots organization and the message obtained is sent via AS4 as a payload.
Kafka
The Apache Kafka instance used in DE4A is an open source distributed event streaming platform. It is used to receive messages from the different services used in the project. The received messages can be viewed online as they are received in the Package Tracker made available for everyone. The main purpose is to be able to quickly see if there are any problems between services and what type of problem (e.g. connection problems) to help pilot partners and developers find those issues. Therefore, the main use case is distributed event logging and not digital traffic reporting where full logs are sent and stored off-site. The reasoning for this is that the Kafka instance on the eGovlab servers is fully open, meaning that anyone can read and write messages on that service. To reduce security risks, the Kafka instance on the eGovlab premises does not store messages sent for extended periods of time, i.e. messages are essentially ephemeral.
The particular configuration used for the DE4A project uses ZooKeeper as the coordinator, Kafka as the message broker and the Package Tracker as the web service configured to consume and display any messages received by the Kafka broker. The Package Tracker can only display "live" messages, i.e. any messages received in the past will never be displayed.