DE4A Playground - Revision history

Antonio.osuna at 15:36, 22 February 2022

2022-02-22T15:36:38Z

Antonio.osuna: /* Mocked IDK */

2022-02-22T12:43:47Z

Mocked IDK

Antonio.osuna at 12:34, 22 February 2022

2022-02-22T12:34:37Z

Antonio.osuna at 13:54, 7 February 2022

2022-02-07T13:54:23Z

Show changes

Antonio.osuna at 12:33, 7 February 2022

2022-02-07T12:33:37Z

Antonio.osuna: Fix links

2022-02-07T09:00:12Z

Fix links

Antonio.osuna at 07:30, 7 February 2022

2022-02-07T07:30:59Z

Antonio.osuna at 07:30, 7 February 2022

2022-02-07T07:30:44Z

Javier.ferrero: Quality review

2022-02-04T17:27:00Z

Quality review

Javier.ferrero: Quality review

2022-02-04T17:15:40Z

Quality review

@@ Line 16: / Line 16: @@
 * Mocked IDK
 * Kafka Server
-The DO and the DT are deployed at SGAD (ES). All other components, including the Kafka Server, are deployed at SGAD subcontractor facilities (ES).
+The DO and the DT are deployed at SGAD (ES). All other components, including the [[Usage of third party specifications and components|Kafka]] Server, are deployed at SGAD subcontractor facilities (ES).
 == Electronic Certificates ==
@@ Line 116: / Line 116: @@
 ==== Data Evaluator ====
-The mocked DE is deployed together with the Playground DR Connector at Maribor University at https://de4a-demoui.informatika.uni-mb.si/. It is deployed behind an Nginx reverse proxy. The mock DE is configured to send messages to the Kafka server to make it easy to follow the processes when testing.
+The mocked DE is deployed together with the Playground DR Connector at Maribor University at https://de4a-demoui.informatika.uni-mb.si/. It is deployed behind an Nginx reverse proxy. The mock DE is configured to send messages to the [[Usage of third party specifications and components|Kafka]] server to make it easy to follow the processes when testing.
 ===== Point to TLS configuration =====
@@ Line 349: / Line 349: @@
 == Kafka server ==
-The Kafka instance running on the SGAD backend is an open source distributed event streaming platform. In the DE4A project it is used to receive messages from the different services used in the project. The received messages can be viewed online as they are received in the [https://de4a.simplegob.com/package-tracker/#!detail Package Tracker] made available for everyone. This is mainly for the purpose of quickly seeing if there are any problems between services and what type of problem i.e. connection problems to help finding issues for pilot partners and developers.
+The [[Usage of third party specifications and components|Kafka]] instance running on the SGAD backend is an open source distributed event streaming platform. In the DE4A project it is used to receive messages from the different services used in the project. The received messages can be viewed online as they are received in the [https://de4a.simplegob.com/package-tracker/#!detail Package Tracker] made available for everyone. This is mainly for the purpose of quickly seeing if there are any problems between services and what type of problem i.e. connection problems to help finding issues for pilot partners and developers.
 The main use case for this is thus distributed event logging and not digital traffic reporting where entire logs are sent and stored off site. The reasoning for this is that the Kafka instance at SGAD servers is fully open, meaning anyone can read and write messages to that service. To reduce the security risks the Kafka instance on SGAD premises does not store the sent messages for any longer periods, i.e. the messages are essentially ephemeral.

← Older revision		Revision as of 12:43, 22 February 2022
Line 208:		Line 208:

	== Mocked IDK ==		== Mocked IDK ==
		+	Testing component used to simulate the Information Desk behaviour and provide APIs to the Playground based on the [https://wiki.de4a.eu/index.php/Information_Desk IDK definition by WP3]. This mocked component comes from the necessity of the a IDK funcionality and the lack of developments around this centraliced component. Therefore WP5 mocked the funcionality to be used in the playground of the iteration 1 and 2.
		+
	To have a more realistic scenario around the Connector, a mocked IDK component has been developed to simulate the data exchanged with the component. It is just a web application that stores information about the data providers (DOs) of each evidence and country, modelled on tables. It also provides the corresponding interfaces to consume that information by an external component, following the current functional and technical definition of the IDK.		To have a more realistic scenario around the Connector, a mocked IDK component has been developed to simulate the data exchanged with the component. It is just a web application that stores information about the data providers (DOs) of each evidence and country, modelled on tables. It also provides the corresponding interfaces to consume that information by an external component, following the current functional and technical definition of the IDK.

← Older revision		Revision as of 12:34, 22 February 2022
Line 358:		Line 358:
	=== Usage of a REST API for receiving messages from a Connector through a proxy ===		=== Usage of a REST API for receiving messages from a Connector through a proxy ===
	By default, the Kafka instance at SGAD servers only communicates via the TCP protocol which created problems for SGAD. Their service runs behind a proxy server which only allows outbound HTTP(S) traffic. The resolution to this problem was to set up an additional REST interface to Kafka on SAGD backend, so that the messages can also be transmitted via the HTTP protocol.		By default, the Kafka instance at SGAD servers only communicates via the TCP protocol which created problems for SGAD. Their service runs behind a proxy server which only allows outbound HTTP(S) traffic. The resolution to this problem was to set up an additional REST interface to Kafka on SAGD backend, so that the messages can also be transmitted via the HTTP protocol.
		+	[[Category:Wip]]

@@ Line 1: / Line 1: @@
-== Introduction ==
+The Playground is a set of predefined and pre-deployed components to be used as is. The “DE4A Playground” was designed to help pilot partners to onboard the DE4A network. It is essentially a temporary infrastructure with two purposes:
 The target audience for this chapter are primarily software developers that try to connect their DE/DR/DT/DO system to DE4A. Bellow, a quick introduction to the main structure and list of components involved:
-[[File:Playground-schema.png|alt=DE4A Playground Big Picture|none|thumb|528x528px|DE4A Playground Big Picture]]
+* It simulates a real scenario where the User can request data (evidence) about a subject (citizen or company). Thus, it allows testing the performance of the DE4A Connector by using mocked components and fake data.
 The Playground consists of the following components:

@@ Line 24: / Line 24: @@
 === Data Evaluator ===
-The Data Evaluator deployed in the Playground was developed as a separate piece of software. The source code can be found at https://github.com/de4a-wp5/wp5-demo-ui and the deployed instance of this software is available at https://de4a-dev-mock.egovlab.eu/public.
+The Data Evaluator deployed in the Playground was developed as a separate piece of software. The source code can be found at https://github.com/de4a-wp5/wp5-demo-ui and the deployed instance of this software is available at https://de4a-demoui.informatika.uni-mb.si/.
 All the IEM handling functionality is taken from the shared “de4a-commons” library available at https://github.com/de4a-wp5/de4a-commons.
@@ Line 223: / Line 223: @@
 === Deployment and configuration ===
 Regarding the deployment of the mocked IDK, it is not necessary to consider any extra configuration. It is just a standalone web application that can be deployed on any application server once the war file is generated. [6]
-The mocked IDK deployed instance is at the eGovlab servers, and it can be reached by accessing the URL: https://de4a-dev-idk.egovlab.eu/
+The mocked IDK deployed instance is at the eGovlab servers, and it can be reached by accessing the URL: https://de4a-idk.informatika.uni-mb.si/
 == eDelivery ==
@@ Line 343: / Line 343: @@
 == Kafka server ==
-The Kafka instance running on the eGovlab backend is an open source distributed event streaming platform. In the DE4A project it is used to receive messages from the different services used in the project. The received messages can be viewed online as they are received in the [https://de4a-dev-pt.egovlab.eu/#!detail Package Tracker] made available for everyone. This is mainly for the purpose of quickly seeing if there are any problems between services and what type of problem i.e. connection problems to help finding issues for pilot partners and developers.
+The Kafka instance running on the eGovlab backend is an open source distributed event streaming platform. In the DE4A project it is used to receive messages from the different services used in the project. The received messages can be viewed online as they are received in the [https://de4a.simplegob.com/package-tracker/#!detail Package Tracker] made available for everyone. This is mainly for the purpose of quickly seeing if there are any problems between services and what type of problem i.e. connection problems to help finding issues for pilot partners and developers.
 The main use case for this is thus distributed event logging and not digital traffic reporting where entire logs are sent and stored off site. The reasoning for this is that the Kafka instance at eGovlab servers is fully open, meaning anyone can read and write messages to that service. To reduce the security risks the Kafka instance on eGovlab premises does not store the sent messages for any longer periods, i.e. the messages are essentially ephemeral.

@@ Line 1: / Line 1: @@
 == Introduction ==
-As the necessary infrastructure setup is quite complex, WP5 designed the “DE4A Playground” which helps pilot partners to onboard the DE4A network. The Playground is a set of predefined and pre-deployed components to be used as is. It is essentially a temporary infrastructure with two purposes:
+As the necessary infrastructure setup is quite complex, the “DE4A Playground” was designed to help pilot partners to onboard the DE4A network. The Playground is a set of predefined and pre-deployed components to be used as is. It is essentially a temporary infrastructure with two purposes:
 * It simulates a real scenario where the User can request data (evidence) about a subject (citizen or company). Thus, it allows testing the performance of the DE4A Connector by using mocked components and fake data.
-* It allows DE4A pilot participants to deploy their own infrastructure in an orderly manner. The internal document “Playground joining stages and information required from pilot partners” sets out a three-stage process for deploying their software components: first their own developed entities (DE/DO), secondly the DE4A Connector provided by WP5, and finally the SMP of the eDelivery infrastructure. At the end of each stage, the pilot partner can test their deployment by making used of the rest of the Playground components.
+* It allows DE4A pilot participants to deploy their own infrastructure in an orderly manner. A [[DE4A Playground joining stages for the first iteration|three-stage process]] was set out for the pilot participants to deploy their software components: first their own developed entities (DE/DO), secondly the DE4A Connector, and finally the SMP of the eDelivery infrastructure. At the end of each stage, the pilot partner can test their deployment by making used of the rest of the Playground components.
 The target audience for this chapter are primarily software developers that try to connect their DE/DR/DT/DO system to DE4A.
 [[File:Playground-schema.png|alt=DE4A Playground Big Picture|none|thumb|528x528px|DE4A Playground Big Picture]]
@@ Line 29: / Line 29: @@
 === Data Evaluator ===
-The Data Evaluator deployed in the Playground was developed as a separate piece of software by WP5. The source code can be found at https://github.com/de4a-wp5/wp5-demo-ui and the deployed instance of this software is available at https://de4a-dev-mock.egovlab.eu/public.
+The Data Evaluator deployed in the Playground was developed as a separate piece of software. The source code can be found at https://github.com/de4a-wp5/wp5-demo-ui and the deployed instance of this software is available at https://de4a-dev-mock.egovlab.eu/public.
 All the IEM handling functionality is taken from the shared “de4a-commons” library available at https://github.com/de4a-wp5/de4a-commons.
@@ Line 35: / Line 35: @@
 '''Note''': the functionality of this tool may be extended during the further development of the project. The primary goal is to support Member States in onboarding as good as possible.
-'''Note''': the terminology used in the demo application is very simplified and may not suffice the needs of real-world applications. Real pilot applications should align the texts with their local needs – recommendations may be provided by WP7.
+'''Note''': the terminology used in the demo application is very simplified and may not suffice the needs of real-world applications. Real pilot applications should align the texts with their local needs.
 ==== System requirements ====
@@ Line 166: / Line 166: @@
 === Deployment and configuration ===
-The DR connector deployed in the Playground is done so with mainly default settings and uses the built in Phase4 AS4 gateway. The only local configurations needed are the certificates used by the AS4 gateway, the URL of the local DE/DO and the Kafka logging specifics. The routing information is fetched from the IDK and SMP. The AS4 certificates for the Playground are all signed by the DE4A self-signed test CA and can be obtained from WP5.
+The DR connector deployed in the Playground is done so with mainly default settings and uses the built in Phase4 AS4 gateway. The only local configurations needed are the certificates used by the AS4 gateway, the URL of the local DE/DO and the Kafka logging specifics. The routing information is fetched from the IDK and SMP. The AS4 certificates for the Playground are all signed by the DE4A self-signed test CA.
 ==== Point to TLS configuration ====
@@ Line 178: / Line 178: @@
 === Deployment and configuration ===
-The DT connector deployed in the Playground is done with mainly default settings. The only local configurations needed are the certificates used by the AS4 gateway, the URL of the local DE/DO and the Kafka logging specifics. The routing information is fetched from the IDK and SMP. The AS4 certificates for the Playground are all signed by the DE4A self-signed test CA and can be obtained from WP5.
+The DT connector deployed in the Playground is done with mainly default settings. The only local configurations needed are the certificates used by the AS4 gateway, the URL of the local DE/DO and the Kafka logging specifics. The routing information is fetched from the IDK and SMP. The AS4 certificates for the Playground are all signed by the DE4A self-signed test CA.
 ==== Point to HTTP Proxy ====
@@ Line 209: / Line 209: @@
 == Mocked IDK ==
-To have a more realistic scenario around the Connector, a mocked IDK component has been developed by WP5, to simulate the data exchanged with the component. It is just a web application that stores information about the data providers (DOs) of each evidence and country, modelled on tables. It also provides the corresponding interfaces to consume that information by an external component, following the current functional and technical definition of the IDK.
+To have a more realistic scenario around the Connector, a mocked IDK component has been developed to simulate the data exchanged with the component. It is just a web application that stores information about the data providers (DOs) of each evidence and country, modelled on tables. It also provides the corresponding interfaces to consume that information by an external component, following the current functional and technical definition of the IDK.
 === Simulated functionalities ===
@@ Line 228: / Line 228: @@
 === Deployment and configuration ===
 Regarding the deployment of the mocked IDK, it is not necessary to consider any extra configuration. It is just a standalone web application that can be deployed on any application server once the war file is generated. [6]
-The mocked IDK deployed instance is at the eGovlab servers, and it can be reached by accessing the URL:  https://de4a-dev-idk.egovlab.eu/
+The mocked IDK deployed instance is at the eGovlab servers, and it can be reached by accessing the URL: https://de4a-dev-idk.egovlab.eu/
 == eDelivery ==

@@ Line 230: / Line 230: @@
 The mocked IDK deployed instance is at the eGovlab servers, and it can be reached by accessing the URL:  https://de4a-dev-idk.egovlab.eu/
-=== eDelivery ===
+== eDelivery ==
 === SML ===
@@ Line 244: / Line 244: @@
 The task of a Service Metadata Publisher (SMP) is to create the link between the technical identifiers and the effective endpoint URL to exchange messages with. Additionally, it provides X.509 certificates that will be used to encrypt messages for a specific recipient. Each participant has a list of so called “Endpoints” that are uniquely identified by a combination of a document type identifier, a process identifier and a transport profile. An SMP makes it possible for two parties in the domain to exchange documents with dynamic discovery and utilizing the 4 Corner Model used for eDelivery within the DE4A project.
-The DE4A Playground contains a single SMP that is deployed at the URL <nowiki>https://de4a-smp.egovlab.eu</nowiki>, operated by eGovlab. It is an instance of the Open Source SMP solution “phoss SMP”[1] that was configured according to the project needs. At the time of writing of this document, check the participant identifiers registered at [https://de4a-smp.egovlab.eu/ eGovLab SMP instance]
+The DE4A Playground contains a single SMP that is deployed at the URL <nowiki>https://de4a-smp.egovlab.eu</nowiki>, operated by eGovlab. It is an instance of the Open Source SMP solution [https://github.com/phax/phoss-smp+for+further+details phoss SMP] that was configured according to the project needs. At the time of writing of this document, check the participant identifiers registered at [https://de4a-smp.egovlab.eu/ eGovLab SMP instance]
 Each of the participants has registered for different endpoints, and this document will not duplicate the information easily retrievable from the SMP.
@@ Line 346: / Line 346: @@
 ** Dynamic discovery mode
 ** DE4A SML configured
 == Kafka server ==
-The Kafka instance running on the eGovlab backend is an open source distributed event streaming platform. In the DE4A project it is used to receive messages from the different services used in the project. The received messages can be viewed online as they are received in the Package Tracker[1] made available for everyone. This is mainly for the purpose of quickly seeing if there are any problems between services and what type of problem i.e. connection problems to help finding issues for pilot partners and developers.
+The Kafka instance running on the eGovlab backend is an open source distributed event streaming platform. In the DE4A project it is used to receive messages from the different services used in the project. The received messages can be viewed online as they are received in the [https://de4a-dev-pt.egovlab.eu/#!detail Package Tracker] made available for everyone. This is mainly for the purpose of quickly seeing if there are any problems between services and what type of problem i.e. connection problems to help finding issues for pilot partners and developers.
 The main use case for this is thus distributed event logging and not digital traffic reporting where entire logs are sent and stored off site. The reasoning for this is that the Kafka instance at eGovlab servers is fully open, meaning anyone can read and write messages to that service. To reduce the security risks the Kafka instance on eGovlab premises does not store the sent messages for any longer periods, i.e. the messages are essentially ephemeral.
@@ Line 361: / Line 359: @@
 === Usage of a REST API for receiving messages from a Connector through a proxy ===
 By default, the Kafka instance at eGovlab servers only communicates via the TCP protocol which created problems for SGAD. Their service runs behind a proxy server which only allows outbound HTTP(S) traffic. The resolution to this problem was to set up an additional REST interface to Kafka on eGovlab backend, so that the messages can also be transmitted via the HTTP protocol.