Second phase

Building Block Assessment - 2nd Phase

The second phase of the assessment of the architecture building blocks (BBs) used in the DE4A project comes after the cataloging of an initial set of relevant BBs for the Project Start Architecture (PSA). While in the previous phase the BBs were assessed for their technical, administrative and operational maturity, in this phase a more constrained set of BBs actually implemented by the pilots was evaluated from a wider perspective.

Methodology

The methodology for evaluation was designed following general systemic principles, with a set of indicators including: usability, openness, maturity, interoperability, etc. The identification of the evaluation criteria and the analysis of the results have been approached from several perspectives: literature survey and thorough desktop research; revision and fine-tuning of the initial BB set and evaluation methodology, and an online questionnaire designed for gathering feedback by the relevant project partners:

-      Member States (MSs)

-      WP4 partners - Pilots (Doing Business Abroad - DBA, Moving Abroad - MA, and Studying Abroad - SA)

-      WP5 partners - for specific components

To revise and fine-tune the evaluation methodology developed in the first phase, as well as the set of BBs selected for assessment as a result, several meetings and consultations were held with the aforementioned project partners. This led to a narrower set of relevant BBs consisting of a subset of the ones assessed in the first phase, and news ones resulting from the piloting needs and implementation. To capture these specificities, a wide set of questions from the survey capture all aspects important for the decisions made and for future reuse. Several iterations over the initial set of questions were performed, determining the relevance according to the roles of the project partners that provided feedback. Finally, a process of feedback coordination was also determined for the pilot leaders, who gathered additional information by the pilot partners’ implementation of some of the BBs (or important aspects of them).

It is important to note that the division of partners according to their role in the project was done due to the difference in the set of BBs that they had experience with. This implies that not each partner evaluated the whole set of BBs, but only the BBs relevant for their practice and research in the DE4A project.

The final set of relevant BBs per partner type is given in Table 1, together with statistics on the number of assessments obtained for each BB.

Table 1 List of building blocks evaluated in the second phase, per relevant partner

#	Building Block	# of assessments on the BB	Relevant partner(s)
	Common Components
1	eDelivery (data exchange)	3	WP5/MSs
2	SMP/SML	2	MSs via pilots
3	DE4A Connector	2	MSs via pilots
4	DE4A Playground	6	DBA/SA/MA
	Semantic
5	Information Exchange Model	7	DBA/SA/MA/WP5
6	Canonical Data Models	6	DBA/SA/MA DE and DO pilot partners
7	ESL (implemented as part of SMP/SML)	1	WP5
8	IAL	1	WP5
9	MOR	1	MA pilot partners
	eID/PoR
10	SEMPER	1	DBA
	VC Pattern
11	SSI Authority agent	4	SA
12	SSI User agent (mobile)	4	SA
13	EBSI-ESSIF (CEF Blockchain)	1	WP5 (T5.4)

Questionnaire design

This questionnaire aims to analyze the extent and the ways of employment of a list of BBs, whichwere cataloged during the first assessment phase of this task. Through a number of assessment categories and indicators assigned to them, the questionnaire supports a methodology that allows us to qualify and quantify the applicability, functionality, maturity and potential for reusability of each of the BBs, especially in view of the DE4A project.

The questionnaire contains 9 sections, i.e. categories, inquiring on:

• Inventory of BBs and functionalities;
• Interoperability;
• Maturity;
• Openness;
• Ease of implementation;
• Meeting pilot requirements;
• Performance (Non-functional requirements);
• Patterns; and
• Trust, Identity, Security, Privacy, Protection.

The questionnaire was distributed among the relevant (aforementioned) partners, and the feedback has been obtained in the period 28.11.2022 - 12.12.2022. Based on the insights from the provided feedback, we are able to also extract valuable lessons learned through the implementation practices, as well as recommendations for future reuse of the analyzed BBs.

Note: The BB assessment includes the BBs used in both iterations of the pilots. It also includes BBs used in the playground and the mocked DE/DO.

Data and preprocessing

After the data gathering period, a total of 10 surveys were received with valid input, representing full coverage of the BBs by foreseen parties (See Table 2). This provides sufficient statistical significance to proceed with the analysis of the results and finalize the BB assessment.

Table 2.Distribution of received valid feedback per partner type

Partner type	# of assessments done
1 (WP5 representative)	2
2 (DBA pilot representative)	1
3 (SA pilot representative)	4
4 (MA pilot representative)	1
5 (MS representative)	2
Valid	10

Data analysis

In this section, we present the analysis of the obtained data and put the results in both the DE4A and the wider context relevant for reusing the analyzed BBs. We then provide comparative analysis between the two evaluation phases of the BBs carried out in the DE4A project.