Difference between revisions of "Goals and success criteria"

From DE4A
Jump to navigation Jump to search
 
(44 intermediate revisions by 3 users not shown)
Line 5: Line 5:
 
[[Current status of pilot|Back to Previous Chapter: 2. Current Status of Pilot]]
 
[[Current status of pilot|Back to Previous Chapter: 2. Current Status of Pilot]]
  
−
== 3.1 Goals and pilot success criteria ==
+
[Work in progress]
−
''Objectives and how they are satisfied in relation to success criteria. Use D4.6 section 2.1 (Final version of success criteria and Common Pilot Criteria) as a basis.''
 
  
−
<span style="color: blue"> GOALS (BLUE TEXT IS COPIED FROM D4.6) </span>
+
== 3.1 Introduction ==
 +
The analyses conducted with regards to challenges that the implementation of the SDGR introduces, as wel as the realization of the infrastructure implementing the SDGR and conclusions of these analyses, proved very fruitful. This chapter summarizes the lessons learned form these activities, and provides suggestions for the implementation and adoption of the SDGR implementation.
 +
 
 +
==3.2 Learning towards Adoption==
 +
 
 +
==== ''3.2.1 Lessons learned from analysing and designing national integration of cross-border OOP'' ====
 
{| class="wikitable" style="width: 80%;"
 
{| class="wikitable" style="width: 80%;"
−
!<span style="color:blue">Actor</span>
+
! style="width: 2%" |ID
−
!<span style="color: blue">ID</span>
+
! style="width: 10%" | Topic
−
!<span style="color: blue">Goal</span>
+
! style="width: 33%" |Suggestions for adoption
 +
! style="width: 55%" | Lessons learned
 
|-
 
|-
−
!<span style="color: blue">Public authorities</span>
+
|1
−
|<span style="color: blue">A</span>
+
|Design process
−
|<span style="color: blue">Improve  the quality of Company data within the service fulfilment process by re-using  data from authentic sources, thereby reducing manual work and lowering  processing costs.</span>
+
|DBA advises Member States to invest time to bring together the eIDAS and OOTS knowledge. This requires organising and prioritising as this knowledge is scarce.
 +
|Designing national integration required in-depth knowledge of both eIDAS and OOTS. This knowledge (specifically the combination of both) is not broadly available in Member States. Knowledge of both domains should be brought together in order to prevent designs based on false assumptions of the other domain.
 
|-
 
|-
−
!<span style="color: blue">Companies</span>
+
|2
−
|<span style="color: blue">B</span>
+
|Scoping
−
| <span style="color: blue">Reduce manual work, lower transaction costs and improving enrolment speed for the company when using  the Once Only Principle</span>
+
|DBA advises the European Commission and Member States not to solve all user scenario's at once, but to focus on the most frequently used ones. Please focus on core functionality. And at the same time organise follow-ups on improvements and additions to address later on.
 +
|The project ran into many complex topics that needed to be solved in the pilot design phase. The pilot lead has organised a series of meetings to discuss these topics.  
 +
To keep focus at the core research questions and to limit resources needed, the pilot partners agreed to simplify whenever adequate, e.g. focussing at most important evidence type instead of all possible types, accepting request for one single evidence type at the time (instead of allowing requests for multiple evidence types), limiting to full powers validation to start with. The pilot secured progress and focus by scoping strictly.
 
|-
 
|-
−
! rowspan="2" | <span style="color: blue">Project</span>
+
|3
−
|<span style="color: blue">C</span>
+
|Company representation
−
|<span style="color: blue">Evaluate  the OOP-components supporting the cross-border information flow: </span>
+
|DBA advises the European Commission to clarify in advance which version of the eIDAS specification should be implemented for the SDGR to prevent incompatibility between Member States.
−
 
+
|Use of eIDAS including legal entity attributes (company representation) is not widespread in the EU. Currently, there are just two eID scheme's notified including legal person attributes. For piloting the partners set up a pilot network of eIDAS nodes including legal person attributes to allow for piloting eProcedures for companies. In preparing for the pilot, Member States turned out to communicate company representation in different ways. Especially regarding the use of the eIDAS representative attributes (representative prefix). Furthermore, during pilot preparation eIDAS node 2.4 became available. This version of the CEF reference software enforced the eiDAS 1.2-specification that turned out to be in conflict with the agreed use of eIDAS attributes in the DBA pilot. The eIDAS 1.2 specification regarding representation is not necesarilly backwards compatible. As a result, this raised additional confusion and led to inconsistent deployments.
−
<span style="color: blue">-          Assess (technical) impact on national services/registers already in place</span>
 
−
 
 
−
<span style="color: blue">-          Evaluate connections of national systems to the OOP TS </span>
 
 
|-
 
|-
−
|<span style="color: blue">D</span>
+
|4
−
| <span style="color: blue">Evaluate whether the solutions designed to the DBA specific challenges  have proven adequate in piloting the DBA eProcedures:</span>
+
|Powers validation
−
 
+
|DBA advises Member States to focus at implementing full-powers validation flows to start with. Adding more fine grained powers validation is required for 100% implementing the eProcedures, but also requires a more advanced solution.
−
<span style="color: blue">-          Usability of harmonised  Company Evidence model</span>
 
−
 
 
−
<span style="color: blue">-          Degree to which powers must  be validated </span>
 
−
 
 
−
<span style="color: blue">-          Scalability of solution for  powers validation</span>
 
  
−
<span style="color: blue">-          Usability and security of  Explicit Request and Preview</span>
 
  
−
<span style="color: blue">-          Need for record matching on  Natural Persons</span>
+
Furthermore, DBA advises the European Commission to facilitate validating full-powers using the currently available eIDAS. This requires an additional policy rule (please see DBA design documentation regarding this topic).
 +
|Validating full powers has been proven to be a first (and good) step in implementing cross-border OOP for businesses (requiring company representation). It allows for moving ahead with eIDAS as is available today and seems fitting for SME's (it will be an official representative initiating business abroad most of the time).
  
−
<span style="color: blue">-          Adequacy of patterns to keep  data up-to-date</span>
 
−
|}
 
−
 
−
<span style="color: blue">-          Success Criteria for Public Authorities</span>
 
−
{| class="wikitable" style="width: 80%;"
 
−
!<span style="color: blue">-          ID</span>
 
−
!<span style="color: blue">-          Criterion</span>
 
−
!<span style="color: blue">-          Technical Common Criteria</span>
 
−
!<span style="color: blue">-          Principles</span>
 
 
|-
 
|-
−
! colspan="4" |
+
|5
−
<span style="color:r blue d">Pilot goal A: Improve the quality of Company data within the service fulfilment process by re-using  data from authentic sources, thereby reducing manual work and lowering  processing costs</span>
+
|Record matching
 +
|DBA advises Member States to use the national company ID's as eIDASLegalIdentifiers when extending the pilot to SDG-wide implementation.
 +
|The pilot partners agreed to provide the national company registry numbers as eIDASLegalIdentifier in the authentication flow (eIDAS authentication proxy role). This diminished the need to do record matching on companies at the data owner. There was no substantial need to do record matching on the natural person by the data owners of the DBA pilot.
 
|-
 
|-
−
!<span style="color: blue">- A1</span>
+
|6
−
|
+
|Explicit request
−
 
+
|DBA advises data evaluators to integrate (1) request to consent and (2) explicit request into one joint question to the user to prevent adding to the confusion - of course in case both are applicable at the same time.
−
<span style="color: blue">- The DE recognizes  the company data is of higher quality, more reliable and easier to process  when using the OOP TS to retrieve company data directly from the DO. (e.g.  can data is available in an electronic and structured format for easy  processing in the systems of the DE, data requires less correcting, data is kept  up to date automatically, data is reliable and leads to less exceptions when  processing, data is more meaningful, has less inconsistencies and errors, is  more complete).</span>
+
|In some cases, users need to express consent for the retrieval of attributes (GDPR). In almost all cases when using the OOTS, the user needs to express explicit request (SDGR). Although legally sound, in practise the difference between both is difficult to understand for data evaluators. DE's furthermore expect that users will ignore such requests and just click "ok".
−
|
 
−
 
 
−
<span style="color: blue">- Reusability, Transparency, Effectiveness & Efficiency, Administrative  Simplification</span>
 
−
|
 
−
 
 
−
<span style="color: blue">- U, A, L, V</span>
 
−
|-
 
−
!<span style="color: blue">-  A2</span>
 
−
|
 
−
<span style="color: blue">-          The DE recognizes  the method of powers validation to provide data of higher quality and  reliability, proving that the representative is sufficiently authorized to represent the company (e.g. authorisation data is easier to interpret,  authenticity is clear, data is trustworthy, there is less manual work in validating the users powers to represent the company with documents proving  the relationship of the user to the company, authorization data requires less  correcting, verification is easier).</span>
 
−
|
 
−
<span style="color: blue">-          Reusability, Transparency, Effectiveness & Efficiency, Administrative  Simplification</span>
 
−
|
 
−
<span style="color: blue">-          U, A, L, V</span>
 
−
|}
 
−
<span style="color: blue">Success criteria for Companies applying for a service</span>
 
−
{| class="wikitable" style="width: 80%;"
 
−
!<span style="color: blue">ID</span>
 
−
!<span style="color: blue">Criterion</span>
 
−
!<span style="color: blue">Technical Common Criteria</span>
 
−
!<span style="color: blue">Principles</span>
 
−
|-
 
−
! colspan="4" |
 
−
<span style="color: blue">Pilot  goal B: Reduce manual work, lower transaction costs and improving enrolment  speed for the company when using the Once Only Principle </span>
 
 
|-
 
|-
−
!<span style="color: blue">B1</span>
+
|7
−
|
+
|Multiple-MS scenario's
−
<span style="color: blue">The user acknowledges the procedure for applying for a service to be  effective and efficient (e.g. the procedure requires acceptable effort and  cost, the procedure is not complex, has no language barriers, no  interruptions. The user spends little manual time to correct company data,  and experiences no errors after finishing the enrolment process).</span>
+
|DBA advises Member States to make an early start with the analysis of the SDG-implementation where data exchange involves more than 2 Member States.
−
|
+
|The pilot involved 2 Member States in the exchange of information on companies and representatives. The level of complexity for analysis increases vastly with each additional Member state that is involved in the exchange of information on representatives and companies. An example of a 3 MS-scenario could be a natural person (representative) from MS A, representing a legal person (represented) also from Member State A, which applies for a service from a Service Provider in Member State B and having to hand over evidence that is available in Member State C. Such an analysis introduces a level of complexity that exceeded the constraints of the pilot.  
−
<span style="color: blue">Reusability, Effectiveness & Efficiency, Administrative  Simplification, Transparency</span>
 
−
|
 
−
<span style="color: blue">U, A, L, V</span>
 
 
|-
 
|-
−
!<span style="color: blue">B2</span>
+
|8
−
|
+
|eIDAS non-notified eID
−
<span style="color: blue">The user acknowledges the method to proof their authorisation as  effective and efficient (e.g. requires little effort, is established with  simple and effective communication, is reliable).</span>
+
|DBA advises The European Commission and the Member States without notified eID's to agree on a temporarily solution for using non-notified eID's in SDG-procedures.
−
|
+
|Most of the participating Member States dind't operate a notified eID at the moment of piloting. On a bilateral basis non-notified eID's were accepted for piloting purposes, although pilot partners expressed their doubts regarding acceptance of non-notified eIDs for large scale SDG. Notification of eID's is a strong prerequisite for implementing SDG. Mandatory eID-notification as expected under the new eIDAS regulation (eIDAS revision) will not be in time for SDG-implementation.
−
<span style="color: blue">Reusability, Effectiveness & Efficiency,  Transparency, Security and Privacy</span>
 
−
|
 
−
<span style="color: blue">U, A, L, V</span>
 
 
|-
 
|-
−
!<span style="color: blue">B3</span>
+
|9
−
|
+
|Sector specific systems
−
<span style="color: blue">The user acknowledges the duration of completing the online eProcedure  activities to apply for a service as acceptable.</span>
+
|Integration of the OOTS with sectoral systems (BRIS in this pilot) has proven to be not so straight forward as many expected at the start of the project.
−
|
+
|For the DBA pilot alignment to - or integration with - BRIS has been an important topic from the start of the project. Much time has been spent on workshops, desk research and analysis. In the end, re-use of BRIS has been limited to semantics. Re-use of information flows, building blocks, etc. was not possible due to difference in legal framework, governance, authorities involved, solution implemented, etc.The solutions have been developed for different purposes and hence are not easily aligned.
−
<span style="color: blue">Effectiveness & Efficiency, Administrative  Simplification</span>
 
−
|
 
−
<span style="color: blue">V, A</span>
 
 
|-
 
|-
−
!<span style="color: blue">B4</span>
+
|10
−
|
+
|User interaction design
−
<span style="color: blue">The user saves time and/or cost when completing the eProcedure using  the OOP TS.</span>
+
|DBA advises the European Commission to provide wireframes in order to have generic steps (like Explicit Request and Preview) implemented in a similar way in all MS.
−
|
+
|Several data evaluators needed to implement the same logic in their specific systems, including user iteraction (general explanation, explicit request, preview). The user interaction design across participating Member States turned out to show some differences in informative texts, detail of explanation, use of buttons, etc. This may lead to confusion for the user that deals with multiple data evaluators as well as a slow learning curve. DBA decided to provide a pilot-wide reference in the form of wireframes to allow for more uniformity across the pilot.
−
<span style="color: blue">Effectiveness & Efficiency</span>
 
−
|
 
−
<span style="color: blue">V, A</span>
 
 
|}
 
|}
  
−
Success criteria and research questions for Pilot Technical Goals
+
==== ''3.2.2 Lessons learned from implementing and testing the DE4A OOTS'' ====
−
 
+
{| class="wikitable" style="width: 80%;"
−
{| class="wikitable" style="width: 80%;"
+
! style="width: 2%" |ID
−
!<span style="color: blue">'''ID'''</span>
+
! style="width: 10%" | Topic
−
!<span style="color: blue">'''Criterion'''</span>
+
! style="width: 33%" |Suggestions for adoption
−
!<span style="color: blue">'''Technical Common Criteria'''</span>
+
! style="width: 55%" | Lessons learned
−
!<span style="color: blue">'''Principles'''</span>
 
 
|-
 
|-
−
!<span style="color: blue"> colspan="4" |Pilot goal C: Evaluate the OOP-components supporting  the cross-border information flow:
+
|1
 +
|Planning and organising tasks
 +
|DBA advises to allocate a multi-month phase for establishing alignment, priorities, financial means etc. for all organizations involved.
  
−
·          Assess technical impact on national services/registers  already in place
+
Furthermore, it is necessary to have a coordinating team (equipped with sufficient knowledge about the solution) in each Member State to make sure that legal, semantical, technical and managerial issues are being resolved in a timely manner.
−
 
+
|The components to be used (in the pilot) were distributed over several authorities in a Member State, requiring the commitment from all authorities. This commitment is not obvious and must be secured beforehand. Also, as the systems are distributed, the teams working on the systems are distributed as well. Collaboration took more time and in each team, priorities were a continuous battle.
−
·          Evaluate connections of national systems to the OOP  TS</span>
 
 
|-
 
|-
−
!<span style="color: blue">C1 </span>
+
|2
−
|<span style="color: blue">The DO believes the cost and effort for integrating to the DE4A Connector will eventually be outweighed by the benefits.</span>
+
|Handing over
−
|<span style="color: blue">Openness, Technical Neutrality and Data Portability</span>
+
|DBA advises the European Commision to put additional efforts in explaining the workings of the SDG OOTS components to public authorities involved. The better the solution is understood by all, the smooher the SDG implementation will be.
−
|<span style="color: blue">U,  A, V</span>
+
The national complexity that the SDG imposes on Member States (e.g. record matching) is easily underestimated.  
 +
|Design documents and specification have sometimes been interpreted by different pilot partners in different ways. During preperation of the pilot or during interoperability testing such differences surfaced. It would be better to have a detailled common understanding of all the design details prior to the testing phase. Take the time for handing over Solution Architecture to WP3 and 5, and make sure that everything is understood.
 
|-
 
|-
−
!<span style="color: blue">C2</span>
+
|3
−
|<span style="color: blue">The DE believes the cost and effort for integrating to the DE4A Connector will eventually be outweighed by the benefits.</span>
+
|Documenting
−
|<span style="color: blue">Openness, Technical Neutrality and Data  Portability</span>
+
|DBA advises the European Commission to invest in proper and clear documentation for developers in Member States, so they can get the OOTS up and running with as less as possible efforts. Documentation should not be too cryptic and short, but definately also be not too extensive. Feedback on the documentation from first movers has proven to be very useful in the DBA pilot.
−
|<span style="color: blue">U, A, V</span>
+
Additionally, installing a small central team of technical experts providing support technical experts fin Member States, could be considered.
 +
|For developers of the common components, there's a lot of logic behind its internal logic, structure, configuration, etc. Deploying these components by the Member States in the DBA pilot raised several questions regarding the use of Docker images, configuration items that needed to be set correctly, required firewall and DNS settings, etc. Things were not always as straight forward as expected.
 
|-
 
|-
−
!<span style="color: blue">C3</span>
+
|4
−
|<span style="color: blue">The DO believes the cost and effort for integrating to the Mandate Management System will eventually be outweighed by  the benefits.</span>
+
|Configuring
−
|<span style="color: blue">Openness, Technical Neutrality and Data  Portability</span>
+
|DBA advises Member States to prepare for the steps to be taken to request the certificates needed to operate the OOTS.
−
|<span style="color: blue">U, L, V</span>
+
DBA advises the European Commission to investigate whether the process for acquiring the OOTS certificates can be simplified.
 +
 
 +
DBA advises the European Commission to design a procedure for communication between Member States in case of change of certificates and to provide for certificate-rollover to guarantee OOTS-connectivity.  
 +
|The components needed for SDG rely heavily on use and exchange of certificates for server authentication, signing, etc. The process of acquiring the certificates turned out to be time-consuming and error-prone (all details must be in place when requesting the certificates). Furthermore, the procedure of requesting certificates is regulated in a way it requires signatures of responsible people within the requesting institution that do not on a daily basis work with - and understand the use of - certificates. Or people that are not available imidiately (company executives).
 
|-
 
|-
−
!<span style="color: blue">C4</span>
+
|5
−
|<span style="color: blue">The participating Member States believe the  cost and effort for setting up and deploying the DE4A Connector in their  national infrastructure will eventually be outweighed by the benefits.</span>
+
|Integrating DE and DO
−
|<span style="color: blue">Openness, Technical Neutrality and Data  Portability</span>
+
|DBA advises Member States to take impact on existing systems into account. Including the backlog of items that might need to be resolved before being able to connect to the OOTS.
−
|<span style="color: blue">U, L, V</span>
+
|
 +
When integrating to the DT/DR, expect to run into existing problems in the DO/DE systems that need resolving as well. This will create extra work, although the work is not directly being created due to integration with the DT/DR. The problems in the DE/DO systems were existing already, but were not causing real issues until then (problems were accepted) but might need to be resolved in order to achieve good integration to the DT/DR.
 
|-
 
|-
−
! <span style="color: blue">colspan="4" |Pilot  goal D: Evaluate whether the solutions designed to the DBA specific  challenges have proven adequate in piloting the DBA eProcedures</span>
+
|6
 +
|Interopability testing
 +
|Wider OOTS implementation requires more inter-Member State coordination regarding exchange of connectivity details, configuration and cross-border interoperability testing. Planning of these activities requires much attention and flexibility from the Member States. DBA advises to take this into account when connecting the decentralised SDG OOTS components. We refer also to the eIDAS lessons learned with regards to exchange fo certificates etc.
 +
|The speed of development varies per Member State. Therefore readiness for cross-border testing (and piloting, for that matter) is also distributed in time. MS A can have their DE ready months before MS B has (due to several national impediments). Testing on fixed moments in time for all DEs and all DOs has proven not realistic, as does starting pilots at one moment in time.
 
|-
 
|-
−
!<span style="color: blue">D1</span>
+
|7
−
|<span style="color: blue">Has the Company Evidence Model proven adequate for cross-border exchange of information on companies for the DBA  eProcedures?</span>
+
|Interopability testing
−
|<span style="color: blue">Openness, Neutrality and Data Portability,  Reusability</span>
+
|Establish clear readiness criteria for DE/DO DE4A Connector before starting connectathons.
−
|<span style="color: blue">U, V, L</span>
+
|The DBA pilot has proven that a lot of settings need to be configured correctly to allow succesful cross-border evidence exchange. During interoperability testing (connecthatons) Member States sometimes had different views on what components or parameters had to be set in order to start testing. As a result, not in all cases the complete flow could be tested at once.
 
|-
 
|-
−
!<span style="color: blue">D2</span>
+
|8
−
|<span style="color: blue">Have the solutions to validate powers  proven adequate for the eProcedures involved in piloting?</span>
+
|Interoperability testing
−
|<span style="color: blue">Reusability, Administrative Simplification</span>
+
|DBA advises the European Commission to coordinate exchange of test credentials between Member States. Many-to-many "requesting and sending of eID's on a bilateral basis" should be prevented.
−
|<span style="color: blue">U, L</span>
+
|Proper interoperability testing is only possible with the required test eID means. These national eID means have not always been easily available (depending on the MS-specific situation - dependancies on IdP's may exist). This hindered cross-border interoperability testing at some occasions. The effect of lacking test credentials will be much greater in case of large scale implementing the SDGR.
 
|-
 
|-
−
!<span style="color: blue">D3</span>
+
|9
−
|<span style="color: blue">Have the explicit request and preview requirements as specified in the SDGR proveD5n suitable for company  eProcedures (representation scenarios)?</span>
+
|Reliance on eIDAS
−
|<span style="color: blue">Administrative Simplification, User  Centricity, Inclusion and Accessibility</span>
+
|DBA advises the Member States to setup and test national eIDAS deployment prior to implementing the SDGR, to prevent this delaying SDGR.
−
|<span style="color: blue">U, L</span>
+
|DBA pilotting - just as SDG implementation - relies on use of eIDAS. Unfortunately, eIDAS is not fully up and running in all Member States. In preparing for the DBA pilot, Member States had to setup eIDAS as well (pilot network of eIDAS nodes). In interoperability testing, several eIDAS related setup-issues needed to be solved.
 
|-
 
|-
−
!<span style="color: blue">D4</span>
+
|10
−
|<span style="color: blue">Have the mechanisms for record matching  at the DC proven adequate for the DBA eProcedures?</span>
+
|SDG implementing acts
−
|<span style="color: blue">Administrative Simplicity</span>
+
|DBA advises the European Commission and Member States to be aware no such thing as 'a final version' exists in the area of inter-Member State information exchange. Moving forward step-by-step with versions currently available is crucial to progress. Note that continuous alignment with all European initiatives during single steps is not feasible and will delay each initiative started.
−
|<span style="color: blue">U, L</span>
+
|DBA pilot implementation has been delayed by numerous discussions (within Member States and between Member States) on alignment with the SDG OOTS that was being sketched at the same time. Although this approach had been deliberately chosen and agreed upon at the start of the DBA project (to enable real piloting and provide input to SDG), in practise discussions were raised over and over again and caused prioritization challenges for the pilot activities of partners.
 
|-
 
|-
−
!<span style="color: blue">D5</span>
+
|11
−
|<span style="color: blue">Have the mechanisms to keep the company  information up-to-date (second pilot iteration) proven adequate</span>
+
|Cooperation
−
|<span style="color: blue">Administrative Simplicity, Effectiveness &  Efficiency</span>
+
|DBA advises to facilitate technical experts of the Commission and the Member States to easily ask each other questions, respond, etc. using a tool for this purpose, e.g. Slack.
−
|<span style="color: blue">U, V</span>
+
|Slack seems to be a good means to have developers of different MS / WPs collaborate.
 
|}
 
|}
  
−
==3.2 Pilot dimensions==
+
==== ''3.2.3 Technical, semantic and organisational/legal knowledge provided to other WPs'' ====
−
''Qualitative description on lessons learned (technical, functional, proess, data, usability etc) and preliminary conclusions on these dimensions, based on metrics, questiuonnaires and interviews? The dimensions target the scope of the piloted functionality and patterns (until delivery of the report)''
+
*
  
−
===3.2.1 Use ===
+
*
 +
*
 +
{| class="wikitable" style="width: 80%;"
 +
!style="width: 2%"|ID
 +
!style="width: 10%"| Topic
 +
! style="width: 33%" |Suggestions for adoption
 +
! style="width: 55%" | Lessons learned
 +
|-
 +
|1
 +
|Communication
 +
|Use visual tools to show the benefits of OOP to users, e.g. presentations and videos.
  
−
*''Overview''
+
Prepare the creation of an animation by setting up a good storyline and slides that illustrate the flow of the animation.
−
*''Initial feedback from focus group and real users''
+
|Implementation of the Oncle Only Principle might be interpreted as abstract by users / companies that might benefit from it. From a user perspective, there's not too much to see in the OOP-process. OOP might be interpreted as 'not a big deal' by the user. Large parts of the solution are "complexity under the hood". Hence, additional efforts are needed to explain in an understandable way the hugh difference that OOP makes.
−
*''Initial results from use related metrics (logs)''
+
|}
−
*''Usefulness of DE4A patterns and components related to internal stakeholders take-up''
 
−
*''Strategy on pilot use until final report''
 
  
−
=== 3.2.2 Value===
+
==== ''3.2.4 Pilot learning for Sustainable impact and new governance models'' ====
−
 
+
{| class="wikitable" style="width: 80%;"
−
*''Verified benefits with users and DEs / DOs''  
+
! style="width: 2%" |ID
−
**''Contribution of pilot to DE4A benefits and to external community of SDG stakeholders''
+
! style="width: 10%" | Topic
−
**''Pilot specific benefits''
+
! style="width: 33%" |Suggestions for adoption
−
 
+
! style="width: 55%" | Lessons learned
−
===3.2.3 Learning towards Adoption===
+
|-
−
 
+
|1
−
*''Approach to knowledge-building''
+
|Harmonisation
−
*''Lessons learned from integration and testing useful for Adoption''
+
|DBA advises the European Commission to organise the harmonisation process of services for cross-border powers validation (see SEMPER project results and DBA pilot for sample harmonisation).
−
**<span style="color:red"> components to be used (in the pilot) are sometimes distributed over several authorities in a Member State, requiring the commitment from all authorities. This commitment is not obvious and must be secured beforehand. Also, as the systems are distributed, the teams working on each system are distributed. Collaboration takes more time and in each team, a battle for prioritizing DE4A needs to be fought. Finally, because there are many systems involved (on both the DO and the DE- side) it is necessary to have a coordinating team (having sufficient knowledge about the DE4A solution) in each Member State to make sure that planning and communication issues are being resolved.<span>
+
|For fine-grained powers validation, Member States need to agree on a harmonised set of services. In the DBA pilot: the SDG procedures of annex II to start with.
−
** <span style="color:red">The speed of development varies per Member State. Therefore readiness for testing (and piloting, for that matter) of combinations of Data Owners and Data Evaluators from different Member States is also distributed in t<span style="color:red"><span>ime. MS A can have their DE ready months before MS B has (due to several impediments). Testing on fixed moments in time for alle DEs and all DOs has proven not realistic, as does starting pilots at one moment in time. DE/DO combinations will become available for testing and piloting over a period of several weeks or months. </span>
+
|-
−
** <span style="color:red"> Establish clear readiness criteria for DE/DO/DE4A Connector before starting connectathons.</span>
+
|2
−
** <span style="color:red"> Take the time for handing over Solution Architecture to WP3 and 5, and make sure that everything is understood. </span>
+
|Harmonisation
−
** <span style="color:red"> Keep audit-trail and error-logging simple, considering the limited number of participating companies and the highly controlled fashion of the pilot.</span>
+
|DBA advises the European Commission to organise the harmonisation process of event types for cross-border subscription & notification. See DBA pilot for example of harmonisation.
−
** <span style="color:red"> Collect pilot data in the most simple way possible, without risking data loss or integrity-loss. But don't set up complex systems to vcollect data for metrics. </span>
+
|For subscribing and notifying on company events / changes there needs to be a specified set of harmonised company event types.
−
** <span style="color:red"> When integrating to the DT/DR, expect to run into existing problems in the DO/DE systems that need resolving as well. This will create extra work, although the work is not directly being created due to integration with the DT/DR. The problems in the DE/DO systems were existing already, but were not causing real issues until then (problems were accepted) but might need to be resolved in order to achieve good integration to the DT/DR.</span>
+
|}
−
** <span style="color:red"> Work with wireframes in order to have Generic steps (Like Explicit Request and Preview) implemented in a similar way in all MS.</span>
 
−
** <span style="color:red"> Invest in good and clear documentation for developers in MS, so they can get the DE4A Connector up and running, as well as integrate to the DE4A Connector wit minimal effort.</span>
 
−
** <span style="color:red"> Perhaps a note on using Docker for the DE4A Connector? This caused issues during deployment as well as testing .</span>
 
−
** <span style="color:red"> Perhaps a note on firewalls and DNS? These things caused several issues during connectathons.</span>
 
−
** <span style="color:red"> Make sure to have a sufficient number of test eIDs available during development and testing. </span>
 
−
** <span style="color:red"> Something on the availability and usability of eIDAS nodes for piloting? There were many challenges in this domain.</span>
 
−
** <span style="color:red"> Working with working assumptions to reduce uncertainties and secure progress seems lto be good practice. </span>
 
−
*** <span style="color:red"> 1 evicende type, no multi-evidence, use full powers validation etc. </span>
 
−
** <span style="color:red"> Obtaining certificates proves not easy due to signing of documents and actually receiving the certificates.</span>
 
−
*''Technical, semantic and organisational/legal knowledge provided to other WPs''
 
−
**<span style="color:red"> Prepare the creation of an animation by setting up a good storyline and slides that illustrate the flow of the animation. </span>
 
−
** <span style="color:red"> Slack seems to be a good means to have developers of different MS / WPs collaborate </span>
 
−
** <span style="color:red"> Also use input from additional questionnaire regarding technical issues and process of connecting (connectathons) and documentation etc </span>
 
−
*''Pilot learning for “Sustainable impact and new governance models” WP (to be agreed e.g. Sustainability recommendations, standardisation needs)''
 
−
*''Lessons being learned from users (questionnaires & interviews)''
 
−
*''Lessons being learned from DEs and DOs (results and outputs questionnaires & interviews)''
 
−
** <span style="color:red"> Bank account information is not part of the CompanyEvidence. But it can be requested to provide afterwards, in screens in the eProcedure after exchange of evidence by the OOP TS. It seems that BIC-codes are unique, but BANK-codes (like INGB etc) are not unique across Europe (but are unique within one country). The DE might do some kind of validation on the bankcode and might then run into issues because of new bank-codes (from Romania, for example) that are identical to bank-codes in the country of the DE (The Netherlands for example). DE systems should therefore consider closely how to work with bank information (when entered manually, but also in case of future extension of CompanyEvidence). </span>
 
−
*''Other lessons from interaction with other initiatives (SEMPER, EBSI…)''
 
−
** <span style="color:red"> The added value of the OOP TS compared to other developments / pilots / experiments in the EU must be broadcasted continuously in order to keep authorities involved and secure commitment. </span>
 
−
** <span style="color:red"> (Potential) differences between the DE4A scope and the Implementing Act distract and demotivate participants from prioritizing development and deployment of changes in their national infrastructures.</span>
 
−
 
 
−
==3.3 Technical common criteria (questionnaire for evaluation?) ==
 
−
''[Qualitative description of preliminary conclusion per criterium. Explanation (in written) on how success criteria /metrics are related with Technical common criteria, distributed by pilot dimensions]''
 
−
 
 
−
''Openness => U, A''
 
−
 
 
−
''Transparency => U, V''
 
−
 
 
−
''Reusability => V, L''
 
−
** <span style="color:red"> eIDAS tech profile 1.2 is more prescriptive than profile 1.1. Depending on the way profile 1.1 is implemented in a MS, it might not work with the way 1.2 prescribes the implementation of this version, introducing an error in the authentication process between MS A (using 1.1) and MS B (using 1.2). </span>
 
−
 
 
−
''Technological neutrality and data portability => L''
 
−
 
 
−
''User-centricity => V''
 
−
 
 
−
''Inclusion and accessibility => U''
 
−
 
 
−
''Security and privacy => U''
 
  
−
''Administrative simplification => A, V''
 
  
−
''Effectiveness and efficiency => A, V''
 
−
** <span style="color:red"> Not sure if this is the right pace, but we should address the design decisions we made too. And for that matter, also say something about how we comply to the SDG (or where we deviate and why, for example we work with 1 company evidence type while the SDG states that a DE should not receive more attributes than they strictly need for the procedure, which might not always be the case in DBA). Another example is that the preview is provided by the DE, meaning that the transfer has already taken place.</span>
 
  
−
''[Qualitative comments, and follow-up with quantitative comments on second iteration]''
 
  
 
[[Pilot Procedures|Next Chapter: 4. Pilot Procedures]]
 
[[Pilot Procedures|Next Chapter: 4. Pilot Procedures]]

Latest revision as of 16:27, 10 February 2022

Back to Doing Business Abroad main page

Back to main page of D4.7 Initial Running Phase Report

Back to Previous Chapter: 2. Current Status of Pilot

[Work in progress]

3.1 Introduction

The analyses conducted with regards to challenges that the implementation of the SDGR introduces, as wel as the realization of the infrastructure implementing the SDGR and conclusions of these analyses, proved very fruitful. This chapter summarizes the lessons learned form these activities, and provides suggestions for the implementation and adoption of the SDGR implementation.

3.2 Learning towards Adoption

3.2.1 Lessons learned from analysing and designing national integration of cross-border OOP

ID Topic Suggestions for adoption Lessons learned
1 Design process DBA advises Member States to invest time to bring together the eIDAS and OOTS knowledge. This requires organising and prioritising as this knowledge is scarce. Designing national integration required in-depth knowledge of both eIDAS and OOTS. This knowledge (specifically the combination of both) is not broadly available in Member States. Knowledge of both domains should be brought together in order to prevent designs based on false assumptions of the other domain.
2 Scoping DBA advises the European Commission and Member States not to solve all user scenario's at once, but to focus on the most frequently used ones. Please focus on core functionality. And at the same time organise follow-ups on improvements and additions to address later on. The project ran into many complex topics that needed to be solved in the pilot design phase. The pilot lead has organised a series of meetings to discuss these topics.

To keep focus at the core research questions and to limit resources needed, the pilot partners agreed to simplify whenever adequate, e.g. focussing at most important evidence type instead of all possible types, accepting request for one single evidence type at the time (instead of allowing requests for multiple evidence types), limiting to full powers validation to start with. The pilot secured progress and focus by scoping strictly.

3 Company representation DBA advises the European Commission to clarify in advance which version of the eIDAS specification should be implemented for the SDGR to prevent incompatibility between Member States. Use of eIDAS including legal entity attributes (company representation) is not widespread in the EU. Currently, there are just two eID scheme's notified including legal person attributes. For piloting the partners set up a pilot network of eIDAS nodes including legal person attributes to allow for piloting eProcedures for companies. In preparing for the pilot, Member States turned out to communicate company representation in different ways. Especially regarding the use of the eIDAS representative attributes (representative prefix). Furthermore, during pilot preparation eIDAS node 2.4 became available. This version of the CEF reference software enforced the eiDAS 1.2-specification that turned out to be in conflict with the agreed use of eIDAS attributes in the DBA pilot. The eIDAS 1.2 specification regarding representation is not necesarilly backwards compatible. As a result, this raised additional confusion and led to inconsistent deployments.
4 Powers validation DBA advises Member States to focus at implementing full-powers validation flows to start with. Adding more fine grained powers validation is required for 100% implementing the eProcedures, but also requires a more advanced solution.


Furthermore, DBA advises the European Commission to facilitate validating full-powers using the currently available eIDAS. This requires an additional policy rule (please see DBA design documentation regarding this topic).

Validating full powers has been proven to be a first (and good) step in implementing cross-border OOP for businesses (requiring company representation). It allows for moving ahead with eIDAS as is available today and seems fitting for SME's (it will be an official representative initiating business abroad most of the time).
5 Record matching DBA advises Member States to use the national company ID's as eIDASLegalIdentifiers when extending the pilot to SDG-wide implementation. The pilot partners agreed to provide the national company registry numbers as eIDASLegalIdentifier in the authentication flow (eIDAS authentication proxy role). This diminished the need to do record matching on companies at the data owner. There was no substantial need to do record matching on the natural person by the data owners of the DBA pilot.
6 Explicit request DBA advises data evaluators to integrate (1) request to consent and (2) explicit request into one joint question to the user to prevent adding to the confusion - of course in case both are applicable at the same time. In some cases, users need to express consent for the retrieval of attributes (GDPR). In almost all cases when using the OOTS, the user needs to express explicit request (SDGR). Although legally sound, in practise the difference between both is difficult to understand for data evaluators. DE's furthermore expect that users will ignore such requests and just click "ok".
7 Multiple-MS scenario's DBA advises Member States to make an early start with the analysis of the SDG-implementation where data exchange involves more than 2 Member States. The pilot involved 2 Member States in the exchange of information on companies and representatives. The level of complexity for analysis increases vastly with each additional Member state that is involved in the exchange of information on representatives and companies. An example of a 3 MS-scenario could be a natural person (representative) from MS A, representing a legal person (represented) also from Member State A, which applies for a service from a Service Provider in Member State B and having to hand over evidence that is available in Member State C. Such an analysis introduces a level of complexity that exceeded the constraints of the pilot.
8 eIDAS non-notified eID DBA advises The European Commission and the Member States without notified eID's to agree on a temporarily solution for using non-notified eID's in SDG-procedures. Most of the participating Member States dind't operate a notified eID at the moment of piloting. On a bilateral basis non-notified eID's were accepted for piloting purposes, although pilot partners expressed their doubts regarding acceptance of non-notified eIDs for large scale SDG. Notification of eID's is a strong prerequisite for implementing SDG. Mandatory eID-notification as expected under the new eIDAS regulation (eIDAS revision) will not be in time for SDG-implementation.
9 Sector specific systems Integration of the OOTS with sectoral systems (BRIS in this pilot) has proven to be not so straight forward as many expected at the start of the project. For the DBA pilot alignment to - or integration with - BRIS has been an important topic from the start of the project. Much time has been spent on workshops, desk research and analysis. In the end, re-use of BRIS has been limited to semantics. Re-use of information flows, building blocks, etc. was not possible due to difference in legal framework, governance, authorities involved, solution implemented, etc.The solutions have been developed for different purposes and hence are not easily aligned.
10 User interaction design DBA advises the European Commission to provide wireframes in order to have generic steps (like Explicit Request and Preview) implemented in a similar way in all MS. Several data evaluators needed to implement the same logic in their specific systems, including user iteraction (general explanation, explicit request, preview). The user interaction design across participating Member States turned out to show some differences in informative texts, detail of explanation, use of buttons, etc. This may lead to confusion for the user that deals with multiple data evaluators as well as a slow learning curve. DBA decided to provide a pilot-wide reference in the form of wireframes to allow for more uniformity across the pilot.

3.2.2 Lessons learned from implementing and testing the DE4A OOTS

ID Topic Suggestions for adoption Lessons learned
1 Planning and organising tasks DBA advises to allocate a multi-month phase for establishing alignment, priorities, financial means etc. for all organizations involved.

Furthermore, it is necessary to have a coordinating team (equipped with sufficient knowledge about the solution) in each Member State to make sure that legal, semantical, technical and managerial issues are being resolved in a timely manner.

The components to be used (in the pilot) were distributed over several authorities in a Member State, requiring the commitment from all authorities. This commitment is not obvious and must be secured beforehand. Also, as the systems are distributed, the teams working on the systems are distributed as well. Collaboration took more time and in each team, priorities were a continuous battle.
2 Handing over DBA advises the European Commision to put additional efforts in explaining the workings of the SDG OOTS components to public authorities involved. The better the solution is understood by all, the smooher the SDG implementation will be.

The national complexity that the SDG imposes on Member States (e.g. record matching) is easily underestimated.

Design documents and specification have sometimes been interpreted by different pilot partners in different ways. During preperation of the pilot or during interoperability testing such differences surfaced. It would be better to have a detailled common understanding of all the design details prior to the testing phase. Take the time for handing over Solution Architecture to WP3 and 5, and make sure that everything is understood.
3 Documenting DBA advises the European Commission to invest in proper and clear documentation for developers in Member States, so they can get the OOTS up and running with as less as possible efforts. Documentation should not be too cryptic and short, but definately also be not too extensive. Feedback on the documentation from first movers has proven to be very useful in the DBA pilot.

Additionally, installing a small central team of technical experts providing support technical experts fin Member States, could be considered.

For developers of the common components, there's a lot of logic behind its internal logic, structure, configuration, etc. Deploying these components by the Member States in the DBA pilot raised several questions regarding the use of Docker images, configuration items that needed to be set correctly, required firewall and DNS settings, etc. Things were not always as straight forward as expected.
4 Configuring DBA advises Member States to prepare for the steps to be taken to request the certificates needed to operate the OOTS.

DBA advises the European Commission to investigate whether the process for acquiring the OOTS certificates can be simplified.

DBA advises the European Commission to design a procedure for communication between Member States in case of change of certificates and to provide for certificate-rollover to guarantee OOTS-connectivity.

The components needed for SDG rely heavily on use and exchange of certificates for server authentication, signing, etc. The process of acquiring the certificates turned out to be time-consuming and error-prone (all details must be in place when requesting the certificates). Furthermore, the procedure of requesting certificates is regulated in a way it requires signatures of responsible people within the requesting institution that do not on a daily basis work with - and understand the use of - certificates. Or people that are not available imidiately (company executives).
5 Integrating DE and DO DBA advises Member States to take impact on existing systems into account. Including the backlog of items that might need to be resolved before being able to connect to the OOTS.

When integrating to the DT/DR, expect to run into existing problems in the DO/DE systems that need resolving as well. This will create extra work, although the work is not directly being created due to integration with the DT/DR. The problems in the DE/DO systems were existing already, but were not causing real issues until then (problems were accepted) but might need to be resolved in order to achieve good integration to the DT/DR.

6 Interopability testing Wider OOTS implementation requires more inter-Member State coordination regarding exchange of connectivity details, configuration and cross-border interoperability testing. Planning of these activities requires much attention and flexibility from the Member States. DBA advises to take this into account when connecting the decentralised SDG OOTS components. We refer also to the eIDAS lessons learned with regards to exchange fo certificates etc. The speed of development varies per Member State. Therefore readiness for cross-border testing (and piloting, for that matter) is also distributed in time. MS A can have their DE ready months before MS B has (due to several national impediments). Testing on fixed moments in time for all DEs and all DOs has proven not realistic, as does starting pilots at one moment in time.
7 Interopability testing Establish clear readiness criteria for DE/DO DE4A Connector before starting connectathons. The DBA pilot has proven that a lot of settings need to be configured correctly to allow succesful cross-border evidence exchange. During interoperability testing (connecthatons) Member States sometimes had different views on what components or parameters had to be set in order to start testing. As a result, not in all cases the complete flow could be tested at once.
8 Interoperability testing DBA advises the European Commission to coordinate exchange of test credentials between Member States. Many-to-many "requesting and sending of eID's on a bilateral basis" should be prevented. Proper interoperability testing is only possible with the required test eID means. These national eID means have not always been easily available (depending on the MS-specific situation - dependancies on IdP's may exist). This hindered cross-border interoperability testing at some occasions. The effect of lacking test credentials will be much greater in case of large scale implementing the SDGR.
9 Reliance on eIDAS DBA advises the Member States to setup and test national eIDAS deployment prior to implementing the SDGR, to prevent this delaying SDGR. DBA pilotting - just as SDG implementation - relies on use of eIDAS. Unfortunately, eIDAS is not fully up and running in all Member States. In preparing for the DBA pilot, Member States had to setup eIDAS as well (pilot network of eIDAS nodes). In interoperability testing, several eIDAS related setup-issues needed to be solved.
10 SDG implementing acts DBA advises the European Commission and Member States to be aware no such thing as 'a final version' exists in the area of inter-Member State information exchange. Moving forward step-by-step with versions currently available is crucial to progress. Note that continuous alignment with all European initiatives during single steps is not feasible and will delay each initiative started. DBA pilot implementation has been delayed by numerous discussions (within Member States and between Member States) on alignment with the SDG OOTS that was being sketched at the same time. Although this approach had been deliberately chosen and agreed upon at the start of the DBA project (to enable real piloting and provide input to SDG), in practise discussions were raised over and over again and caused prioritization challenges for the pilot activities of partners.
11 Cooperation DBA advises to facilitate technical experts of the Commission and the Member States to easily ask each other questions, respond, etc. using a tool for this purpose, e.g. Slack. Slack seems to be a good means to have developers of different MS / WPs collaborate.

3.2.3 Technical, semantic and organisational/legal knowledge provided to other WPs

ID Topic Suggestions for adoption Lessons learned
1 Communication Use visual tools to show the benefits of OOP to users, e.g. presentations and videos.

Prepare the creation of an animation by setting up a good storyline and slides that illustrate the flow of the animation.

Implementation of the Oncle Only Principle might be interpreted as abstract by users / companies that might benefit from it. From a user perspective, there's not too much to see in the OOP-process. OOP might be interpreted as 'not a big deal' by the user. Large parts of the solution are "complexity under the hood". Hence, additional efforts are needed to explain in an understandable way the hugh difference that OOP makes.

3.2.4 Pilot learning for Sustainable impact and new governance models

ID Topic Suggestions for adoption Lessons learned
1 Harmonisation DBA advises the European Commission to organise the harmonisation process of services for cross-border powers validation (see SEMPER project results and DBA pilot for sample harmonisation). For fine-grained powers validation, Member States need to agree on a harmonised set of services. In the DBA pilot: the SDG procedures of annex II to start with.
2 Harmonisation DBA advises the European Commission to organise the harmonisation process of event types for cross-border subscription & notification. See DBA pilot for example of harmonisation. For subscribing and notifying on company events / changes there needs to be a specified set of harmonised company event types.



Next Chapter: 4. Pilot Procedures

Retrieved from ‘https://wiki.de4a.eu/index.php?title=Goals_and_success_criteria&oldid=4459’