Difference between revisions of "Subscription and Notification Pattern"
Line 27: | Line 27: | ||
# Extended Evidence Type: The Specific Evidence Update can be used to derive relevant Events at the DC, if the evidence type definition is scoped, or extended, so that all relevant events are represented by data changes in the evidence type data set. This would require the DC to run a event identification routine on each received update to react accordingly. This hybrid solution would also meet its limitations for events that are dealt with by procedural means at the DP-side, rather than by data mutations. The extension of evidence type definitions beyond what is actually required by the eProcedure is, again, not optimal from a data minimization principle perspective. | # Extended Evidence Type: The Specific Evidence Update can be used to derive relevant Events at the DC, if the evidence type definition is scoped, or extended, so that all relevant events are represented by data changes in the evidence type data set. This would require the DC to run a event identification routine on each received update to react accordingly. This hybrid solution would also meet its limitations for events that are dealt with by procedural means at the DP-side, rather than by data mutations. The extension of evidence type definitions beyond what is actually required by the eProcedure is, again, not optimal from a data minimization principle perspective. | ||
− | # Extended Notification Message: The pure Event Notification could be extended to carry an updated evidence as part of the Notification message, essentially piggy-bagging the evidence on an event notification. This approach would need to extend the event-centric subscription system to maintain with underlying evidence type, or potentially multiple evidence types, are potentially relevant for each event type, so that evidence can be retrieved and packed as extension of the Notification Message. This solution would consequently end up to be rather complext and again not optimal from a from a data minimization principle perspective. | + | # Extended Notification Message: The pure Event Notification could be extended to carry an updated evidence as part of the Notification message, essentially piggy-bagging the evidence on an event notification. This approach would need to extend the event-centric subscription system to maintain with underlying evidence type, or potentially multiple evidence types, are potentially relevant for each event type, so that evidence can be retrieved and packed as extension of the Notification Message. This solution would consequently end up to be rather complext and again not optimal from a from a data minimization principle perspective. [A BPMN Process Analysis of this approach is available on request] |
# Event Notification + Lookup: The pure Event Notification is used as described above, while it is made sure that the agreed event set of canonical DE4A events is complete in the sense that in covers also relevant update needs, for example a company that moved HQ means that the adres (part of the Company Registration evidence) changed as an effect. It is then left at the discretion of the DC, whether a particular event requires an update of the evidence on record. It could be that, i.e. the postal adres of a company is nor relevant for the continuation of the public service provisioning, in which case a flag that the adres on record is not up to date, or a change to "unknown" would be sufficient as event response. This is fully in line with the data minimization principle perspective. If the DC considers that an an update of one evidence or potentially several evidences is required as part of the event response, then the DC would use the [[Lookup Pattern]] to request new copies of evidences. | # Event Notification + Lookup: The pure Event Notification is used as described above, while it is made sure that the agreed event set of canonical DE4A events is complete in the sense that in covers also relevant update needs, for example a company that moved HQ means that the adres (part of the Company Registration evidence) changed as an effect. It is then left at the discretion of the DC, whether a particular event requires an update of the evidence on record. It could be that, i.e. the postal adres of a company is nor relevant for the continuation of the public service provisioning, in which case a flag that the adres on record is not up to date, or a change to "unknown" would be sufficient as event response. This is fully in line with the data minimization principle perspective. If the DC considers that an an update of one evidence or potentially several evidences is required as part of the event response, then the DC would use the [[Lookup Pattern]] to request new copies of evidences. | ||
Revision as of 12:49, 4 June 2021
Is used by Doing Business Abroad Pilot in Use Case "Doing Business in Another Member State" (DBA UC2)
After receiving evidence from a Data Owner (DO), it can be essential for a Data Evaluator (DE) to be informed on changes regarding the subject of this evidence to be able to take appropriate action. The goal of this interaction pattern is to allow the DE to subscribe to a service of the DO that provides autmatic and regual notifications. The cross-border message exchange for the subscriptions and notifications are put in the responsibility of the Data Requester (DR) and the Data Transferor (DT) respectively to allow an easier distribution of responsibility on national level, i.e. to intermediary platforms an national gateway providers.
Functional Variants of the Subscription and Notification Pattern
There are two distinct purposes, or business requirements for Subscription and Notification, which both are relevant for the DE4A Doing Business Abroad Pilot: Evidence update notification and Event notification
Evidence Update Notification
The goal is to keep previously shared evidence data that is stored at the DE up to date.
Description: Data may change in the base register. In case the DE needs an exact copy of the evidence data on record, they need to be notified in case the data has changed in teh base registry.
Business or Life Event Notification
The goal is to assess the impact of company changes on the eServices provided by the data evaluator.
Description: Some eServices oblige the subject (i.e. company or citizen) to continue in a specifc situation or state to remain entitled to the benefits of an eService provided to the subjcet. E.g., an agricultural company may receive a subsidy for its permanent pasture. As a prerequisite, the company must preserve the pasture for five consecutive years. The data evaluator needs to be notified of the company ending its operations and hence not meeting the five-year requirement. “ending its operation” is an example of a business event. Others examples are: going bankrupt, merge, etc.
Alternative Solution Approaches
Essentially there are two ways to approach the dual requirements for Subscription and Notification, either specific solutions for each requirement or hybrid approaches that can serve both.
Looking at specific solutions would mean that two specialized systems would need to be developed and implemented:
- Specific Evidence Update Solution: The subscription would be linked directly to the previously exchanged evidence, hence the subscription would need to register the evidence type the subject ID and the subscriber ID. For any data change in the evidence type data set at the DO, the DP would need to push a complete new evidence to the DC, irrespective of that specific change being relevant for the public service provided to the user by the DC. Apart from this being not optimal from a data minimization principle perspective, it can not cover changes of the subject that are not represented in the evidence type data set, giving rise to a second subscription system for event. In addition, the subscription system would be impacted by changes in the canonical evidence type definitions over time. This approach might, however, be easier to integrate in the legal framework of the SDGR (see Legal Considerations below).
- Pure Event Notification: The subscription would not need to be directly related to a previously shared exchanges evidence, but the subscription would be registered for an agreed set of events relating to a given subject. The subscription system would consequently be based on a list of events, rather than a list of evidence types. Quite obviously, this requires an agreement on a set of DE4A events, or canonical cross-border events. We expect that resolving existing legal barriers especially in the citizen domain would be easier for pure event notifications, becausi theNotification would only contain identifiers rather than the underlying (personal) data.
Technical approach and business requirement do not relate one two one, giving rise to several hybrid approaches, where one solution is used to cater for both requirements:
- Extended Evidence Type: The Specific Evidence Update can be used to derive relevant Events at the DC, if the evidence type definition is scoped, or extended, so that all relevant events are represented by data changes in the evidence type data set. This would require the DC to run a event identification routine on each received update to react accordingly. This hybrid solution would also meet its limitations for events that are dealt with by procedural means at the DP-side, rather than by data mutations. The extension of evidence type definitions beyond what is actually required by the eProcedure is, again, not optimal from a data minimization principle perspective.
- Extended Notification Message: The pure Event Notification could be extended to carry an updated evidence as part of the Notification message, essentially piggy-bagging the evidence on an event notification. This approach would need to extend the event-centric subscription system to maintain with underlying evidence type, or potentially multiple evidence types, are potentially relevant for each event type, so that evidence can be retrieved and packed as extension of the Notification Message. This solution would consequently end up to be rather complext and again not optimal from a from a data minimization principle perspective. [A BPMN Process Analysis of this approach is available on request]
- Event Notification + Lookup: The pure Event Notification is used as described above, while it is made sure that the agreed event set of canonical DE4A events is complete in the sense that in covers also relevant update needs, for example a company that moved HQ means that the adres (part of the Company Registration evidence) changed as an effect. It is then left at the discretion of the DC, whether a particular event requires an update of the evidence on record. It could be that, i.e. the postal adres of a company is nor relevant for the continuation of the public service provisioning, in which case a flag that the adres on record is not up to date, or a change to "unknown" would be sufficient as event response. This is fully in line with the data minimization principle perspective. If the DC considers that an an update of one evidence or potentially several evidences is required as part of the event response, then the DC would use the Lookup Pattern to request new copies of evidences.
The reference Architecture for the DE4A projects focusses on the third option, on Event Notification combined with the Lookup Pattern to cover both business requirements with one hybrid solution, rather then setting up two specific solutions next to each other. The combination of Event Notification and Lookup Pattern appears to be the most flexible approach is expected to have the following advantages:
- Data minimization: Evidences are only requested if they are actually needed for the public service provided by the DC
- Low complexity of message definitions: The required message definitions for the notifications and updates between DP and DC remains low in complexity: A simple event notification (consisting of event type, identifiers and timestamp) and and evidence response analogous to the event response message of the Intermediation Pattern and User-supported Intermediation Pattern.
- Multiplicity: Case that single events require multiple evidences to be updated, or several events require the update of the same evidence can be covered quiet easily without requiring overly complex message definitions.
- Flexibility in legal basis and authorizations: As the Event Notification does only contain identifiers and not the underlying (personal) data, the legal basis and corresponding authorizations might differ between Notification and Lookup, adding flexibility to the overall solution.
- Independence from a previously shared evidence: The solution approach is not directly linked to a previously shared evidence, which means that a subscription and subsequent lookup can also be applied in cases where the original procedure (leading to the public service being provided) was completed without any automated evidence exchange, i.e. evidence was provided by the user electronically or in person, provided that there is a legal basis (see below).
Legal Considerations
Differnce between evidence update case (possible in framework SDGR?) and Event Notifications
Existence of consent / Publicly available information
Business vs. Citizen: For Citizen anything that is not based on Consent will be difficult.
Delimitation to BRIS (DBA specific)
As concluded in “DE4A Discussion paper BRIS 20200708 Final.docx” BRIS will not be used in the DBA pilot. This also applies for the subscription and notification pattern. The notifications send resorting under BRIS regulation are out of scope of DE4A. The DE4A pattern has its own subscriptions and notifications that do no relate to BRIS regulation. Note that in some cases this could lead to the situation that a Data Evaluator receives both a BRIS and a DE4A notification for one business event.
Note the BRIS proposed change in the technical working group: Subscription will be replaced by a push notification through registering international branches in the registry of the mother company. Thereby reversing the responsibilities.
Business Process
The subscription and notification pattern realizes the goal to inform the data evaluator of relevant changes in the subject (i.e. company or citizen). This is done in two steps:
- In the subscription step the DE expresses the need to be informed on changes regarding a certain subject and this need is registered as a subscription.
- In the notification step the DO monitors the subject and if a change occurs, all subscribers will be informed on this change.
Event Subscription
Business Process
Activity Table
Activity / UC | Role | Activity Type* | Description |
Trigger: Need for a subscription idenfitied | Public Service Procedure | Process | A procedure of a public service provider (e.g.: subsidy) leads to the registration of a subject. After this registration events can occur to the subject that could have impact on the service delivery to this subject. In order to be informed on these events, the public service provider can subscribe to the life-events or business-events of the subject.
The subscription process can also be triggered for technical reasons: for instance to resend failed subscription requests. E.g.: In the pilot Doing Business Abroad the subject is the represented company itself or a new branch of the represented company (the parent-company of the branch) and Data Evaluators subscribe to be notified on the business events of the represented company. |
Trigger: Change subscription | eProcedure / Public service / Notification | Process | Potential triggers to change a subscription are:
|
Initiate subscription | DE | Service | To initiate subscription data is collected and the subscription need is formulated:
The subscription need is forwarded to the Data Requestor (if applicable). |
Change subscription | DE | Service | To change a subscription data is collected and the changed subscription need is formulated:
The cancellation of a subscription is thus a a change of the end date the current date. The changed subscription need is forward to the Data Requestor (if applicable). |
Lookup event provider routing information | DR | Service | The DR uses the data owner identifier to look up routing information of the competent authority that facilitates subscription service. It is possible that at the DP-side the service provider of the subscription is different from the service provider of the evidence. |
REMOVE: Inform National Contact point that subscription or change was not successful | DR | User | In case of the cancellation of a subscription there is a slight chance the participant ID of DO has changed: a manual process deals with this (unhappy) flow. |
Send subscription request | DR | Service | The request to subscribe or unsubscribe is send to the participant facilitating the subscription service using the previously retrieved routing information.
|
Exception: Forward subscription error | DR | Service | The subscription error message received from Data Transferor is forwarded to the Data Evaluator. |
Exception: Investigate reason for subscription error | DE | User | The received error message is analysed and appropriate actions are taken. This exception flow is not further detailed in this design. |
Forward confirmation | DR | Service | The confirmation of the subscription (received from the DT) is forwarded to the Data Evaluator. |
Log subscription information | DE | Service | The confirmation is logged to complete the audit trail.
Note: register in a way that it is easily readable (optional: include subscription id). |
Validate subscription request | DT | Service | The request is validated on a technical level and check on DE authorisation. If the request validates it is forwarded to the Data Owner. [Technical exceptions are out of scope.] |
Evaluate subscription request | DO | Service | The subscription request is evaluated to check if the request can be completed: Subscription functional checks: does subject exists, is event catalogue supported, is the subscription changing an existing subscription
If the request does not pass the functional checks, the request is rejected and an error message will be send. |
Exception: Prepare subscription error message | DO | Service | Collect the content of the error message and send it to the Data Transferor (if applicable).
|
Exception Send subscription error message | DT | Service | The error message is forwarded to the Data Requestor from whom the request was received. |
Register subscription | DO | Service | The Data Owner creates or changes the subscription according the subscription request. |
Confirm subscription | DO | Service | The confirmation of the subscription is created and send to the Data Transferor (if applicable).
The confirmation message contains: subscription result (success), timestamp of subscription, subscription request reference, subscription id. |
Send subscription confirmation | DT | Service | The subscription confirmation is sent to the Data Requestor from whom the request was received. The subscription confirmation message is added to the log. |
*Activity Type and Task Type in accordance with BPMN 2.0
Design decisions
Explicit request and preview
The nature of the subscription and notification pattern leads to a different use of the explicit request and preview as stated in the SDG-regulation:
- Notification is performed without a user transaction, making real-time explicit request and preview impossible.
- Fraud prevention outweighs user centricity, making explicit request and preview less opportune.
- The public nature of company data relaxes the need of explicit request and preview.
Implementing an explicit request for future notifications introduces the burden of creating and maintaining an explicit request administration, with for instance options to retract a previously given explicit request
Design decision: for the Business event notification explicit request and preview as defined in the SDG-regulation is not applicable.
Positioning of subscription registration
For the location of the subscription registration certain options can be considered:
- Fully at the data providing MS: The subscription is registered at the data providing member state. The DE sends messages to the DO via DR and DT to manage the subscription (subscribe, cancel subscription).
- Splitted between data provider and data consuming MS: It is a possibility to split the register; the DO then only registers which MS subscribe to a certain company, and the DC MS registers which DE subscribe to the company. The process flow would be: (1) a central component at the DT registers which DE’s subscribe to which subjects of which data owners; (2) the DT registers as a MS for this company; (3) the DO registers which MS subscribes to which company and sends notifications to the DT (4) the DT distributes the notification to all DE.
- At a central component: The DE4A-architecture implements the four-corner model, a central subscription register would conflict with this principle. Moreover, a subscription is a direct relation between a DO and a DE regarding a subject, there will be no added value to place such a subscription in an external, central component.
Design decision: The registration of subscriptions is placed within the data providing member state. DP MS is free to choose in which environment this is (DT, DO or another environment). The assumption for the design of the S&N pattern is that the subscription registration is fully placed in the environment of the DO.
Subscription period
Including a subscription period with start and end date has several consequences... todo
Design decision: a subscription period with mandatory start and end dates is included in the subscription.
Catalogue of DE4A business events
tbd, see table below
Subscription register requirements
Evidence exchange and subscription request
The main flow of the DBA-pilot is that the intermediation pattern triggers the subscription and notification pattern. Part of the intermediation pattern is the exchange of evidence: DE requests a specific evidence type from a specific company from the DO. In the happy flow the DE subscribes to receive notifications regarding this company. This trigger can be implemented in different ways:
- As a part of the request for evidence. The evidence exchange request then consists at least of: company identifier, requested evidence type, DE identifier, subscribe y/n.
- In a separate subscription request message: after a user consent to the preview and a successful completion of the registration process a separate subscription message is sent.
Design decision: the subscription request is not combined with the evidence exchange request but is sent after successful registration of the company/branch.
Combining the subscription request with the evidence exchange would be too early in the process. It would lead to (possibly a relative high number of) requests to cancel the subscription if no consent was given or the registration was not successful. This would also introduce the risk of unauthorised subscriptions and notifications if the cancellation fails.
There are two main choices for the subscription criteria, subscribe to changes in:
- business events
- evidences
Design decision: not all business events are related to an evidence, subscriptions must therefore be made to business events to cover all notifications to be sent, see table below.
Business event - evidence mapping
Below example for Doing Business Abroad Pilot. Each pilot (domain) must define their common event list including a clear semantic explanation of each business or life event in order for the Data Evaluator to be able to interpret the event correctly and define an appropriate course of action, e.g. request an updated evidence or discontinue the public service.
DE4A Business event | Event includes changes in | Related evidence type | Comment |
---|---|---|---|
Company info event | EUID, Updated company name/updated address/updated head quarters/changed identifier (EUID)/changed legal form/changed representatives | Company registration | Split this event in: address change, name change, ...? And with a status change, notify the specific status? |
Annual report event | Annual reports | Annual report | What is the event? New annual report available, annual report changed, ...? Split this event? |
Directors disqualification event | Disqualifications updated for directors | Directors disqualification | |
National merger event | Merger initiated/completed/cancelled, transferring company identifier, aquring company identifier | Notifications are only relevant when the transferring company has subscribers. | |
Cross border merger event | Merger initiated/completed/cancelled, transferrring company identifier, aquaring company identifier | Notifications are relevant when the transferring company has subscribers.
This event is also relevant without a subscription. The business register of a company involved in a merger wants notifications even if there is no prior subscription to the foreign company in the cross border merger. |
Event Notification
Business Process
Activity Table
Activity / UC | Role | Type | Description |
Identify cross-border event | DO | Service | The Data Owner evaluates all events identified in the register and identifies events that are potentially relevant for cross-border notifications.
Note that the DO must have a mapping of it's own business events to the list of DE4A business events. |
Check subscription | DO | Service | The subscription register is queried for subscribers to the subject related to the event:
|
Prepare notification message and subscriber list | DO | Service | Make a list of the subscribers to notify in terms of cross-border participant identifiers and create the payload of the notification, mainly the DE4A event name and subject identifer and the timestamp the event took place. |
Resolve service metadata | DT | Service | Single notifications messages (one for each subscriber) are created from the list of subscribers and the notification payload. The routing information for each notification message is looked up.
The messages contain: subject identifier, secondary subject identifier (in case the identifier changed), subscriber identification, event identification, event timestamp, subscription reference (optional). |
Resolve subscriber participant ID and inform National Coordinator | DT | User | If address / DE participant id is not found in the metadata, manual action is needed: contact DE / MS of participant id to analyse the exception and take appropriate measures. |
Request from DE to resend event notifications | DE | User | Exception flow for missed notifications (failed or non-failed): if a DE misses notifications a resend of notifications can be requested. |
Resend past event notifications | DT | User | The resending of previously sent notifications requires a manual action at the DT, based on logs. |
Send event notification | DT | Service | The event notification is send to the Data Requestor, a technical acknowledgement that the notification has been received by the DR is received. The audit log is updated with both the event notification and the acknowledgement. |
Validate event notification | DR | Service | The Data Requestor performs a technical validation of the event notification and forwards it to the Data Evaluator. A technical exception flow is out of scope of this process description. |
Determine event response | DE | Service | The notified event is analysed and the appropriate response is determined.
Depenting on the event, different courses of action are possible (see details below):
The determination result is logged as a part of the audit trail:
|
Request change of subscription | DE | Subprocess | When the company cannot be identified or the registered company or branch is no longer active a change (i.e. cancellation) of the subscription is requested. Afterwards this will be analysed to find the cause of this apparently wrong subscription and to take appropriate measures. This subprocess includes user tasks ad is not end-to-end automated. |
Dismiss event | DE | Service | The notified event doesn't have impact on the procedures of the Data Evaluator and is dismissed. No further actions need to be taken. |
Trigger evidence lookup | DE | Service | The lookup pattern is triggered to request a new, current, copy of the relevant evidence.
The Doing Business Abroad Pilot will e.g.: request the Company Registration Evidence. |
Identify business responses and notify responsible party | DE | Subprocess | The responsible organisational unit is identified and informed in order to take appropriate action. It depends on the specific process if this action can be performed automated or manually. |
Design decisions
Response on event notification
Functional responses from Data Evaluator to Data Owner after receiving an event notification, for example to inform that appropriate actions have been taken, are out of scope of the Subscription and Notification pattern.
Notifications from Subscriber
Notifications from Data Evaluator to Data Owner, for example to inform Data Owner on changes in the branch registration, are out of scope of the Subscription and Notification pattern.
Process Realisation
Used by the following use cases
Application collaborations
TBD