Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Name: MTS 1.0
Date: 6th October, 2022
Name: MTS 1.0.1
Date: 9th November, 2022
MOSIP Token Seeder (MTS) is a standalone service that outputs MOSIP Token ID for a given input list of UIN/VIDs after performing authentication with IDA. The service is a convenience module that makes it easy for Relying parties to perform bulk authentication to onboard users to their systems. Refer use cases section for details on the usage of MTS.
Bulk upload
Support for multiple delivery types
REST interface
PII at rest is encrypted. Further, the PII is erased after processing
Works in asynchronous mode - queues all the requests
Processes multiple records per request
Processes multiple requests simultaneously
Enables configurations:
Output formatting of fields
Setting up fields as mandatory/optional
Defining data clean-up policy
One of the intended use cases of MTS is to seed existing beneficiary registries for deduplication.
ODK based
Google Sheets (TBD)
Form.IO Sheets (TBD)
Verifiable Credentials (VC) (TBD)
CSV
JSON
Download
Synchronous response
WebSub (TBD)
SFTP (TBD)
Process multiple records per request
MTS is capable of processing millions of records per request. There is no specific limitation to the number of records it can handle per request. Refer here for details.
Process multiple requests simultaneously
MTS processes multiple requests in a simultaneous manner, rather than a sequential pattern. Refer here for details.
Output formatting of fields
When MTS receives an authentication and tokenization request, it processes the request and sends out the output, in the requested format. Additionally, whilst formulating the output response, MTS is capable of sending the response back, based on the preferred field mapping. e.g. A request received may carry fields First name and Last name. However, the requesting party may need the response to carry fields as per the mapped naming convention: First name to FN and Last name to LN. MTS enables this requirement, through configuration of the output format template which provides the flexibility to define the field mapping as preferred. Refer here for details.
Setting up fields as mandatory/optional
Every request that MTS receives comprises of a set of fields. However, MTS provides the flexibility of defining which field is mandatory/optional as part of the request. This is a one time activity that will have to be carried out at the time of initial installation setup of MTS. Based on this definition, MTS validates each request for the presence of mandatory fields. If this config file is not setup distinctly, then the default IDA setup will be considered. Refer here for details. Refer <installation guide> for setup related information.
Defining data clean-up policy
For each request received and processed, the data is held in MTS. Whilst the data is held in-memory, MTS provides a feature to clean-up the data held and also define the timeframe based on which the data clean-up job may be run. This will help control the volume of data stored and also limit the availability of data for potential security threats. The process of defining the policy is a one time activity that will have to be carried out at the time of initial installation setup of MTS. Refer here for details. Refer <installation guide> for setup related information.
Authtoken API is a RESTful interface to accept various auth request input for the Token Seeder system. The API works in a complete asynchronous mode. Relying party is returned a request identifier when they make successful authtoken request. Status check API can be used to poll the status of the request placed. In case the status returns a processed state, the output can be accessed, as configured in the primary request for. Eg. If the request was for a file download, the file download API can be called to return the output file. Refer MOSIP Token Seeder API for a detailed API documentation.
Token seeder is a batch processing module which initiates the token authentication process. Once a new request is enqueued into the token seeder, it fetches the same and sends the request on a record level to the authenticator module. Token seeder is also responsible for updating the success and failure status to the database. There is also a expiry program for clearing off the request already processed from the system based on the expiry settings configured.
Authenticator process takes in a valid authentication request and performs the demographic authentication with the MOSIP IDA server. Each auth request is well formed, encrypted and signed before its sent to the MOSIP IDA. It passes on the response received to the caller regardless of the status received. Authenticator module can also be used as a individual library outside of MOSIP token seeder for any use case it applies to.
Refer API
Refer GitHub
Refer Test cases
Refer README
OpenG2P-registry MTS Connector (OMC) is a OpenG2P module which will be an addon to Odoo. OMC can generate a MOSIP Auth Token against the any record in OpenG2P registry by calling MOSIP Token Seeder (MTS) and store the same in beneficiary registry. This will be an important module in deduplication process when OpenG2P system uses MOSIP as its ID platform.
Generates MOSIP token against the OpenG2P registry by calling MTS.
Uses callback
delivery type of MTS
Completely asynchronous execution
OpenG2P can schedule a daily job to fetch the delta for the day
A manual import feature will also be provided
Removes VID if MOSIP Auth Token is generated
In OpenG2P, the user can configure for following fields to setup an interface with MTS through OMC.
Name: A string to identify the connector
URL to reach MTS: URL for MTS API
MTS Input type: OMC option could be proceeded by selecting "OpenG2P Registry".
Mapping: MTS Field mapping as required by the API. Please refer MTS Documentation. Format of Mapping would be JSON.
Output Type: MTS-C only supports JSON output type of MTS.
Output Format: Output format is a JQ string which will be used by MTS to format its output to suite the caller's requirement.
Delivery Type: Currently supporting only "Callback". Callback feature can be used to make MTS do a submission of results onto an API within Odoo. The output formatting will help in making the desired input for the api.
Job Type: MTS-C provide both recurring and one time execution. Recurring can be configured to do continuous pull from the ODK over MTS.
MOSIP Language: Mosip language setup. Default is "eng".
Interval in minutes: Interval at which the MTS-C job runs.
Filters to apply to Registry: A domain filter can be used to identify the records for tokenization. For. eg. Only records which have VID associated with it and is not tokenized need to be picked for tokenization.
List of fields to be used: List of fields which will be supplied as auth data. This field list may be a superset of fields required for auth as it may contain data required by the callback API. This list should be a valid JSON string array.
Callback URL: A URL end point which would be called upon successful processing at MTS
Callback HTTP Method: HTTP Method (POST/PUT/GET/PATCH) used while MTS makes the callback
Callback Timeout: Timeout awaited by the callback until acknowledged with a response.
Callback Auth Type: Type of authentication expected by callback URL. MTS-C currently support Odoo type which uses the session-based authentication implemented by Odoo.
Callback Auth Database: DB instance used by Odoo.
Callback auth username: Username to access callback api
Callback auth password: Password to access callback api
This page details only on very specific areas of MOSIP Token Seeder. For a elaborate understanding on the MTS API, please refer the API documentation page.
Refer .
When using above format, you may not need any mapping configuration. But in case you change any column name in the csv, please do provide the same in the mapping configuration.
Output might have mix of successful and failed records except for the case where whole of the input throws error. If successful, the record would be having the token placed against the vid. And if there is error processing a record, the same is updated against the vid and the and description is mentioned along.
submitted
submitted
is the first status immediately after you have placed a authtoken request.
invalid
If in case there is basic validation error such that the request could not be processed, the request in marked as invalid
.
submitted_with_errors
Once the system passes through the basic validation but has found that none of the records can go through due to varied record level validation issues, the system will update the request with submitted_with_errors
status.
processing
This status is updated when the seeder enqueues the request for processing
.
processed
Once the request is completed processing and ready for a result, the system updates the record with processed
status. There can be error codes mentioned in the output line items in case few but not entire records have any processing error.
processed_with_errors
When the request is processed but every record in the request has some or other processing errors, the system updates the request with processed_with_errors
status.
In case there is a prior request placed with considerably higher number of records, and you have placed subsequent request submitted even before getting output for your earlier request, the system might take a while to update the status of your newer request. It might be still in the submitted
state until the system finds a window to start processing.
Call back functionality enables MTS api to submit the output to a specified URI. Callback might also need authorization on most occasions. MTS currently supports following authorization types.
If the caller can supply a token while calling the MTS api, it can be configured in the authStaticBearer
attribute of callbackProperties
section.
The functionality is added to support the OpenG2P use case where in OpenG2P odoo modules can call the MTS and get the result directly imported to odoo database. Following are the parameters supported.
There are cases where MTS might successfully pass on the request but IDA generates error based on the its implementation scenario. MTS will log such error directly to the output json/csv/file. In any case there are uncaught errors thrown by IDA, MTS will output the same as unknown error (ATS-REQ-100).
The names on the left side of the mapping config denotes the original expected names and the value part will hold the change of name, if any. The mapping config if not supplied, the system will assume that there are no changes in the names used. The same applies if any one or more element of the mapping is not provided.
Except for name and full address, the majority of the fields in authdata are direct mapping. MTS expects that there can be exceptions for name and full address for which the mapping is configured as a string array. For example, if the calling application or program stores the address fields in separate variables or columns like addressline1, addressline2, street, area, or zipcode; the same can be supplied directly as authdata with mapping supplied as a list of variable or column names as in the calling program.
**input
**keyword in the output format string is used to represent the data you input into MTS as authdata parameter. output
keyword is used to identify that you intend to pick the data from the default auth request result. Please find below a sample for the default auth request output.
This page details only on very specific areas of MOSIP Token Seeder. For a elaborate understanding on the MTS API, please refer the API documentation page.
Refer .
When using above format, you may not need any mapping configuration. But in case you change any column name in the csv, please do provide the same in the mapping configuration.
Output might have mix of successful and failed records except for the case where whole of the input throws error. If successful, the record would be having the token placed against the vid. And if there is error processing a record, the same is updated against the vid and the and description is mentioned along.
submitted
submitted
is the first status immediately after you have placed a authtoken request.
invalid
If in case there is basic validation error such that the request could not be processed, the request in marked as invalid
.
submitted_with_errors
Once the system passes through the basic validation but has found that none of the records can go through due to varied record level validation issues, the system will update the request with submitted_with_errors
status.
processing
This status is updated when the seeder enqueues the request for processing
.
processed
Once the request is completed processing and ready for a result, the system updates the record with processed
status. There can be error codes mentioned in the output line items in case few but not entire records have any processing error.
processed_with_errors
When the request is processed but every record in the request has some or other processing errors, the system updates the request with processed_with_errors
status.
In case there is a prior request placed with considerably higher number of records, and you have placed subsequent request submitted even before getting output for your earlier request, the system might take a while to update the status of your newer request. It might be still in the submitted
state until the system finds a window to start processing.
There are cases where MTS might successfully pass on the request but IDA generates error based on the its implementation scenario. MTS will log such error directly to the output json/csv/file. In any case there are uncaught errors thrown by IDA, MTS will output the same as unknown error (ATS-REQ-100).
The names on the left side of the mapping config denotes the original expected names and the value part will hold the change of name, if any. The mapping config if not supplied, the system will assume that there are no changes in the names used. The same applies if any one or more element of the mapping is not provided.
Except for name and full address, the majority of the fields in authdata are direct mapping. MTS expects that there can be exceptions for name and full address for which the mapping is configured as a string array. For example, if the calling application or program stores the address fields in separate variables or columns like addressline1, addressline2, street, area, or zipcode; the same can be supplied directly as authdata with mapping supplied as a list of variable or column names as in the calling program.
MTS Connector (MTS-C) is a OpenG2P module which will be an addon to . MOI will help in fetching the tokenized data from the ODK Central by calling the (MTS) and store the same in beneficiary registry. This will be an important module in deduplication process when OpenG2P system is using MOSIP as its id platform.
Generates MOSIP token while fetching from the ODK
Uses callback delivery type of MTS
Completely asynchronous execution
OpenG2P can schedule a daily job to fetch the delta for the day
A manual import feature will also be provided
In OpenG2P, the user can configure for following fields to setup an interface with MTS.
Name: A string to identify the connector
URL to reach MTS: URL for MTS API
MTS Input type: MTS-C connects over "ODK" which is the first option in this selection. OMC option could be proceeded by selecting "OpenG2P Registry".
Output Type: MTS-C only supports JSON output type of MTS.
Delivery Type: Currently supporting only "Callback". Callback feature can be used to make MTS do a submission of results onto an API within Odoo. The output formatting will help in making the desired input for the api.
Job Type: MTS-C provide both recurring and one time execution. Recurring can be configured to do continuous pull from the ODK over MTS.
Interval in minutes: Interval at which the MTS-C job runs.
MOSIP Language: Mosip language setup. Default is "eng".
ODK Base URL: Base URL or the complete domain address for the ODK central installation
ODK Odata url : OData service (.svc) URL for the ODK form to fetch the submissions.
ODK User email: Email Id to authenticate MTS for accessing Odata URL
ODK User password: Password used to authenticate Odata URL
Callback URL: A URL end point which would be called upon successful processing at MTS
Callback HTTP Method: HTTP Method (POST/PUT/GET/PATCH) used while MTS makes the callback
Callback Timeout: Timeout awaited by the callback until acknowledged with a response.
Callback Auth Type: Type of authentication expected by callback url. MTS-C currently support Odoo type which uses the session-based authentication implemented by Odoo.
Callback Auth Database: DB instance used by Odoo.
Callback auth username: Username to access callback api
Callback auth password: Password to access callback api
protocol is also supported in MTS callback functionality. This helps numerous systems which implements OAuth specification to integrate with MTS seamlessly. Below are the configurations available to setup OAuth based callback.
With the output formatting capability, MTS can give you the result exactly the way in which you would want it ready for your further processing of data. The output format string to be supplied follows the . In case the output format is not supplied, the default output will be generated which will be the same as the previous versions. The below section details on an example case for output formatting.
Mapping: MTS Field mapping as required by the API. Please refer Format of Mapping would be JSON.
Output Format: Output format is a string which will be used by MTS to format its output to suite the caller's requirement.