Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
In the context of MOSIP, identifiers are alphanumeric digital handles for identities in the system. While a person's identity is represented as a collection of biographic and biometric attributes that can uniquely identify the person, the identity is referred to using identifiers.
Unique Identification Number (UIN), as the name suggests, is a unique number assigned to a resident. UIN never changes and is non-revocable. UIN is randomized such that one should not be able to derive any Personal Identifiable Information (PII) from the number itself.
The rules that govern generation of a UIN are listed here.
The VID / Virtual ID is an alias identifier that can be used for authentication transactions. The key characteristic of such an identifier is that it expires. This allows for free disclosure and usage of this identifier in various contexts. It is privacy friendly in a way such that it can be revoked, configured for one time usage and is not linkable. Since these are used for authentication transactions, such identifiers are to be known to the user only or generated with their participation.
The Application ID (AID) refers to the unique identifier given to a resident during any ID lifecycle event, such as ID Issuance, ID Update, or Lost ID retrieval, at the registration center. It serves as a distinguishing factor for each specific event and can later be utilized by the resident to check the progress or status of the event. Previously, in MOSIP, the Application ID was referred to as the RID (Registration ID) and/or PRID (Pre-registration ID).
The PRID is a specialized RID used in the pre-registration system.
The Token identifier/PSUT is a system provided customer reference number for relying parties to uniquely identify the users in their system. The token identifier is an alias meant for the partner/relying party typically unique (Configured through PMS policy, in case uniqueness is not the need then partner policy can be set to provide random number) to them. This identifier is included in the response of the authentication transactions. One key differentiator is that the PSUT is not accepted as an identifier for authentication transactions. This ensures that the partner who has the PSUT can not reauthneticate.
Digital Identification is a means of identifying 'who you are' through some kind of digital medium. The digital medium could be any device such as a mobile phone or a computer or anything that has a digital means of identifying 'who you are' in both offline and online mode with your consent irrespective of your religion, caste, creed, gender, country and ethnicity. The Internet is only an enabler but is not the mandatory requirement for digital identification. While identifying 'who you are', the user bearing the identity must have absolute control of revealing a controlled set of information that they wish to reveal and not everything in order to become eligible for getting a service rendered.
Governments are exploring the development of multipurpose foundational ID systems, in which individuals receive a unique identifier from the government that they can use for identity assertion and verification. It can then be used to access a wide variety of government and private services.
Functional IDs are the IDs that are used in specific use cases. By design, they are created having the final usecase in mind such as healthcare, finance, banking, insurance, social and civil registry etc. The Functional ID systems can leverage the Foundational ID system.
MOSIP is a robust, secure, open-source platform for building foundational identity systems. MOSIP is configurable and flexible to adapt to a country’s national ID requirements.
MOSIP needs to work along with other ecosystem players to create a solution for a particular country.
Key MOSIP offerings are:
ID Lifecycle Management
ID Authentication
National ID systems can leverage MOSIP as the base platform and configure, customize, and extend it to build systems just the way needed. The image below depicts how MOSIP provides the base layer to build a national ID platform.
The Modular Open Source Identity Platform is an open-source, open standards-based foundational identity platform. MOSIP is an API-first platform that can be used by countries to build their own national ID platform. MOSIP offers ID Lifecycle Management features and identity verification capabilities out of the box. Conceived to help build global digital public goods in the space of digital identification and governance, MOSIP is anchored at the International Institute of Information Technology, Bangalore (IIIT-B). It harnesses the power of open-source technologies and embraces the best practices of scalability, security and privacy. Learn more >>
The key objectives of MOSIP are to:
Provide the basic framework to create a fully functional foundational identity system
Provide flexibility for a country to choose and customize the features from the basic framework according to their requirements
Maintain privacy, security and confidentiality of an individual's data
Provide a scalable and accessible solution to cater to a wide range of populations (a few thousand to several hundreds of millions.
The latest release of MOSIP, version 1.2.0 LTS is here! Check out the exciting new services and enhancements in the documentation.
To learn more, see our Releases Notes.
To read through the previous version of the documentation, see the Older version documentation (1.1.5).
Source Code: GitHub Repositories
Containers: Docker Repository
Maven Repository: Nexus Repository
Presentations: mosip.io
Training: MOSIP Academy
Learning videos: YouTube Channel
Community: MOSIP Community on Discourse
When a country is implementing and running the ID program, people at the forefront, like policymakers and other executives, will need to monitor the progress. Progress can be measured with the help of various attributes, like:
total enrollment count
gender profile for enrollment
age group profile
enrollment numbers and profiles per registration centre
quality of biometric data captured
Information like this will allow policymakers to take corrective measures as and when required.
Some examples are given below:
Example 1: If registration centres are setup for enrolling residents and if they see that the number of women enrolling in a particular area is less because of cultural factors, they can introduce a door-to-door enrollment process to increase coverage.
Example 2: The quality of biometrics captured for a particular registration centre or region can be monitored. And if it is found to be unacceptable, they can proceed to replace the biometric devices in that centre.
Example 3: They can compare the total number of enrollments against the total number of UINs issued. If there is a big gap, they can then address this by increasing the capacity of the registration processor module to handle and process more packets.
In order to achieve this, we have published a fixed anonymized profile of the users and ensured the same is accessible to a search engine such as elastic search so that it can be used for analytics. The basic guideline followed to create these profiles is that the limited dataset should not violate the privacy of the person or point to specific individuals. These JSON profiles are not configurable in the current design and a code change is required to change them.
This dataset is called an anonymous profile and is captured at various stages in the ID lifecycle like pre-registration, registration processing, ID issuance and authentication.
As a part of this implementation, a new anonymous_profile table is created in each of these modules and is populated as per the JSON structure given below for each profile.
Note: New DB tables are added for the anonymous profile because data in existing tables (except the pre-registration module) are encrypted and cannot be used to create reports and dashboards.
This profile will be used to capture data about enrollment. This suggests that the user has started the registration process.
This data is captured at two stages, i.e., during pre-registration and registration processing.
The same registration profile JSON is re-used to capture data in these two modules.
This profile data is captured in anonymous_profile
tables under the mosip_prereg
mosip_regprc
schemas.
We can configure the stage at which the anonymous profile data in the registration processor needs to be captured. This can be configured as a part of the registration processor camel routes in the mosip-config
repository as shown below.
The profile will be available from version 1.2.0 and above.
JSON structure of the registration profile is given below
Below is the image for the Anonymous profile table created in the Pre-registration schema
Below is the image for the Anonymous profile table created in the Registration processor schema
This profile data will be captured during the identity issuance process when an entry is made in the ID repository.
The profile data is captured in a anonymous_profile
table under the mosip_idrepo
schema.
The profile will be available from 1.1.5.5 and above.
JSON structure of the identity issuance profile is given below:
Below is the image for the Anonymous profile table created in the ID repository schema
This profile data will be captured when the resident performs authentication.
The profile data is captured in an anonymous_profile
table under the mosip_IDA
schema.
The profile will be available from 1.2.0 and above.
JSON structure of the Authentication profile is given below:
Below is the image for the Anonymous profile table created in the IDA schema
MOSIP as it stands is a modular microservice based architecture. It's modularity helps in adoption of MOSIP at complex situations. Most of the MOSIP modules are designed to be a strong foundation infrastructure modules and can be adopted in several projects.
MOSIP is designed with the following architecture principles. These architecture principles are core to the development of the system's feature and has a great influence on how and why specific software design patterns are used within.
Data Privacy
No Vendor Lock-in
Open Standards
Async/ Offline First
Commodity Computing
Fault tolerant
Manageable
Secure By Default
To know how MOSIP can be deployed, refer Getting Started. The different installation models are detailed out in the Deployment section.
The High-Level Reference Functional Architecture serves as a blueprint outlining the system's high-level functioning and interactions, providing a structured framework.
ID Schema is a standard that defines dataset that can be stored in the Identity repository for a resident. The schema allows the adopters to customize the fields that's needed for running the identity program.
Defining the ID Schema is the first step towards creating a foundational ID system. Once defined, all applications built on top of the MOSIP platform must conform to the same.
One should not confuse ID Schema with what is seen on the screen of the Registration client/ Pre-registration. ID Schema is the final data that you would like to store against each user in the final ID Repository. Quite often we collect more data than listed in ID Schema. This is essential to validate the user's claim. We should consider these data as transactional and it will never reach the final ID Repository.
This guide is intended for adopters who would customize the default ID Schema to suit the needs of a specific deployment.
Field: Unit of data collected from residents (eg. fullName
, dateOfBirth
, proofOfIdentity
etc).
Field attribute: Qualification of Field (e.g.: fieldCategory
, fieldType
, etc).
Definition: Custom data types are defined for collecting different types of data:
simpleType
: Multiple languages.
documentType
: Document metadata.
biometricType
: Biometric file metadata
bioAttributes
:
leftEye
, rightEye
, leftIndex
, leftRing
, leftLittle
, leftMiddle
, rightIndex
, rightRing
, rightMiddle
, rightLittle
, rightThumb
, leftThumb
, face
,
fieldCategory
none
: Cannot be used for any purpose. But will be stored in id.json (default subpacket). These are used in exceptional cases when we need to store some data for future reference in case of investigating an ID fraud or if law mandates the storage of such data.
pvt
: Private information, can be used in authentication. A limited data that can be used for authentication & kyc.
kyc
: Information that can be disclosed to partners either as a credential or through the ekyc API's.
evidence
: Field is treated as proof and may be subjected to removal. In certain countries law mandates deletion of such data once the Identity of the user is verified. Marking some of the fields as evidence
helps in deleting them without violating the source of truth signatures.
optional
: Field is treated as proof and will be removed after a predefined interval (defined as policy).
format
lowercased:
Value stored in lower case format
uppercased:
Value stored in upper case format
none:
No format applied to the data
validators
type
: Validation engine type. Supported type as of now is regex
.
validator
: Based on the type the actual script is placed here. In case the type
is regex
then the actual regex pattern is used here.
arguments
: Array to hold parameter or dependent field IDs required for validation.
subType
For every documentType
field, document category code
must be the value of this key. This document category code is used to validate the provided document types in the ID object.
ID schema is loaded as a part of master data to identity_schema
table in mosip_masterdata
DB.
If any changes are made to the default ID Schema, make sure the following dependencies are updated as well:
UI Specifications
ID Schema is identified based on it's version in the MOSIP system. On publishing of an ID Schema, the schema is versioned. Every ID Object stores the ID Schema version which is validated during ID Object validation.
The following is the list of good practices that MOSIP recommends for creating your ID Schema.
As a privacy by design practice, it is recommended that the number of fields is kept to a usable minimum in order to avoid profiling.
Larger number of data results in serious data quality issues.
Keeping the fields minimum ensures everyone is inclusively added to the foundational identity.
As a best guide, limit the total number of fields to be less than 10.
Stick to one version of ID Schema for better compatibility.
Identity Lifecycle Management refers to the process of issuing and managing user identities in a given system. An individual can visit a registration centre to get a new ID or update their ID details. A registration officer would typically capture the individual’s demographic (name, date of birth, gender, etc.) and biometric (face, iris, and fingerprint image) details.
The various life cycle events are briefly explained below:
New ID issuance (for adults and infants)
ID data update or update individual’s information
De-activate or re-activate the individual’s ID
Lost ID
Correction process
A resident can access the pre-registration portal and do the following:
Enter demographic data and upload supporting documents
Book an appointment for one or many users for registration by choosing a suitable registration centre and a convenient time slot
Receive appointment notifications
Reschedule, update and cancel appointments
The resident visits a registration centre.
The registration officer captures demographic details, biometrics and documents and uploads the same for processing.
Upon successful registration, a registration ID (RID) is allocated to the resident and an acknowledgement slip containing the captured details and the RID is issued (printed) to the resident as proof of registration.
The registration processor processes the registration details perform quality checks and checks for duplicate entries (de-duplication).
Upon successful processing, a Unique Identification Number (UIN) is allocated to the resident and a notification is sent to the resident on the registered phone number and/or email.
Upon a failure in processing, the registration process is discarded and a notification is sent to the resident on the registered phone number and/or email.
A child visits the registration centre along with a guardian/parent.
The registration officer captures the demographic details along with the face photo.
Note: For infants/children less than 5 years old, the Registration Client does not capture the biometrics as they are still evolving. However, the country may choose to override the same by modifying the rules accordingly.
Parent/guardian' UIN or RID and the biometrics needed for authentication are captured.
Additionally, a proof of relationship document is uploaded.
Upon successful registration, an acknowledgement receipt containing the registration ID is issued to the parent/guardian.
Residents can update their information in two ways:
By visiting the registration centre: The demographic and biometric information can be updated at the centres.
Registration receipt containing the Registration Identity(RID), labels and data in the configured language, and QR code (of the RID) provided to the resident at the centre.
Updated ID credentials were sent to the resident via the country’s configured printing and postal service.
Notifications were sent to the resident using the email ID and mobile number provided as a part of demographic data collection.
De-activate ID means an individual will not be able to authenticate themselves by using the UIN or VID.
Likewise, a country can also re-activate an individual’s ID as need be.
In the event of loss of the ID, the resident can go to the nearest registration centre and provide his/her biometrics and optionally provide demographic details. The details are sent to the MOSIP server which performs a biometric and demographic match. Upon a successful match, MOSIP provides mechanisms to retrieve the UIN of the resident which may be printed and delivered to the resident. Notifications are sent to the resident's registered email and/or phone.
In case the system finds an error in the demographic data, documents or biometric data provided by an individual, the correction flow is triggered.
Before issuing the UIN for the individual, the incorrect or incomplete information provided by the individual is intimated to them.
Once the corrected information is received by the system, a correction procedure is triggered.
Updated ID credentials are sent to the resident via the country’s configured printing and postal service.
Module | Table name | Profile name |
---|
Reports and dashboards can be created using anonymous profile data. The used for the platform can be used to push this data into elastic search and dashboards can be configured using Kibana. A dashboard created using ID Issuance Anonymous profile data is available as a part of the reference implementation. The same is shown below.
More details about reporting module and dashboards can be found .
Refer to the . A guide to customising the same is given below.
is a resident-facing web-based portal that allows a resident to provide registration data, upload document proofs and book an appointment with a registration centre to complete the rest of the registration process. This data can be accessed by the registration officers who could then complete the registration process such as collecting biometrics, verifying the documents and other formalities thus saving time and effort at the registration centre.
Registration is a process that allows a resident to provide demographic information and biometrics by visiting a registration centre. The operated by a registration officer is used to securely capture the details and send them to for processing and issuance of an ID. If a resident has pre-registered, the registration officer can retrieve the registration data by giving the pre-registration ID to the Registration Client.
Update only the demographic information using .
If a country wants to deactivate an individual’s ID for any specific reason, the system deactivates the individual’s ID after certain validations are performed in the .
Pre-registration | anonymous_profile | Anonymous Registration profile |
Registration Processor | anonymous_profile | Anonymous Registration profile |
ID Repository | anonymous_profile | Anonymous Identity Issuance profile |
Authentication | anonymous_profile | Anonymous Authentication profile |
MOSIP offers identity verification services that enable the usage of identity in various contexts. After the successful issue of the ID, identity verification services become available for the resident. Online identity verification is enabled through MOSIP's ID Authentication (IDA) module. As MOSIP is a foundational ID system, different services (both government and private) may rely on the foundational ID system to validate the identity of a resident rather than implementing multiple authentication systems. A typical authentication flow is illustrated below:
Refer to the video below to learn more!
MOSIP offers a yes/no API that can be used for the verification of attributes supplied along with authentication factors. The API verifies the identifier and the provided demographic attributes and also validates other authentication factors such as the OTP or biometrics and responds with a yes or a no. Successful verification of the data results in a yes. This kind of API can be typically used to support the verification of a limited set of demographic data about the person or for simple presence verification when biometrics are used.
MOSIP additionally offers a KYC API, which can be used to get an authorized set of attributes for the resident in the response of the API. This API is intended for use by authorized relying parties to perform KYC requests. The authentication includes an identifier along with authentication factors such as OTP and biometrics. The information returned is governed by a policy. Different relying parties can be provided with different KYC data based on their needs. The policy helps implement selective disclosure as part of the KYC data. The data thus returned is digitally signed by the server and can be used by the relying party with confidence.
The authentication APIs support multiple factors. These can be:
Biometric: Finger, face, iris
Demographic: Name, date of birth, age, gender etc.
OTP: One-Time Password Based on the level of assurance needed for the transaction, the relying party can decide which factors are sufficient for identity verification.
Biometric authentication is performed using third-party matcher SDK that performs 1:1 matches on a given modality. Each biometric modality is treated as an independent factor in authentication.
All authentications in MOSIP essentially perform 1:1 matches. This essentially requires the authentication call to specify an identifier of the person and the authentication factors to perform the match on the individual referred to by the identifier. MOSIP supports the usage of multiple identifiers for a person. This promotes anonymity and also acts as a deterrent to profiling the individual. The individual can be identified in the authentication transaction by the UIN or other alias identifiers/ VIDs. When a VID is used there are additional checks to see if the VID has not expired or has not been revoked. The expiry of the VID is governed by the policy chosen at the time of its creation of the VID.
For understanding VIDs and their characteristics, read more about VID.
In one-off usage contexts, identity verification can be anonymous. In cases where identity verification is linked to transactions that need identity assurance, there is a need to tie the user identity of the user to the transaction. This is achieved by way of providing such relying parties with a "sticky" token identifier that can be used as a reference ID for the individual in their system. The authentication APIs return a token when the authentication is successful. Based on the type of relying party and their policy, the token is either random or sticky. The expected usage is for the relying party to remember the token returned by the authentication API for referring to their customer and for audit/redressal purposes.
Learn more about the Token ID.
Read more about parties and policies.
MOSIP has a provision for specifying the user consent associated with an authentication transaction. This can be stored for audit purposes and the authentication flow can be extended to verify the consent if needed.
MOSIP has a provision to help protect against the misuse of identity. For any report of lost identity or detection of fraudulent activity by fraud, the module can require the temporary suspension of authentication activities on a user. This is enabled by the hotlisting feature. The authentication service checks if the identifier used is hotlisted and if so, the authentication process is aborted and fails. The hotlisting service can be used by helpdesk and fraud solutions to list and delist the identifiers that need to be blocked temporarily.
MOSIP's fundamental architecture and design incorporate the highest levels of privacy and security.
Key security features:
Encryption of data in-flight or rest. (See )
Integration with trusted applications only.
Fraud avoidance - association of authentication only with specific transactions.
Misuse prevention - user can lock or unlock their authentication.
Virtual ID and Tokens to prevent identity theft.
All data sent out of MOSIP will be digitally signed.
All incoming data will be signed by the respective entity.
Any data sent to a relying party will be encrypted.
Protection against internal attacks with every record in DB protected with integrity.
Centralized key management.
All API's are protected with OAUTH 2.0.
Key privacy features:
Minimal data with selective disclosure on a need-to-know basis.
Sensitive data protected (not stored or logged in clear form).
Consent support – the user decides who can receive what credentials & what attributes.
No search on the database (You can find a record only if you know the ID).
Clear segregation of Biometric & Demographic data.
De-centralised ID usage and data (cannot profile based on usage).
Users are not limited to one permenant ID - Virtual ID.
All relying party gets a privacy enabled tokens to prevent profiling across transactions. Permenant ID is never shared.
Supports Wallet based decentralized ID issuance and usage.
Face data is not sent to ABIS for deduplication.
Providing a unique identity for a resident is one of the key features of any identity platform. To achieve this, MOSIP interfaces with an Automated Biometric Identification System (ABIS) to perform the de-duplication of a resident's biometric data.
The ABIS system never learns about residents' identities. Any Personally Identifiable Information (PII), such as demographic details or an applicant's AID (application ID), is not shared with the ABIS system. Internally, MOSIP maintains a mapping between the ABIS-specific reference ID and the AID of the resident.
ABIS is used for 1:N deduplication. For 1:1 authentication, is used. MOSIP does not recommend using an ABIS for 1:1 authentication.
ABIS must support the following types of biometric images:
Individual fingerprint images (segmented)
Iris images (left, right)
Face image
It is recommended that ABIS be deployed in the same secure zone (military zone) where the registration processor is deployed.
ABIS system is not recommended to connect to any external network.
The right to privacy is a fundamental right in many contexts. Privacy protection or preservation can be ensured in an application by adopting a privacy friendly design stance.
Privacy takes many forms. From an identity system perspective, the confidentiality of identity information and anonymity when using the identity offers privacy.
MOSIP views the identity system as a custodian of the individual's data. This data has to be protected in order to protect the individual from privacy and security risks. Privacy protection measures include , transparency, user control, confidentiality, selective disclosure, user anonymity and intrusion protection.
MOSIP addresses privacy design at four levels.
Functional privacy
Selective disclosure
Anonymization
Need to know
Encryption
Tokenization
Security
Trusted applications
Access control
User centricity
User control
Consent
Usability
Inclusion
Transparency
Openness
Verifiability
Governance
The security of user data is given the highest priority in MOSIP. Data is protected in flight and rest using strong cryptographic techniques. All operations on decrypted data are done in memory.
Various flows with encryption are illustrated below. Refer to for all references of the type 'Kx' and 'KPx'.
The UINs are hashed, encrypted and stored in uin
the table of mosip_idrepo
DB. (K7.4)
Biometrics are shared and encrypted with the ABIS partner's key (PK1).
Registration processor stores encrypted demographic data in mosip_regprc
DB. (K11)
Data shared with all partners like ABIS, Print, Adjudication, IDA etc. is encrypted using partners' public key. Note that IDA is also a partner, however, a special partner in the sense that data is additionally zero-knowledge encrypted before sending to IDA (see the section below).
Generate master symmetric encryption key K9.
Generate a 10,000 symmetric keys pool (ZKn). Encrypt each ZKn with K9 and store it in DB. (K12)
Randomly select one key from ZKn, and decrypt using K9.
Derive new key ZKn' = ZKn + UIN/VID/APPID.
Encrypt biometric templates and demographics.
BIO = encrypt(bio/demo with ZKn').
Encrypt ZKn (this is done to share ZKn with IDA).
ZKn-IDA = encrypt(ZKn with K22)
Share the following with IDA:
ZKn-IDA
BIO
Random index (0 - 9999)
SHA-256 hash of UIN/VID/APPID
Generate master symmetric encryption key K18.
Decrypt data in Step 8 above using PK8.
Decrypt ZKn-IDA with K22 to get ZKn.
Encrypt ZKn with K18 and store it at a random index.
Bio-data is stored as is.
The authentication client further encrypts the auth request with IDA-PARTNER public key.
MOSIP interacts with ABIS only via message queues. The JSON format is used for all control messages in the queue. MOSIP sends requests to the inbound queue address and receives responses from the outbound queue address. ABIS must comply with the interface defined in
The interface may be tested using the .
Biometrics data in MOSIP is exchanged as per formats defined in .
MOSIP provides kits to test the interface. Refer to the
ABIS must comply with .
The queues can be configured in the file. The ABIS system connects to the queues using a pre-defined user ID and password.
These design principles have resulted in features as well as development practices in MOSIP that enhance privacy protection. A typical example for a practice is how PII (Personally Identifiable Information) is dealt with when creating application or audit logs. An example of a feature is how our allow selective sharing of information.
are signed by the private key of the device provider (PK2). The signature is verified by the Registration Client.
signs the packet using the TPM key of the machine (K10) and encrypts the packet using MOSIP public key specific to (the registration centre, machine id) combination (K11).
stores packets created in (2) "as is" in .
encrypts biometrics, demographics and documents and stores them in Object Store. (K7.1,K7.2,K7.3)
The module (IDA) is an independent module and may be hosted by several providers. IDA hosts all the biometric templates and demographic data. Unique additional protection is provided here to make sure that mass decryption of user data is very difficult to achieve. The data can only be decrypted if the user's UIN is provided. Here is the encryption scheme:
Share data in step 7 via standard (which encrypts entire data with PK8).
L1 devices contain to encrypt (DE1, K21) and sign (FK1) biometrics at the source and send them to the authentication client.
IDA decrypts zero-knowledge data as given in and then performs a demographic and/or biometric authentication.
The match result is returned to Auth client. In the case of KYC, the KYC attributes are encrypted with the Partner's public key (as in ).
The Foundational Trust Module (FTM) is created using a secure microprocessor capable of performing all required biometric processing and secure storage of keys. The foundational device trust would satisfy the below requirements.
The module has the ability to securely generate, store, and process cryptographic keys.
Generation of asymmetric and symmetric keys with TRNG.
The module has the ability to protect keys from extraction.
The module has to protect the keys from physical tampering, temperature, frequency, and voltage-related attacks.
The module could withstand Hardware cloning.
The module could withstand probing attacks
The module provides memory segregation for cryptographic operations and protection against buffer overflow attacks
The module provides the ability to withstand cryptographic side-channel attacks like Differential Power analysis attacks, and timing attacks.
CAVP validated the implementation of the cryptographic algorithm.
The module has the ability to perform a cryptographically valid, secure boot.
The module has the ability to run trusted applications.
The foundational device trust derived from this module is used to enable trust-based computing for biometric capture. The foundational device trust module provides a trusted execution environment based on the following:
Secure Boot
Ability to cryptographically verify code before execution.
Ability to check for integrity violations of the module or device.
Halt upon failure.
The ability to securely upgrade and perform forward-only upgrades to thwart downgrade attacks.
SHA256 hash equivalent or above should be used for all hashing requirements
All root of trust is provisioned upon first boot or before.
All upgrades would be considered successful only after a successful boot with proper hash and signature verification.
The boot should fail upon hash or signature failures and never operate in an intermediary state.
A maximum of 10 failed attempts should lock the upgrade process and brick the device. However, chip manufacturers can decide to be less than 10.
Secure application
Ability to run applications that are trusted.
Protect against the downgrading of applications.
Isolated memory to support cryptographic operations.
All trust is anchored during the first boot and not modifiable.
The FTM should have at least one of the following certifications in each category to meet the given requirement.
Category: Cryptographic Algorithm Implementation
CAVP (RSA, AES, SHA256, TRNG (DRBGVS), ECC)
The supported algorithm and curves are listed here
Category: FTM Chip
(ONE of the following certifications)
FIPS 140-2 L3 or above
PCI PTS 5 or above (Pre-certified)
PCI - PED 2.0 or above (Pre-Certified)
One of the following Common Criteria (CC) certification
https://www.commoncriteriaportal.org/files/ppfiles/pp0035a.pdf
https://www.commoncriteriaportal.org/files/ppfiles/pp0084a_pdf.pdf
System/Device Level Tamper (optional)
System/Device Level Tamper Responsiveness is recommended (not mandatory). In this case, FTM should be capable of showcasing Tamper Responsiveness (keys must be erased) against a tamper at the system/device level.
The FTM should protect against the following threats.
Hardware cloning attacks - Ability to protect against attacks that could result in a duplicate with keys.
Hardware Tamper attacks
Physical tamper - No way to physically tamper and obtain its secrets.
Voltage & frequency related attacks - Should shield against voltage leaks and should prevent low voltage. The FTM should always be in either the state operational normally or inoperable. The FTM should never be operable when its input voltages are not met.
Temperature attacks on the crypto block - Low or High the FTM are expected to operate or reach an inoperable state. No state in between.
Differential Power Analysis attack.
Probing attacks - FTM should protect its surface area against any probe-related attacks.
Segregation of memory for execution of cryptographic operation (crypto block should be protected from buffer overflow type attacks).
Vulnerability of the cryptographic algorithm implementation.
Attacks against secure boot & secure upgrade.
TEE/Secure processor OS attack.
Upon the FTM provider's approval by the MOSIP adopters, the FTM provider would submit a self-signed public certificate to the adopter. Let us call this the FTM root. The adopter would use this certificate to seed their device's trust database. The FTM root and their key pairs should be generated and stored in FIPS 140-2 Level 3 or more compliant devices with no possible mechanism to extract the keys. The foundational module upon its first boot is expected to generate a random asymmetric key pair and provide the public part of the key to obtain a valid certificate. The FTM provider would validate to ensure that the chip is unique and would issue a certificate with the issuer set to FTM root. The entire certificate issuance would be in a secured provisioning facility. Auditable upon notice by the adopters or its approved auditors. The certificate issued to the module will have a defined validity period as per the MOSIP certificate policy document defined by the MOSIP adopters. This certificate and private key within the FTM chip are expected to be in its permanent memory.
The validity of the chip certificate can not exceed 20 years from the date of manufacturing.
The FTM should have at least one of the following certifications in each category to meet the given requirement.
Secure provisioning applies to both the FTM and the Device providers.
The devices and FTM should have a mechanism to protect against fraudulent attempts to create or replicate.
The device and FTM trust should be programmed in a secure facility which is certified by the respective MOSIP adopters.
The organization should have a mechanism to segregate the FTMs and Devices built for MOSIP using a cryptographically valid and repeatable process.
All debug options within the FTM or device should be disabled permanently
All key creations needed for provisioning should happen automatically using FIPS 140-2 Level 3 or higher devices. No individual or group or organization should have a mechanism to influence this behaviour.
Before the devices/FTM leaves the secure provisioning facility all the necessary trust should be established and should not be re-programmable.
As there is no adopter-specific information being exchanged at the management server or the FTM provisioning server, there are no mandates from MOSIP where these are located globally. However, the adopter is recommended to have an audit and contractual mechanisms to validate the compliance of these components at any point in time.
Registration Client.
Backend quality check.
Biometric authentication during onboarding (internal auth).
ID Authentications.
The library is used by Registration Client to perform 1:N match, segmentation, extraction etc. For more information on integration with Registration Client, refer to Registration Client Biometric SDK Integration Guide.
A simulation of this library is available as Mock BioSDK. The same is installed in the MOSIP sandbox.
For 1:1 match and quality check of biometrics at the MOSIP backend, the BioSDK must be running as a service that can be accessed by Registration Processor and IDA Internal Services. The service exposes REST APIs specified here.
A simulation (mock) service has been provided. The mock service loads mock BioSDK internally on the startup and exposes the endpoints to perform 1:N match, segmentation, extraction as per IBioAPI.
The service may be packaged as a docker running inside MOSIP Kubernetes cluster or running separately on a server. It is important that the scalability of this service is taken care depending on the load on the system, i.e., the rate of enrolment and ID authentication.
BioSDK library: IBioAPI.
BioSDK service: TBD.
BioSDK server request/response may be tested using BioSDK testing kit.
The following properties in application-default.properties
needs to be updated to integrate the BioSDK library and service with MOSIP.
Biometric devices capture individuals' biometric data (fingerprint, iris scan, photo) and send it to a registration client or authentication client (app). The functional architecture of the various entities involved is shown below.
* An adopter may choose to have different subtypes, however, the certification needs to be adhered to.
The following calculator may be used to estimate the number of devices required for a rollout.
Providers of biometric devices are Partners of MOSIP and need to be onboarded to a given deployment of MOSIP. Specifically,
Purpose | Type | Subtype* | Certification | Specification |
---|
compliance of a device may be tested using an .
Registration | Fingerprint | Slap scanner | SBI 1.0 |
Registration | Iris | Double eye scanner | SBI 1.0 |
Registration | Face | Camera | SBI 1.0 |
Authentication | Fingerprint | Single finger scanner | SBI 2.0 |
Authentication | Iris | Single eye scanner | SBI 2.0 |
Authentication | Face | Camera | SBI 2.0 |
Standards:
ISO 19785-3
OASIS patron format ISO/IEC JTC 1 SC 37 - biometrics
Patron identifier 257
Patron format identifier 11
OASIS Binary Data Block Format Identifiers for Format Type ISO/IEC JTC 1 SC 37-biometrics
Patron identifier 257
BDB patron format identifier
7 for finger image
2 for finger minutiae
8 for face image
9 for iris image
MOSIP Schema is similar to OASIS Schema, with an optional attribute called others added from the LTS version.
MOSIP's CBEFF Utils can be used to create and validate CBEFF XML data.
The admin application is a web-based application used by a privileged group of administrative personnel to manage various master data. The various resources that can be managed by an administrator are:
Center (Registration centers)
Device
Machine
User (Admin, Registration staff)
Along with resource and data management, the admin can generate master keys, check registration status, retrieve lost RIDs, and resume processing of paused packets.
Masterdata Service exposes API to perform CRUD operations on masterdata through Admin service.
Hotlist Service provides functionality to block/unblock any IDs with option of expiry. This hotlisted information will also be published to MOSIP_HOTLIST WebSub topic.
Sync Data Service can be accessed only by the privileged group of admin personnel and enables default configurations and seed data to be setup when the MOSIP platform gets initialized.
The admin module has four services:
Admin service
Kernel Masterdata service
Kernel Syncdata service
Hotlist service
The documentation here will guide you through the pre-requisites required for the developer' setup.
JDK 11
Any IDE (like Eclipse, IntelliJ IDEA)
Apache Maven (zip folder)
pgAdmin
Postman
Git/GitHub Desktop
Notepad++ (optional)
lombok.jar (file)
settings.xml
Follow the steps below to set up Admin Services on your local system:
Download lombok.jar and settings.xml.
Unzip Apache Maven and move the unzipped folder in C:\Program Files
and settings.xml
to conf
folder C:\Program Files\apache-maven-3.8.4\conf
.
Install Eclipse, open the lombok.jar
file and wait for some time until it completes the scan for Eclipse IDE and then click Install/ Update
.
Check the Eclipse installation folder C:\Users\userName\eclipse\jee-2021-12\eclipse
to see if the lombok.jar
is added. By doing this, you don't have to add the dependency of lombok
in your pom.xml
file separately as it is auto-configured by Eclipse.
Configure the JDK (Standard VM) with your Eclipse by traversing through Preferences → Java → Installed JREs
.
For the code setup, clone admin-services repository and follow the guidelines mentioned in the Code Contributions.
Open the project folder where pom.xml
is present.
Open command prompt from the same folder.
Run the command mvn clean install -Dgpg.skip=true -DskipTests=true
to build the project and wait for the build to complete successfully.
After building of a project, open Eclipse and select Import Projects → Maven → Existing Maven Projects → Next → Browse to project directory → Finish
.
After successful importing of project, update the project by right-click on Project → Maven → Update Project
.
For the environment setup, you need an external JAR that is available here with different versions. (E.g.: You can download kernel-auth-adapter.jar
and add to project Libraries → Classpath → Add External JARs → Select Downloaded JAR → Add → Apply and Close
).
Clone admin-services repository.
Any changes in the properties for Masterdata and Admin services should be done in application-local1.properties
file.
By default the Admin-services is connected to dev environment.
To run the specific service from IDE, Open IDE -> Specific service -> src/main/java/io.mosip.specific service -> Right click on the file and select run as Java Application
.
For example, to run the Admin service, open IDE -> admin-service -> src/main/java/io.mosip.admin -> Right click on AdminBootApplication.java and select run as Java Application
.
To run the specific service from Command Prompt, Open Project folder -> open command prompt from same folder -> Execute java -jar target/specific-service-1.2.0.jar
.
For example, to run the admin service, Open Project folder -> open command prompt from same folder -> Execute java -jar target/admin-service-1.2.0-SNAPSHOT.jar
.
The service should now be up and running.
For API documentation, refer here.
The APIs can be tested with the help of Swagger-UI and Postman.
Swagger is an interface description language for describing restful APIs expressed using JSON. You can access Swagger-UI of admin-services for dev-environment from:
Admin service– http://dev.mosip.net/v1/admin/swagger-ui/index.html?configUrl=/v1/admin/v3/api-docs/swagger-config#/
and localhost from http://localhost:8098/v1/admin/swagger-ui/index.html?configUrl=/v1/admin/v3/api-docs/swagger-config#/
.
Masterdata- http://dev.mosip.net/v1/masterdata/swagger-ui/index.html?configUrl=/v1/masterdata/v3/api-docs/swagger-config#/
and localhost from http://localhost:8086/v1/masterdata/swagger-ui/index.html?configUrl=/v1/masterdata/v3/api-docs/swagger-config#/
.
Syncdata- http://dev.mosip.net/v1/syncdata/swagger-ui/index.html?configUrl=/v1/syncdata/v3/api-docs/swagger-config#/
and localhost from http://localhost:8089/v1/syncdata/swagger-ui/index.html?configUrl=/v1/syncdata/v3/api-docs/swagger-config#/
.
Hotlist- http://dev.mosip.net/v1/hotlist/swagger-ui/index.html?configUrl=/v1/hotlist/v3/api-docs/swagger-config#/
and localhost from http://localhost:8095/v1/hotlist/swagger-ui/index.html?configUrl=/v1/hotlist/v3/api-docs/swagger-config#/
Postman is an API platform for building and using APIs. Postman simplifies each step of the API lifecycle and streamlines collaboration so you can create better APIs—faster. It is widely used tool for API testing.
Create an environment as shown in the image below.
This environment is created for dev. Give the variable name as url
and set both the values as https://dev.mosip.net
.
In the similar way, create another environment for localhost as shown below.
This environment is created for localhost. Give the variable name as url
and set both the values as http://localhost:8099
.
The MOSIP platform is configured via the Admin application. This application can be accessed only by a privileged group of administration personnel. The admin module provides the following functions:
Management of resources via CRUD operations:
Zone
Centers (registration centers)
Device
Machine
Users (Admin, registration staff)
Registration administration
Packet status
Retrieve lost RID
Resume RID
Administrative zones are virtual boundaries which a country can define to better manage their resources that are used during registrations. These resources includes Centers, Users, Machines and Devices. These zones can be defined in a hierarchical fashion and a country can allocate resources to such zones based on their requirements.
Resources under each zone is managed by a Zonal Admin. This is done by assigning an Administrative zone to the Zonal Admin during user creation.
These Zonal Admins can exist at any zonal hierarchy. (For e.g, a Zonal Admin can directly be mapped to the whole country as a Zone or can be mapped to a significantly smaller zone such as a city). Thus, these resources when mapped to an Administrative zone can only be managed by the Admin of that zone.
Deactivation refers to a temporary shutdown. This means that the resource will not be available for use unless and until it is activated later through the admin portal as required by the country.
Decommission refers to a permanent shut down of the resource. This also automatically deactivates it.
The primary difference being that a decommissioned resource cannot be bought into commission again as decommission refers to a permanent shutdown.
Also, in cases where a center has some resources mapped to it (e.g. machines, devices or users), the portal will not allow the admin to decommission such a center.
Note- Activation/Deactivation/Decommission of a center in one language will be applied to the same center created in all the languages.
Admin application is a web-based application used by a privileged group of administrative personnel to manage various master data. The various resources that can be managed by an Admin are:
Center (Registration centers)
Device
Machine
Users (Admin, Registration staff)
Along with the resource and data management, the admin can generate master keys, check registration status, retrieve lost RID, resume processing of paused packets. To start using the Admin portal, an admin user must be assigned to a zone.
To learn more, refer the videos below!
Session1
Session2
Setup of hierarchial zones
Create Admin roles in KeyCloak
Create first admin user in KeyCloak and assign "GLOBAL_ADMIN" role
Note- On login of first admin user, user zone mapping is handled automatically.
The above are done automatically as part of default sandbox installation.
Select the preferred language.
Login with KeyCloak credentials.
Map the other users(admins/registration operators/supervisors) to respective zones
Create centers and assign the users to a particular center
Highly recommended: Ensure to revoke the first super user's zone mapping and role after first user actions are completed.
GLOBAL_ADMIN
ZONAL_ADMIN
REGISTRATION_ADMIN
MASTERDATA_ADMIN
KEY_MAKER
This portal allows an Admin to view, create, edit, activate, deactivate and decommission registration centers.
An Admin can manage only centers under their administrative zones.
The administrator can filter the list of registration centers based on parameters like Center name, Center type, Status, Location code.
The system does not fetch the details of decommissioned registration centers but only active and inactive centers are displayed.
If the admin does not find a center, they can click the Center not available in logged in language button. Clicking on this button, displays the list of centers that are already created in other languages. On selecting a particular center, the information will be auto-populated in the Create page and be made available to the admin for modifications.
Language specific fields can be modified to create a center with the currently logged in language.
A center is created with multiple attributes and is mapped to the administrative zone that it belongs to.
A center can only be mapped to the configured location hierarchy level.
While defining centers, an admin can also define the working days of the week for a center and any exceptional holidays that might be applicable for a particular center.
An admin can update a center even after it has been created. The updates can include adding the details that were missed during creation of the center or changing the details of a center as required.
To update, click the Edit option from the Actions menu against a center name.
Note- Updates made to language specific fields updates data only for that language in the database while updates made to non-language dependent fields updates data against all the language entries for that center.
Select the Deactivate/Decommission option from the Actions menu against the center.
Activation/Deactivation/Decommission of a center in one language will be applied to the same center created in all the languages.
To know more, refer Activate/deactivate/decommission resources
Using this portal, an admin can manage the devices a country will use for registering residents like devices used for bio-metric capture (Fingerprint, Iris, Web camera, etc.), printers, scanners.
This portal allows an Admin to view, create, edit, activate, deactivate and decommission registration centers.
The admin portal allows an admin to view the list of all the devices available in the jurisdiction of their administrative zone.
The system does not fetch the details of decommissioned devices but only the active and inactive devices.
Note
Device entity is language agnostic (independent of languages).
The data collected about Devices is used only for book keeping, i.e., MOSIP system does not use this data for any validation.
The Admin can filter the list of Registration centers based on parameters like Device Name, Mac Address, Serial Number, Status, Map Status, Device Type, Device Spec ID.
A Device can be created with the multiple attributes and be mapped to the Administrative Zone it belongs to.
An admin can update missing information or change device details even after it is created.
To update, click the Edit option from the Actions menu against a device.
Select the Deactivate/Decommission option from the Actions menu against the device.
Admin portal allows an Admin to map/un-map each device to a center.
This mapping specifies as to which center the device will be used in.
A device can only be mapped to a center which belongs under the device’s Administrative Zone.
To do so, select the device and choose a Center Name from the dropdowm.
Admin portal allows an administrator to manage the machines a country will use for registering residents.
This portal allows an Admin to view, create, edit, activate, deactivate and decommission machines.
The admin portal allows an admin to view the list of all the machines available in the jurisdiction of their administrative zone.
The system does not fetch the details of decommissioned machines but only shows the active and inactive machines.
Note:
Machine entity is also language agnostic.
The administrator can filter the list of machines based on parameters like Machine name, Mac address, Serial number, Status, Machine type.
A machine can be created with the attributes like Machine ID, machine name, mac address, serial number, machine spec ID and administrative zone the machine belongs to.
A machine needs to be mapped to an administrative zone.
An admin can update missing details or make changes to machine details even after it is created.
To update, click the Edit option from the Actions menu against a machine.
Note- Updates made to language specific fields updates data only for that language in the database while updates made to non-language dependent fileds updates data against all the language entries for that center.
An admin can deactivate or decommission a machine through the admin portal.
Admin portal allows an Admin to map/un-map each machine to a center.
This mapping specifies as to which center the machine will be used in.
A machine can only be mapped to a center which belongs under the machine’s Administrative Zone.
To do so, select the machine and choose a Center Name from the dropdowm.
MOSIP uses Keycloak as an IAM (Identity access management tool) for managing Users. These users are internal users of MOSIP including Registration Officers, Registration Supervisors, Zonal Admins, Global Admins etc.
using this portal, an Admin can map the users to a zone and a center.
Once the user is created in KeyCloak, they need to be mapped to a zone to get access to specific information available in that zone.
Admin portal allows an admin to map users to a zone. This mapping specifies as to which zone the user will belong to.
A user can only be mapped to a zone which belongs under the user’s Administrative Zone.
A user can later be un-mapped from the zone in case a user needs to be moved to another zone. In such cases, the user will later need to be mapped to the new zone. Below image displays the list of users that are mapped to a zone.
To map a user to a zone,
Click Resources-> User Zone mapping
Click +Map Zone
Select the User Name, Administrative Zone from the dropdown.
Click Save.
To re-map a user to a zone,
Click Resources-> User Zone mapping
Select Remap from the Actions menu against the mapped user.
Update the User Name/ Administrative Zone from the dropdown.
Click Save.
Note- If the center is already mapped, the admin needs to unmap the center to remap the zone.
Once the user is mapped to a zone, they will be listed in the screen below. Now, the user will be mapped to a center to be able to manage their assigned center.
Admin portal allows an admin to map users to a center. This mapping specifies as to which center the user will be used in.
A user can only be mapped to a center which belongs under the user’s Administrative Zone.
A user can later be un-mapped from the Center in cases where a User is needed to be moved to another Center. In such cases, the user will later need to be mapped to the new center. In case the user is required to be mapped to a Registration center outside the Administrative Zonal restriction, the Administrative Zone of the user must be changed.
Map/un-map/re-map user to a registration center
To map a user to a center,
Click Resources-> User Center Mapping
Select Map from the Actions menu against the mapped user.
Select the Center Name from the dropdown against the User Name, Administrative Zone.
Click Save.
To get the results starting with specific character/ set of characters, prepend that specific character/set of characters with asterisk
symbol.
Similarly to get the results ending with specific character/ set of characters, append that specific character/ set of characters with asterisk
.
For the results containing a specific character/ set of characters, prepend and append that specific character/ set of characters with asterisk
.
Below is the image illustrating the same.
A Registration packet generated in Registration client is sent to Registration Processor for further processing and UIN generation.
Using this Portal, A Registration Admin can view the status of a packet by entering the RID of the packet.
The packet status will contain all the stages the packet has passed through along with the last stage the packet is in.
In case the packet has not been processed or is marked for Re-Send/Re-Register, the admin will be able to view specific comments indicating the reason for that particular status.
The Registration Admin has the privilege to view the registration packets that are in a paused state.
Admin can use this portal to resume or reject paused packets. They would have 3 options:
Resume processing (from where it was paused)
Resume from the beginning
Reject
Once processing of a packet is resumed, it will be removed from this list
The Registration Admin can use this feature to retrieve lost RID.
For instance, if the resident did not provide any valid email and/or phone number and has lost the RID slip received during the registration, in order to find their RID details, the resident contact MOSIP helpline and share details such as name, centre name, registration date and postal code to the admin, who will use the lost RID feature and try to retrieve the RID number.
A few filters may be applied to retrieve the RID.
Note: This feature is currently under development.
Admin portal allows an Admin to manage the Masterdata applicable for a country.
These data includes list of Genders, list of Holidays, Templates, Center Types, Machine Types etc.
To know more, refer to Masterdata guide.
If a country decides to uplaod the data through the .csv files, they could use this feature to upload the existing data into the MOSIP platform.
The listing screen displays the uploaded data transaction information.
As the information inside .csv files may be huge, it would go through the batch job to process the information and store it in the tables. Also, it may take time to get unique transaction ID against the particular action.
To upload Master data using Admin portal,
Go to Bulk Upload > Master Data
On the master data dashboard, click Upload Data.
Select the operation (insert/update/delete)
Select the table name into which the data needs to be uploaded into.
Click Choose file to select the data and click Upload
To view the format for inserting data in a particular table, click on the Download icon.
A CSV file gets downloaded in which the first row represents the column names and the rest of the rows are the data which will be inserted into the table(sample).
From 1.2.0.1-B2 version, apart from comma, other special characters (i.e., '|','$'etc.) can also be used as a separator in the csv file used for masterdata bulk upload. This can be done by updating the property mosip.admin.batch.line.delimiter
with the same special character.
Note: While editing CSV files, it is recommended to keep track of the Date format and Time format to be the same as the acceptable formats. The acceptable Date format is YYYY-MM-DD and the acceptable Time format is HH:MM:SS. Any other Date and Time formats in CSV files will result in a DataType Mismatch Error
.
To upload packets using Admin portal,
Go to Bulk Upload > Packets
On the packet upload dashboard, click Upload Packet.
Select the following from the dropdown:
Center name
Source (currently displays Registration Client)
Process (New, Update UIN, Lost, Biometric correction)
Supervisor status (Approved/Rejected)
These details are important if the packet needs to be synced before upload.
Click Choose file to select the packets and click Upload.
How is the packet upload performed with or without DATA_READ role?
For uploading the packets through the Admin portal, ensure that the packets are available in the machine or the external hard disk connected from where the Admin Portal is being used.
With the help of this feature, the Admin user can generate and manage the keys required in MOSIP.
The logged in user with KEY_MAKER
role will have access to view and generate the master key in the Admin portal.
Using this option, the logged in user will be able to generate only the Root key and Module master key. To generate the key, the user has to select the Application ID from the options available in the drop down, leave the Reference ID as blank for Root and Module master key and provide other certificate attributes to be used at the time of generation of certificate for the key.
This certificate attributes in the portal are optional, if not provided, default values configured in Key Manager service will be used.
For Kernel signature key (which is considered as the master key and stored in HSM), Reference ID needs to be provided and the value has to be SIGN
.
Force flag option is available in key generation. The logged in user can select option value True to force invalidating existing key and generate new key in Key Manager service.
The logged in user has to select the return object after the generation of key.
The user can select either Certificate or CSR (Certificate Signing Request). The key will be generated only when the key is not available in Key Manager service otherwise already generated key certificate will be returned for the generation request.
CSR (certificate signing request) is required when there is a need to procure a valid certificate from a valid CA.
GenerateCSR option can be used to request for a CSR and this option will be visible to all the users who log in to the Admin portal.
The logged in user can request for generation of CSR for any key generated in Key Manager service.
The user has to provide the Application ID and Reference ID to get a CSR.
New key will be auto-generated in case the key does not exist and the already existing key has expired for the Module Encryption keys.
The user can get certificate for all the keys generated in Keymanager and any partner certificates uploaded in Keymanager service for partner data share purpose.
GetCertificate option is visible to all the users who log in to the Admin portal.
The user has to provide the Application ID and Reference ID to get a certificate.
New key will be auto generated in case the key does not exist and the already existing key has expired for Module encryption keys.
Only current valid certificates will be returned when the user requests for a certificate.
The logged in user can use this option to update the certificate for all the keys generated in Key Manager service.
This option is used in scenarios where a valid CA certificate has been procured for a key available in Key Manager service.
The logged in use can use this option to upload partner certificate in Key Manager service.
Partner certificates will be used in Key Manager service to encrypt any sharable data using the partner certificate required in datashare from MOSIP to any partner.
Partner certificates can also be used in Key Manager service for signature verification purpose.
Masterdata is the necessary base data to run MOSIP services. The data resides in . This data needs to be customized for a country specific deployment.
Masterdata may be uploaded in the following manner:
One-time bulk upload:
(for sandbox installation): Using . The default data uploaded during sandbox installation is available in .
Country specific data: Using .
Updates: Updates to tables may be done using the .
The tables that need to be modified for country specific data are listed below. Other tables in mosip_master
DB are either system-filled or pre-filled and not to be modified.
Copy Excel files from to a folder.
For all tables listed below modify lang_code
and add corresponding rows for your .
Modify the files for your deployment as per guide below.
Upload first time using scripts given .
Subsequently, update data ONLY using .
This guide provides a comprehensive list of configurable properties for the Android Registration Client. Please note that this list is not exhaustive but serves as a helpful checklist for reviewing commonly configured properties.
It is important to acknowledge that all properties listed in this guide are automatically synchronized with the Android Registration Client. These properties are sourced from the registration-default.properties
file.
application-default.properties
registration-default.properties
Timeouts in milliseconds set during any http calls to SBI.
Quality score threshold based on modality, Possible values 1 to 100
Retry attemps, Possible values 1 to 10
Quality score threshold based on modality for operator authentication, Possible values 1 to 100`
Jobs like RID sync, packet upload, status sync is carried out in batches, number of registration records to be executed in a batch on every trigger.
Default CRON expression for scheduling the Jobs.
mosip.registration.sync_jobs_restart_freq=0 0 */11 ? * *
Enables / disables reviewer authentication on any biometric exception during registration
mosip.registration.reviewer_authentication_configuration=Y
If enabled, cross-checks of residents biometrics with locally stored operator biometric templates.
mosip.registration.mds.deduplication.enable.flag=N
Minimum disk space required to create a packet - in MB
mosip.registration.disk_space_size=5
Maximum no. of days for approved packet pending to be synced to server beyond which Registration Client is frozen for registration
mosip.registration.last_export_registration_config_time=100
No. of days beyond audit creation date to delete audits
mosip.registration.audit_log_deletion_configured_days=10
Maximum duration to which registration is permitted without sync of master data
mosip.registration.sync_transaction_no_of_days_limit=5
Allowed number of invalid login attempts
mosip.registration.invalid_login_count=50
Configuration used to check if any sync job is missed/ failed beyond expected days, this configuration is checked everytime operator clicks on any registration process. We follow below convention to create this config key.
mosip.registration.job api name as in sync_job_def table.frequency=value in days
Date format to be displayed on acknowledgement slip, default value - dd/MM/yyyy hh:mm a
mosip.registration.application_date_format
Date format to be displayed on Registration Client dashboard, default format - dd MMM hh:mm a
mosip.registration.dashboard_date_format
Due to the absence of UI specifications in the 1.1.5.x versions, the android regclient addresses backward compatibility by migrating the schema of these versions to the LTS UI Spec structure.
In order to facilitate this migration, certain configurations and templates have been incorporated to ensure compatibility with the 1.1.5.x server. The list of these configurations is provided below.
mosip.registration.consent-screen-template-name=reg-consent-screen-content-template
Consent screen is not a part of 1.1.5.x schema structure. So, we are completely fetching this consent screen content from master.template
table, and the templateTypeCode
for the consent screen content is mentioned in the above configuration.
mosip.registration.individual-biometrics-id=individualBiometrics
The id of individual biometrics should be mentioned in the above property according to the configured 1.1.5.x schema.
mosip.registration.introducer-biometrics-id=guardianBiometrics
The id of guardian/ introducer biometrics should be mentioned in the above property according to the configured 1.1.5.x schema.
mosip.registration.infant-agegroup-name=INFANT
The age-group name for infants (aged below 5 years) which is configured in the configured server should be mentioned in the above property.
mosip.registration.agegroup-config={"INFANT":{"bioAttributes":["face"],"isGuardianAuthRequired":true},"ADULT":{"bioAttributes":["leftEye","rightEye","rightIndex","rightLittle","rightRing","rightMiddle","leftIndex","leftLittle","leftRing","leftMiddle","leftThumb","rightThumb","face"],"isGuardianAuthRequired":false},"SENIOR_CITIZEN":{"bioAttributes":["leftEye","rightEye","rightIndex","rightLittle","rightRing","rightMiddle","leftIndex","leftLittle","leftRing","leftMiddle","leftThumb","rightThumb","face"],"isGuardianAuthRequired":false}}
The above property indicates list of age-groups, required bio-attributes, and a flag which indicates guardian authentication is required or not. This property should be changed according to the server configuration and requirements.
mosip.registration.allowed-bioattributes=leftEye,rightEye,rightIndex,rightLittle,rightRing,rightMiddle,leftIndex,leftLittle,leftRing,leftMiddle,leftThumb,rightThumb,face
The above property defines the list of bio-attributes that are allowed for scanning during registration. If there are any changes in the server, it should be changed accordingly.
mosip.registration.default-app-type-code=000
The above property defines the default applicantTypeCode. In LTS, we have applicanttype.mvel script to fetch the documents according to the age, gender and some other attributes. Based on the applicant details, the script returns an applicantTypeCode which can be any value from “000” to “014”, and respective documents will be fetched from master.applicant_valid_document table
. Since we do not have this script defined in 1.1.5.x to handle this, we have added a default applicantTypeCode
.
The Android Registration Client is a tablet application that serves as a portable version of the existing desktop . It has been developed to support accessibility on all Android devices. The existance of Android Registration Client came about in order to meet the mobility requirements of countries adopting MOSIP.
The primary objective of the tablet version is to facilitate the registration process for residents, specifically those who are unable to physically visit registration centres and also serve remote locations where setting up Registration centres is not feasible. To address this challenge, the Android Registration Client was created, enabling Operators/ Supervisors to easily reach the remote areas and maximise resident registrations across the country.
To have a quick glance at the features, refer the video below!
The first developer release of Android Registration Client offers the following key features:
Operator/ Supervisor Login (offline and online): Operators can securely log in using their credentials, whether in offline or online mode, to carry out various registration transactions. To enable offline login, the Operator must have previously logged in and synchronized their data over a network.
Multi-language support: The Android Registration Client supports multiple languages for content display and data entry.
Display Language: Display Language refers to the language used for rendering UI elements such as labels and headings. With the Android Registration Client, Operators have the option to choose their preferred language for UI display. This language selection can be made on the login screen. Currently, the supported display languages include Arabic, French, English.
New languages can be added by following the below steps:
i. Additional languages can be configured by adding localisation files in lib/l10n
folder present in the root project directory("android_registration_client").
ii. The languages that are rendered on the UI will be based on the country configuration (after master data sync). The default display language is English. The other languages will be available in the UI after master data sync.
Data Entry Language: The Data Entry Language refers to the specific language utilized by the Operator while gathering data, which is then stored on the server in that selected language. During the registration process, the Operator can choose the language preference for the data collected, allowing applicants to provide information in their desired language. This language selection option becomes available upon initiating a new registration. The responsibility for managing the data entry language lies within the UI Spec, and any modifications or changes can be made through that specification.
Auto-Sync/ manual sync: On launching the Android Registration Client and logging in for the first time, the system automatically syncs the following data:
Configuration sync: Sync of properties which drives in deciding the ARC UI functionality. For example: Invalid login attempts, idle timeout, thresholds, etc.
Masterdata sync: As a part of this sync, supporting information like Dynamic fields data, templates, locations, screen authorization, blocklisted words, etc. are pulled in.
UserDetails sync: userID, along with their status is synced. Only the user details belonging to machine mapped center will be synced.
Certificate sync: Certificates used to validate the server signatures, device CA certificates, public key (specific to a center and machine, also called as policy key) used to encrypt the registration packet will be synced.
Consent: Prior to the registration process, applicants must provide consent to the terms and conditions presented on the consent screen. This explicitly asks the applicant to grant permission for storing and using their Personally Identifiable Information (PII).
Demographic Details: Once the consent is obtained, the Operator will enter the demographic data of the applicant in the language preferred by the applicant. This includes details such as their name, gender, date of birth, and residential address.
Documents Upload: Following the completion of the demographic details, the Operator can select the document type, input the reference, and upload the supporting documents provided by the applicant. Supporting documents may include Proof of Address, Proof of Identity, and Proof of Birth, based on the country-specific requirements.
Biometrics: After the documents have been uploaded, the Operator will proceed to capture the applicant's biometrics. The biometrics captured are as follows:
The acquisition of biometric data is regulated by the country. The country has control over the capture of each type of biometric (fingerprint, iris, or face) through the global configuration. When the Operator selects the Capture button, the biometric SBI application is accessed to capture the biometrics. Once the biometrics are obtained, the data and control are returned to the Android Registration Client. To obtain the resident's biometrics, the quality of the captured image must exceed the threshold specified by the country. The biometrics can be captured set number of times if necessary to meet the quality threshold. In situations where none of the captured images meet the threshold, the image with the highest quality score will be saved.
If the resident has a biometric exception (such as a missing finger/eye or very poor finger/iris quality), the Operator can designate that particular biometric as an exception. However, the Operator must still capture the resident's exception photo.
Preview section: The Operator has the ability to review the data entered by the applicant, including demographic information, uploaded documents, and captured biometrics. This preview allows the Operator to ensure the accuracy of the entered data. If any mistakes are found, the Operator can easily go back to the corresponding section and make the necessary corrections. If the data is correct, the Operator can proceed to the next step, which is to authenticate themselves.
Operator authentication: Once both the Operator and applicant have confirmed that the data is accurately filled, the Operator is required to authenticate themselves using their credentials. After a successful authentication, the data packets are created and only then the sync and upload operations can be performed.
Packet sync: After the applicant's registration form has been completed and the Operator has authenticated themselves, a packet sync must be performed. This can be done either manually or as a background job(auto sync and uplaod of packets). Packet sync ensures that the packet is prepared for uploading and the status of the uploaded packet is synchronized with the server.
Packet Upload: Once the packet sync is successfully completed, the system will proceed to upload the packet to the server when an internet connection is available. If there is no network access, the system will attempt to upload the packet as soon as connectivity is established.
Acknowledgment section: Following the completion of the new registration process, an acknowledgment receipt is generated. This receipt includes the AID(Application ID), captured demographic data in the selected language, a photograph of the resident, and a ranking of each finger from 1 to 10, with 1 representing the finger with the best quality. The receipt is designed to be easily printed.
Download and install the APK on Android tablet.
Once ARC is installed, long press on the logo to copy the machine details.
Go to Resources/Machine
and click on Create machine
Add a new machine and enter the machine details:
Add the specs as Android
Map it to a Zone and Center
Add the Machine spec ID as Android
Enter Device name
Enter Public Key
Enter Sign Public Key
Create the role Default
in KeyCloak with all the other roles.
Create the Operator’s user account in KeyCloak and set the password and assign the role as Default
, REGISTRATION_OFFICER
, Registration Operator
, REGISTRATION_SUPERVISOR
Login into Admin Portal to perform the following and add the user:
After login into Admin Portal, go to User Zone Mapping
and add the zone for the user and activate the zone.
Go to User Center Mapping
and add the center for the user and activate it.
Note: The user should be assigned to the same Zone and Center as the device.
The Android Registration Client is compatible with the following MOSIP platform versions:
1.1.5.x
LTS 1.2.0 and above
Reference implementation of the Admin portal is available in repository.
To know more, refer to the .
To know more about the setup, read .
Refer .
.
GLOBAL_ADMIN | ZONAL_ADMIN | REGISTRATION_ADMIN | MASTERDATA_ADMIN | KEY_MAKER |
---|---|---|---|---|
LoggedIn User Role | Packet Sync | Packet Upload |
---|---|---|
Category | Table | Guide |
---|
To know more, refer .
New Registrations : Operators have the ability to register a resident using the New Registration
feature. The registration process can be customized through the . The required data for registering an applicant are as follows:
On the , using admin credentials, login and perform the following to add the device:
To read through the comprehensive list of configurable properties for the Android Registration Client, refer .
For more details on UI specifications for the Android Registration Client, refer .
Centers
Devices
Packet Status
Devices
GenerateMasterKey
User Zone Mapping
Machines
Pause/ Resume RID
Machines
GenerateCSR
All Master Data
User Zone Mapping
Retrieve Lost RID
All Master Data
GetCertificate
Masterdata Bulk Upload
User Center Mapping
Packet Bulk Upload
Masterdata Bulk Upload
UploadCertificate
All Master Data
UploadOtherDomainCertificate
Masterdata Bulk Upload
With DATA_READ
Yes
only after successful sync
Without DATA_READ
No
Yes
MOSIP provides automation test repositories for the following:
Selenium webdriver-based Admin Portal Automation covers CRUD (create, read, update and delete) operation performed via UI with Chrome driver.
Registration test automation covers these flows: New, Update, Correction, and Lost flows.
To know more about each, click here.
This repository contains API automation tests. The automation is written using Java REST Assured and TestNG framework. The following modules are covered:
Pre-registration
Masterdata
Partner Management
ID Repository
IDA
Resident
Functional tests support multi-languages, i.e., the input can be provided in any of the languages configured in a given MOSIP installation.
This repository contains a test framework for end-to-end testing of MOSIP functionality. The following functionalities are covered:
Registration
Pre-registration + Registration
Authentication
As the name suggests, Commons refers to all the common services (also called "kernel") that are used by other modules of MOSIP. The Kernel services are listed below:
Refer API Documentation.
To know more about the developer setups, read:
The documentation here will guide you through the pre-requisites and the other necessary details required for Android Registration Client developer setup.
The android-registration-client repository contains the Android Registration Client software for MOSIP. The feature-flutter branch focuses on integrating Flutter into the client.
To set up the Android Registration Client with Flutter and Android Studio, follow the steps below:
Flutter SDK (>3.10): Install Flutter by following the official Flutter installation guide.
Android Studio (or Any IDE of your choice): Download and install Android Studio from the official Android Studio website.
The feature-flutter branch of android-reg-client is currently being actively developed. If you wish to access this branch, you can clone the repository by executing the following command in your terminal. Alternatively, you can download one of the releases available in the repository's release section.
Active Branches:
developer-release/flutter/0.9.x (developer release branch)
feature-flutter (active development branch)
To begin, launch Android Studio.
Next, select Open an existing Android Studio project and navigate to the cloned repository.
Open the android-registration-client
directory as a project in Android Studio.
In order to integrate Flutter with Android Studio, install the Flutter plugin by accessing File > Settings > Plugins
and searching for Flutter. Proceed to click on Install to install the plugin.
To ensure proper functionality, configure the Flutter SDK path by navigating to File > Settings > Languages & Frameworks > Flutter
and specifying the Flutter SDK path as the location where you have installed Flutter.
Finally, save the changes by clicking on the "Apply" button.
Customising the Registration Client
Styling of the application can be configured by modifying these files lib/utils/app_style.dart, lib/utils/app_config.dart
Application language bundles can be added to this path lib/l10n
.
Label and application logo can be changed here android/app/src/main/AndroidManifest.xml
The pigeon.sh
file consists of the necessary commands for downloading dependencies and generating Flutter - Android native communication code. Please execute the pigeon.sh
file or execute the commands within the file separately.
Ensure you have connected an Android device or initiated an Android emulator.
Open the terminal within Android Studio or use an external terminal.
Navigate to the android-registration-client
directory.
Run the following command in order to build and execute the application:
Excecute the commands below to debug and release the APK
The Mock MDS tool can be utilized to simulate the functionalities of biometric devices. The Mock MDS application is compliant with CTK standards and can serve as a substitute for Android SBI modules during testing and validation.
Install the Mock MDS application.
Access the Settings menu.
Under Device Configuration, choose Registration from the dropdown menu.
In P12 Configuration:
Enter the necessary credentials for the Device Key and upload the Device P12 file.
Enter the required credentials for the FTM Key and upload the FTM P12 file.
Complete all fields in MOSIP IDA Configuration.
In Modality Configuration, specify the quality score for Face, Finger, and Iris scans(these values can also be adjusted during testing).
Click on the Save button.
Go back to the Home Page and select LOAD AND VALIDATE CERTIFICATES
.
A toast message will be displayed indicating the success of the validation process.
Note: To view the released version of the Mock SBI APK, click here.
To download the Mock SBI APK, click on camera-mds.zip
.
If you would like to contribute to the Android Registration Client, please follow the guidelines outlined here.
The Android Registration Client is licensed under the MIT License.
If you encounter any issues or have any questions, please open an issue on the GitHub repository.
GitHub- mosip/android-registration-client: Reference Android Registration Client Software - WIP
Documents |
| Categories of documents to be captured |
| Specific documents related to a country |
|
Location |
| List of location hierarchy |
| List of locations stored in a hierarchical format |
| Holidays specific to different locations. Used in registration centre creation. |
Machine |
| Example mobile, stationary. Refers to |
| Model, make of the registration machine |
ID Schema |
|
| Dynamic dropdowns used during data capture |
| Only |
|
Registration client |
| List of changeable configurations by the operator |
| Only |
| Only |
| Only |
| Only |
| List of reasons for a reason category |
| Only |
Registration center |
| Type of center |
| List of registration centers |
| Historical data for any modification done on a registration center. One time intialization of this table identical to |
| Exception holiday for a registration center |
| Working and non-working day for a center |
Biometrics |
| Only |
| Only |
Templates |
| Only |
| Only |
| Only |
Others |
| List of blocked words |
| Only |
| Only |
| Only |
| Only |
| Only |
| List of titles used in the country |
| List of administrative zones in a country |
The MOSIP platform requires integration with several other systems. Typically, a System Integrator (SI) would assemble all the pieces together to build a complete national ID solution. All entities that participate in providing the external components are called MOSIP Partners.
* Label: Reference in partner_type
table of mosip_pms
database.
Onboarding of a partner refers to registering a partner in a particular deployment of MOSIP. Partners need to be onboarded to establish trust. The onboarding process consists of loading partner details in the database, exchanging certificates etc, detailed in the later sections. Such onboarding is required to be done on any fresh MOSIP installation. For instance, if you install a sandbox, you would need to follow the onboarding process for each partner.
The sections below describe the onboarding process for each type of partner.
MISP should have a trusted X.509 certificate with a chain of CA certificates.
MISP self-registers on the PMS portal providing partner id, name, organisation name (same as in certificate), partner type (MISP_type
) (This functionality will be available on the portal in the 1.2.x version of MOSIP)
MISP uploads all certificates.
MOSIP Admin generates the MISP license key and provides it to MISP.
AP should have a trusted X.509 certificate with a chain of CA certificates.
AP registers with MISP and obtains the MISP license key (this setup is outside of the MOSIP system).
The MISP used by AP should have been already onboarded onto MOSIP.
AP self-registers on the PMS portal providing partner id, name, organisation name (same as in certificate), partner type (Auth_Partner
) etc.
AP uploaded all certificates.
AP selects the policy group and policy. This request is sent to MOSIP Admin for approval.
On approval, AP generates an API key that can be used along with the MISP license key to interact with the IDA system.
DP should have a trusted X.509 certificate with a chain of CA certificates.
DP self-registers on the PMS portal providing partner id, name, organisation name (same as in certificate), partner type (Device_Provider
) etc.
DP uploads all certificates.
Any approval from MOSIP? (TODO)
FTMP should have a trusted X.509 certificate with a chain of CA certificates.
FTMP self-registers on the PMS portal providing partner id, name, organisation name (same as in certificate), partner type (FTM_Provider
) etc.
FTMP uploads all certificates.
TODO
CP should have a trusted X.509 certificate with chain of CA certificates.
CP self-registers on PMS portal providing partner id, name, organisation name (same as in certificate), partner type (Credential_Partner
) etc.
CP uploades all certificates.
CP selects the policy group and policy.
CP adds biometric extractors for the policy.
OVP should have a trusted X.509 certificate with a chain of CA certificates.
OVP self-registers on the PMS portal providing partner id, name, organisation name (same as in certificate), partner type (Credential_Partner
) etc. (Using APIs, as OVP support on PMS Portal is available in the later version of MOSIP.)
OVP uploads all certificates.
OVP selects the policy group and policy.
OVP adds biometric extractors for the policy.
The MOSIP Partner Programme (MPP) was initiated to help stakeholders connect with MOSIP, and become part of an ecosystem invested in building foundational digital ID systems that are trustworthy, secure, efficient, and interoperable while being customised to specific needs.
The DataShare service is used to share data with other entrusted services and partners. The mechanism shares are the following:
Retrieves and stores data to be shared in and returns a Datashare URL.
It fetches data from the Object Store when the Datashare URL is called.
The sharing of data is controlled by the .
Data is encrypted before sharing it with partners. Learn more about .
The relationship of Datashare Service with other services is explained here. NOTE: The numbers do not signify the sequence of operations or control flow. Arrows indicate the data flow.
ABIS Handler Stage creates datashare for ABIS.
Manual Adjudication Stage creates datashare for adjudication.
Verification Stage creates datashare for verification.
Retrieves datashare from the object store when the datashare URL is called.
Print Service creates UIN PDF and uploads to datashare through Credential.
Printing Stage creates credentials for printing systems and sends the data through datashare.
module provides audit-related functionalities.
Below is a list of tools required for auditing:
JDK 11
Any IDE (like Eclipse, IntelliJ IDEA)
Apache Maven (zip folder)
PostgreSQL
Any DB client (like DBeaver, pgAdmin)
Postman (any HTTP Client)
Git
Any Editor (like Vscode, Notepad++ etc optional)
lombok.jar (jar file)
settings.xml (document)
1. Download and .
2. Unzip Apache Maven and move settings.xml
to the "conf" folder <apache maven unzip path>\conf
.
4. Check the Eclipse installation folder to see if the lombok.jar
is added.
5. Configure the JDK (Standard VM) with your Eclipse by traversing through Preferences → Java → Installed JREs
.
Open the project folder where pom.xml
is present.
Open the command prompt from the same folder.
Run the command mvn clean install -Dgpg.skip=true -DskipTests=true
to build the project.
After building, open Eclipse and select Import Projects → Maven → Existing Maven Projects → Next → Browse to project directory → Finish
.
After successfully importing of project, update the project by right-clicking on Project → Maven → Update Project
.
4. The audit uses two property files, kernel-default
and application-default
. Please configure them as needed. For instance,
Update the mosip.kernel.auditmanager-service-logs-location
property to update the location of log files.
Update URL's in property files.(It can be either pointed to any remotely or locally deployed services)
java -jar -Dspring.profiles.active=native -Dspring.cloud.config.server.native.search-locations=file:{mosip-config-mt_folder_path}/config -Dspring.cloud.config.server.accept-empty=true -Dspring.cloud.config.server.git.force-pull=false -Dspring.cloud.config.server.git.cloneOnStart=false -Dspring.cloud.config.server.git.refreshRate=0 {jarName}
.
6. Run the server by opening the config-server-start.bat
file.
7. To verify the config-server, hit the below URL:
http://localhost:51000/config/{spring.profiles.active}/{spring.cloud.config.name}/{spring.cloud.config.label}
for instance http://localhost:51000/config/kernel/env/master
.
Audit REST service consist of bootstrap.properties
file in src/main/resources
.
Below properties needed to be modified in order to connect to the config server:
Services can be run using Run As -> Spring Boot App/Java Application
.
The API's can be tried with the help of Swagger-UI and Postman.
Swagger-UI of service can be accessed from (https/http)://(<domain>/<host>:<port>)/<context-path>/swagger-ui/index.html?configUrl=<contect-path>/v3/api-docs/swagger-config
for instance https://dev2.mosip.net/v1/auditmanager/swagger-ui/index.html?configUrl=/v1/auditmanager/v3/api-docs/swagger-config
.
The API's can be tried using postman by copying CURL command below, updating host and importing in Postman.
Mapping of document category, type and
Refer to . Update the JSON in schema_json
column of
UI specification for registration and pre-registration UI screens. See
Partner type | Description | Label* |
---|
Partner policies control the data that needs to be shared with a partner. Learn more about .
Policy for the AP must be pre-defined (see ).
Datashare policy must be pre-defined (see ).
CP maps policy to one of the supported .
Datashare policy must be pre-defined (see ).
OVP maps policy to auth
.
Refer for further details.
Refer to .
Datashare Service calls to get the policy for creating shares.
It calls the Service to encrypt data as per policy.
Stores datashare inside .
External systems like , Print System, Adjudication system etc. calls Datashare Service to get the datashare.
creates credentials for CA certificates to be used by .
Service sends data to ID Authentication system through Credential.
3. Install Eclipse, open the lombok.jar
file, and click Install/Update
.
For the code setup, clone the repository and follow the guidelines mentioned in the .
1. Download and add to project Libraries → Classpath → Add External JARs → Select Downloaded JAR → Add → Apply and Close
.
2. Clone .
3. Refer to deploy local DB.
Secrets can be encrypted using .
5. Download . For Windows, download , linux users can run
For API documentation, refer .
Commons module provides a bedrock to build and run services by providing several significant necessary technical operations. It contains common functionalities which are used by more than one module.
Below is a list of tools required in Commons:
JDK 11
Any IDE (like Eclipse, IntelliJ IDEA)
Apache Maven (zip folder)
PostgreSQL
Any DB client (like DBeaver, pgAdmin)
Postman (any HTTP Client)
Git
Any Editor (like Vscode, Notepad++ etc optional)
lombok.jar (jar file)
settings.xml (document)
Download lombok.jar and settings.xml.
Unzip Apache Maven and move settings.xml
to "conf" folder <apache maven unzip path>\conf
.
Install Eclipse, open the lombok.jar
file and then click Install/Update
.
Check the Eclipse installation folder to see if the lombok.jar
is added.
Configure the JDK (Standard VM) with your Eclipse by traversing through Preferences → Java → Installed JREs
.
For the code setup, clone the repository and follow the guidelines mentioned in the Code Contributions.
Open the project folder where pom.xml
is present.
Open the command prompt from the same folder.
Run the command mvn clean install -Dgpg.skip=true -DskipTests=true
to build the project.
After building, open Eclipse and select Import Projects → Maven → Existing Maven Projects → Next → Browse to project directory → Finish
.
After successful importing of project, update the project by right-click on Project → Maven → Update Project
.
Download Auth adapter and add to project Libraries → Classpath → Add External JARs → Select Downloaded JAR → Add → Apply and Close
.
Clone mosip-config repository.
Refer Commons-DB-deploy to deploy local DB.
For integration with any of our environments, do reach out to our team.
Commons
uses two property files, kernel-default
and application-default
, configure them accordingly. For instance
Update spring.mail.host
property to update SMTP host.
Secrets can be encrypted using config server.
Update Url's in property files.(It can be either pointed to any remotely or locally deployed services).
Download kernel-config-server.jar. For windows download config-server-start.bat, linux users can run java -jar -Dspring.profiles.active=native -Dspring.cloud.config.server.native.search-locations=file:{mosip-config-mt_folder_path}/config -Dspring.cloud.config.server.accept-empty=true -Dspring.cloud.config.server.git.force-pull=false -Dspring.cloud.config.server.git.cloneOnStart=false -Dspring.cloud.config.server.git.refreshRate=0 {jarName}
.
Run the server by opening the config-server-start.bat
file.
To verify the config-server, hit the below URL http://localhost:51000/config/{spring.profiles.active}/{spring.cloud.config.name}/{spring.cloud.config.label}
for instance http://localhost:51000/config/kernel/env/master
.
Commons
repo consists of two types of project- REST services and libraries.
Every REST service consist of bootstrap.properties
file in src/main/resources
.
Below properties needed to be modified in order to connect to the config server:
Services can be run using Run As -> Spring Boot App/Java Application
.
Libraries are not meant to be run alone they are projects used by commons as well as other modules of mosip to perform commons operations.
For API documentation, refer here.
The API's can be tried with the help of Swagger-UI and Postman.
Swagger-UI of services can be accessed from (https/http)://(<domain>/<host>:<port>)/<context-path>/swagger-ui/index.html?configUrl=<contect-path>/v3/api-docs/swagger-config
for instance https://dev.mosip.net/v1/authmanager/swagger-ui/index.html?configUrl=/v1/authmanager/v3/api-docs/swagger-config
.
Context-path of services is present in bootstrap.properties
file in src/main/resources
of every service.
The API's can be tried using Postman. URLs and Body structures can be found in Swagger or curl command can be copied and imported in Postman.
OpenID-Bridge module provides AutnN and AuthZ related funtionalities.
Below is a list of tools required in OpenID Bridge:
JDK 11
Any IDE (like Eclipse, IntelliJ IDEA)
Apache Maven (zip folder)
PostgreSQL
Any DB client (like DBeaver, pgAdmin)
Postman (any HTTP Client)
Git
Any Editor (like Vscode, Notepad++ etc optional)
lombok.jar (jar file)
settings.xml (document)
Download lombok.jar and settings.xml.
Unzip Apache Maven and move settings.xml
to "conf" folder <apache maven unzip path>\conf
.
Install Eclipse, open the lombok.jar
file and then click Install/Update
.
Check the Eclipse installation folder to see if the lombok.jar
is added.
Configure the JDK (Standard VM) with your Eclipse by traversing through Preferences → Java → Installed JREs
.
For the code setup, clone the repository and follow the guidelines mentioned in the Code Contributions.
Open the project folder where pom.xml
is present.
Open the command prompt from the same folder.
Run the command mvn clean install -Dgpg.skip=true -DskipTests=true
to build the project .
After building, open Eclipse and select Import Projects → Maven → Existing Maven Projects → Next → Browse to project directory → Finish
.
After successful importing of project, update the project by right-click on Project → Maven → Update Project
.
1. Clone mosip-config repository.
2. OpenID Bridge uses two property files, kernel-default
and application-default
, configure them accordingly. For instance,
OpenID bridge connects to an IAM which supports Openid and Oauth. For integration with our keycloak, Please reach out to our team.
Update mosip.iam.open-id-url
property to update iam url.
Secrets can be encrypted using config server
Update Url's in property files.(It can be either pointed to any remotely or locally deployed services)
3. Download kernel-config-server.jar. For Windows, download config-server-start.bat, Linux users can run
java -jar -Dspring.profiles.active=native -Dspring.cloud.config.server.native.search-locations=file:{mosip-config-mt_folder_path}/config -Dspring.cloud.config.server.accept-empty=true -Dspring.cloud.config.server.git.force-pull=false -Dspring.cloud.config.server.git.cloneOnStart=false -Dspring.cloud.config.server.git.refreshRate=0 {jarName}
.
4. Run the server by opening the config-server-start.bat
file.
5. To verify the config-server, hit the below URL: http://localhost:51000/config/{spring.profiles.active}/{spring.cloud.config.name}/{spring.cloud.config.label}
for instance http://localhost:51000/config/kernel/env/master
Audit REST service consists of bootstrap.properties
file in src/main/resources
.
Below properties needed to be modified in order to connect to the config server:
Services can be run using Run As -> Spring Boot App/Java Application
.
For API documentation, refer here.
The API's can be tried with the help of Swagger-UI and Postman.
Swagger-UI service can be accessed from (https/http)://(<domain>/<host>:<port>)/<context-path>/swagger-ui/index.html?configUrl=<contect-path>/v3/api-docs/swagger-config
for instance https://dev2.mosip.net/v1/auditmanager/swagger-ui/index.html?configUrl=/v1/auditmanager/v3/api-docs/swagger-config
.
The API's can be tried using postman. URLs and Body structures can be found in Swagger or curl command can be copied and imported in Postman.
Demographic data normalization is the process of applying rules for formatting of the demographic data (such as the address) into a common format before demographic data matching is verified during the demographic authentication in IDA. For example, for address lines, the '1st Street' can be replaced with '1 st' and 'C/o' can be removed from both the input and database data before the match is verified. These rules will be different for different languages, and may be configured/implemented differently.
The ID-Authentication Demographic data normalization mentioned here is specific to the Demo-SDK reference implementation of the Kernel Demographic API. It takes the below configuration to apply the name and address normalization rules.
For any other custom implementation of the normalization, the Demo-SDK needs to be implemented accordingly.
The below configuration is used to define the separator for normalizing regex (pattern) and the replacement word. The default is set to '='.
ida.norm.sep==
The format for configuring the name/address normalization rules for any language is given below:
ida.demo.<name/address/common>.normalization.regex.<languageCode/any>[<sequential index starting from 0>]=<reqular expression>${ida.norm.sep}<replacement string>
If replacement string is not specified, the regular expression will be replaced with empty string.
Note: It is recommended that the sequence is not broken in the middle otherwise all normalization properties will not be read for the particular type.
For non-english languages, the non-english words needs to be converted into UTF-16 and then copied to the configuration. For example, convert the Unicode characters to UTF-16.
Before conversion: ida.demo.address.normalization.regex.hin[0]=पहली${ida.norm.sep}पहला
After conversion: ida.demo.address.normalization.regex.hin[0]=\u092a\u0939\u0932\u0940${ida.norm.sep}\u092a\u0939\u0932\u093e
ID Authentication is built as an independent service that can be seeded with data for authentication by any system, including MOSIP. In the current design, we can have multiple IDA modules running from a single issuer.
The ID Authentication (IDA) module of MOSIP consists of the following services:
Authentication Services
OTP Service
Internal Services
The services mentioned below are used by Authentication or e-KYC Partners.
Authentication Service: used to authenticate an individual's UIN/VID using one or more authentication types.
KYC Authentication Service: used to request e-KYC for an individual's UIN/VID using one or more authentication types.
OTP Request Service is used by Authentication/e-KYC Partners to generate OTP for an individual's UIN/VID. The generated OTP is stored in IDA DB for validation during OTP Authentication.
Internal Authentication Service - The authentication service used by internal MOSIP modules such as Resident Service, Registration Processor and Registration Client to authenticate individuals.
Internal OTP Service - used by Resident Service to generate OTP for an Individual for performing OTP Authentication.
Authentication Transaction History Service - used by Resident Service to retrieve a paginated list of authentication and OTP Request transactions for an individual.
ID Authentication uses the credential data of the individuals for performing authentication.
This credential is requested by ID Repository upon any UIN insertion/update or VID creation.
WebSub invokes the credential-issuance callback in ID Authentication where the credential data is downloaded from Datashare and then stored in IDA DB.
ID Authentication needs the below keys to be generated during the deployment for usage in Authentication Service.
IDA IDENTITY_CACHE
(K18) symmetric key to encrypt and decrypt the Zero-knowledge 10K random keys
IDA ROOT
master key(K15)), IDA module
master key(K16), IDA-SIGN
master key
Base keys CRED_SERVICE
(K22), IDA-FIR
(K21), INTERNAL
(K19), PARTNER
(K20)
This is a reference application to demonstrate how authentication and KYC can be performed by Authentication Partners.
Refer to the repository for more details.
Below is the sample authentication demo UI image.
Refer to ID Authentication Configuration Guide.
To know more about the developer setups, read:
Refer API Documentation.
The services mentioned below are used by Authentication/e-KYC Partners.
Authentication service- used to authenticate an individual's UIN/VID using one ore more authentication types.
KYC Authentication service- used to request e-KYC for an individul's UIN/VID using one ore more authentication types.
The documentation here will guide you through the prerequisites required for the developer' setup.
Below are a list of tools required in ID Repository Services:
JDK 11
Any IDE (like Eclipse, IntelliJ IDEA)
Apache Maven (zip folder)
pgAdmin
Postman
Git
Notepad++ (optional)
lombok.jar (file)
settings.xml (document)
Follow the steps below to set up ID Repository Services on your local system:
1. Download lombok.jar
and settings.xml
from here.
2. Unzip Apache Maven and move the unzipped folder in C:\Program Files
and settings.xml
to "conf" folder C:\Program Files\apache-maven-3.8.4\conf
.
3. Install Eclipse, open the lombok.jar
file and wait for some time until it completes the scan for Eclipse IDE and then click Install/Update
.
4. Check the Eclipse installation folder C:\Users\userName\eclipse\jee-2021-12\eclipse
to see if the lombok.jar
is added. By doing this, you don't have to add the dependency of lombok
in your pom.xml
file separately as it is auto-configured by Eclipse.
5. Configure the JDK (Standard VM) with your Eclipse by traversing through Preferences → Java → Installed JREs
.
For the code setup, clone the repository and follow the guidelines mentioned in the Code Contributions.
Open the project folder where pom.xml
is present.
Open command prompt from the same folder.
Run the command mvn clean install -Dgpg.skip=true
to build the project and wait for the build to complete successfully.
After building of a project, open Eclipse and select Import Projects → Maven → Existing Maven Projects → Next → Browse to project directory → Finish
.
After successful importing of project, update the project by right-click on Project → Maven → Update Project
.
1. For the environment setup, you need an external JAR that is available here with different versions. (E.g.: You can download kernel-auth-adapter.jar
and add to project Libraries → Classpath → Add External JARs → Select Downloaded JAR → Add → Apply and Close
).
2. Clone mosip-config repository.
3. Create an empty folder inside the mosip-config
with sandbox-local
name and then copy and paste all config files inside sandbox-local
folder except .gitignore, README and LICENSE
.
4. As ID Authentication is using two properties files, id-authentication-default
and application-default
, you will have to configure them according to your environment. The same files are available here for reference.
Properties to be updated:
application-default.properties
mosip.mosip.resident.client.secret = <current_password>
.
db.dbuser.password=<password>
.
mosip.kernel.xsdstorage-uri=file:///home/user/Desktop/tspl/mosip-config/sandbox-local/
(i.e. sandbox-local
folder location).
Comment this out auth.server.admin.issuer.internal.uri
in application-default.properties
file because you already have this auth.server.admin.issuer.uri
, and hence there is no need of auth.server.admin.issuer.internal.uri
.
mosip.identity.mapping-file=<Path_to_identity_mapping_json_file>
. (For Example: file:///home/user/Desktop/tspl/mosip-config/sandbox-local/identity-mapping.json
)
id-authentication-default.properties
......
......
5. To run the server, two files are required- kernel-config-server.jar and config-server-start.bat.
6. Put both the files in the same folder and change the location attribute to sandbox-local
folder in config-server-start.bat
file and also check the version of kernel-config-server.jar
towards the end of the command.
Example:
java -jar -Dspring.profiles.active=native -Dspring.cloud.config.server.native.search-locations=file:C:\Users\myDell\mosipProject\mosip-config\sandbox-local -Dspring.cloud.config.server.accept-empty=true -Dspring.cloud.config.server.git.force-pull=false -Dspring.cloud.config.server.git.cloneOnStart=false -Dspring.cloud.config.server.git.refreshRate=0 kernel-config-server-1.2.0-20201016.134941-57.jar
.
7. Run the server by opening the config-server-start.bat
file.
The server should now be up and running.
Below are the configurations to be done in Eclipse:
1. Open Eclipse and run the project for one time as Java application
, so that it will create a Java application which you can see in debug configurations and then change its name. (e.g.: project name with environment - "Auth-Service-Dev").
2. Open the arguments and pass this -Ddomain.url=dev.mosip.net -Dapplication.base.url=http://localhost:8090 -Dspring.profiles.active=default -Dspring.cloud.config.uri=http://localhost:51000/config -Dspring.cloud.config.label=master
in VM arguments.
3. Here, the domain URL represents the environment on which you are working (eg., it can be dev2.mosip.net
or qa3.mosip.net
).
4. Click Apply and then debug it (starts running).
For API documentation, refer here.
OTP Request Service is used by Authentication/e-KYC Partners to generate OTP for an individual's UIN/VID. The generated OTP is stored in IDA DB for validation during OTP Authentication.
The documentation here will guide you through the prerequisites required for the developer' setup.
Below are a list of tools required in ID Repository Services:
JDK 11
Any IDE (like Eclipse, IntelliJ IDEA)
Apache Maven (zip folder)
pgAdmin
Postman
Git
Notepad++ (optional)
lombok.jar (file)
settings.xml (document)
Follow the steps below to set up ID Repository Services on your local system:
1. Download lombok.jar
and settings.xml
from here.
2. Unzip Apache Maven and move the unzipped folder in C:\Program Files
and settings.xml
to "conf" folder C:\Program Files\apache-maven-3.8.4\conf
.
3. Install Eclipse, open the lombok.jar
file and wait for some time until it completes the scan for Eclipse IDE and then click Install/Update
.
4. Check the Eclipse installation folder C:\Users\userName\eclipse\jee-2021-12\eclipse
to see if the lombok.jar
is added. By doing this, you don't have to add the dependency of lombok
in your pom.xml
file separately as it is auto-configured by Eclipse.
5. Configure the JDK (Standard VM) with your Eclipse by traversing through Preferences → Java → Installed JREs
.
For the code setup, clone the repository and follow the guidelines mentioned in the Code Contributions.
Open the project folder where pom.xml
is present.
Open command prompt from the same folder.
Run the command mvn clean install -Dgpg.skip=true
to build the project and wait for the build to complete successfully.
After building of a project, open Eclipse and select Import Projects → Maven → Existing Maven Projects → Next → Browse to project directory → Finish
.
After successful importing of project, update the project by right-click on Project → Maven → Update Project
.
1. For the environment setup, you need an external JAR that is available here with different versions. (E.g.: You can download kernel-auth-adapter.jar
and add to project Libraries → Classpath → Add External JARs → Select Downloaded JAR → Add → Apply and Close
).
2. Clone mosip-config repository.
3. Create an empty folder inside the mosip-config
with sandbox-local
name and then copy and paste all config files inside sandbox-local
folder except .gitignore, README and LICENSE
.
4. As ID Authentication is using two properties files, id-authentication-default
and application-default
, you will have to configure them according to your environment. The same files are available here for reference.
Properties to be updated:
application-default.properties
mosip.mosip.resident.client.secret = <current_password>
.
db.dbuser.password=<password>
.
mosip.kernel.xsdstorage-uri=file:///home/user/Desktop/tspl/mosip-config/sandbox-local/
(i.e. sandbox-local
folder location).
Comment this out auth.server.admin.issuer.internal.uri
in application-default.properties
file because you already have this auth.server.admin.issuer.uri
, and hence there is no need of auth.server.admin.issuer.internal.uri
.
mosip.identity.mapping-file=<Path_to_identity_mapping_json_file>
. (For Example: file:///home/user/Desktop/tspl/mosip-config/sandbox-local/identity-mapping.json
)
id-authentication-default.properties
......
......
5. To run the server, two files are required- kernel-config-server.jar and config-server-start.bat.
6. Put both the files in the same folder and change the location attribute to sandbox-local
folder in config-server-start.bat
file and also check the version of kernel-config-server.jar
towards the end of the command.
Example:
java -jar -Dspring.profiles.active=native -Dspring.cloud.config.server.native.search-locations=file:C:\Users\myDell\mosipProject\mosip-config\sandbox-local -Dspring.cloud.config.server.accept-empty=true -Dspring.cloud.config.server.git.force-pull=false -Dspring.cloud.config.server.git.cloneOnStart=false -Dspring.cloud.config.server.git.refreshRate=0 kernel-config-server-1.2.0-20201016.134941-57.jar
.
7. Run the server by opening the config-server-start.bat
file.
The server should now be up and running.
Below are the configurations to be done in Eclipse:
1. Open Eclipse and run the project for one time as Java application
, so that it will create a Java application which you can see in debug configurations and then change its name. (e.g.: project name with environment - "Auth-Otp-Service-Dev").
2. Open the arguments and pass this -Ddomain.url=dev.mosip.net -Dapplication.base.url=http://localhost:8090 -Dspring.profiles.active=default -Dspring.cloud.config.uri=http://localhost:51000/config -Dspring.cloud.config.label=master
in VM arguments.
3. Here, the domain URL represents the environment on which you are working (eg., it can be dev2.mosip.net
or qa3.mosip.net
).
4. Click Apply and then debug it (starts running).
For API documentation, refer here.
Authentication Partner/Relying Party | Entities that use MOSIP for authentication like banks, telecom, Govt. institutes etc. |
|
Online Verification Partner |
|
Credential Partner | Provider of credentials like printed ID card, QR code etc. to residents |
|
Device Provider | Provider of biometric devices that connect to registration client and authentication apps |
|
FTM Provider |
|
Manual Adjudication | Providers of Manual Adjudication Systems(MAS); enrollment data is shared with MAS |
|
ABIS Partner |
|
MISP Partner | MOSIP Infra Service Provider (MISP) provide network infrastructure/channel/pipe to various Authentication Partners to connect to the MOSIP system. Example, broadband service providers. |
|
Internal Authentication Service: The authentication service used by internal MOSIP modules such as Resident Service, Registration Processor, and Registration Client to authenticate individuals.
Internal OTP Service: used by Resident Service to generate an OTP for an Individual for performing OTP Authentication.
Authentication Transaction History Service: used by Resident Service to retrieve a paginated list of authentication and OTP Request transactions for an individual.
The documentation here will guide you through the prerequisites required for the developer's setup.
Below is a list of tools required in ID Repository Services:
JDK 11
Any IDE (like Eclipse or IntelliJ IDEA)
Apache Maven (zip folder)
pgAdmin
Postman
Git
Notepad++ (optional)
lombok.jar (file)
settings.xml (document)
Follow the steps below to set up ID Repository Services on your local system:
2. Unzip Apache Maven and move the unzipped folder in C:\Program Files
and settings.xml
to "conf" folder C:\Program Files\apache-maven-3.8.4\conf
.
3. Install Eclipse, open the lombok.jar
file, wait for some time until it completes the scan for the Eclipse IDE, and then click Install/Update
.
4. Check the Eclipse installation folder C:\Users\userName\eclipse\jee-2021-12\eclipse
to see if the lombok.jar
is added. By doing this, you don't have to add the dependency of lombok
in your pom.xml
file separately, as it is auto-configured by Eclipse.
5. Configure the JDK (Standard VM) with your Eclipse by traversing through Preferences → Java → Installed JREs
.
Open the project folder where pom.xml
is present.
Open the command prompt from the same folder.
Run the command mvn clean install -Dgpg.skip=true
to build the project and wait for the build to complete successfully.
After building a project, open Eclipse and select Import Projects → Maven → Existing Maven Projects → Next → Browse to project directory → Finish
.
After successfully importing of project, update the project by right-clicking on Project → Maven → Update Project
.
3. Create an empty folder inside the mosip-config
with sandbox-local
name and then copy and paste all config files inside sandbox-local
folder except .gitignore, README and LICENSE
.
Properties to be updated:
application-default.properties
mosip.mosip.resident.client.secret = <current_password>
.
db.dbuser.password=<password>
.
mosip.kernel.xsdstorage-uri=file:///home/user/Desktop/tspl/mosip-config/sandbox-local/
(i.e. sandbox-local
folder location).
Comment this out auth.server.admin.issuer.internal.uri
in application-default.properties
file because you already have this auth.server.admin.issuer.uri
, and hence there is no need for auth.server.admin.issuer.internal.uri
.
mosip.identity.mapping-file=<Path_to_identity_mapping_json_file>
. (For Example: file:///home/user/Desktop/tspl/mosip-config/sandbox-local/identity-mapping.json
)
id-authentication-default.properties
......
......
6. Put both the files in the same folder and change the location attribute to sandbox-local
folder in config-server-start.bat
file and also check the version of kernel-config-server.jar
towards the end of the command.
Example:
java -jar -Dspring.profiles.active=native -Dspring.cloud.config.server.native.search-locations=file:C:\Users\myDell\mosipProject\mosip-config\sandbox-local -Dspring.cloud.config.server.accept-empty=true -Dspring.cloud.config.server.git.force-pull=false -Dspring.cloud.config.server.git.cloneOnStart=false -Dspring.cloud.config.server.git.refreshRate=0 kernel-config-server-1.2.0-20201016.134941-57.jar
.
7. Run the server by opening the config-server-start.bat
file.
The server should now be up and running.
Below are the configurations to be done in Eclipse:
1. Open Eclipse and run the project for one time as Java application
, so that it will create a Java application which you can see in debug configurations and then change its name. (e.g.: project name with the environment - "Auth-Internal-Service-Dev").
2. Open the arguments and pass this -Ddomain.url=dev.mosip.net -Dapplication.base.url=http://localhost:8090 -Dspring.profiles.active=default -Dspring.cloud.config.uri=http://localhost:51000/config -Dspring.cloud.config.label=master
in VM arguments.
3. Here, the domain URL represents the environment on which you are working (eg., it can be dev2.mosip.net
or qa3.mosip.net
).
4. Click Apply and then debug it (starts running).
ID Repository contains the records of identity of an individual, and provides API based mechanism to store, retrieve and update identity details by other MOSIP modules. ID Repository is used by , and .
ID Repository module consists of the following components:
Identity service
VID service
Credential service
Credential Request Generator service
Credential Feeder
Salt generator
Stores, updates, retrieves identity information.
Identity service uses Biometric SDK (server) to extract templates from provided biometric data.
Above is the entity relationship diagram illustrated for Identity service. NOTE: The numbers do not signify sequence of operations or control flow. Arrows indicate the data flow.
Credential request generator issues credentials for new/updated UIN data.
All demographic data of UIN and references to biometric and demographic files stored in object store are stored in mosip_idrepo
DB.
Audit logs are logged into Audit Manager.
Biometric SDK extracts the templates for input biometric data.
Auth Adapter integrates with KeyCloak for authentication.
Masterdata service retreives Identity schema based on input schema version.
Kernel ID generator generates UIN.
VID service fetches the list of VIDs associated with UIN to issue credential of update UIN and to create and activate draft VID.
VID Service provides functionality to create/update Virtual IDs mapped against an UIN. It also provides the facility to update status of VID. VIDs are created based on the VID policy defined in the configuration.
Key Manager encrypts/decrypts data.
Credential request generator issues credentials for new/updated UIN data. 3 All VID related data is stored in mosip_idmap
DB.
Partner management service retrieves online verification partners to issue credentials.
Audit logs are logged into Audit Manager.
Auth Adapter integrates with KeyCloak for authentication.
WebSub publish events related to VID updation.
Kernel ID generator generates VID.
Identity service checks the status of UIN to create VID.
Key Manager encrypts/decrypts data and also used to sign data.
WebSub subscribes to get notifications related to credential status from IDA.
Identity service retrieves identity data for UIN/VID.
Partner management service retrieves policies related to credential type and also retrieves policy for bio-extraction.
Auth Adapter integrates with KeyCloak for authentication.
A credential can be defined as any document, object, or data structure that vouches for the identity of a person through some method of trust and authentication. Simply put, a credential is the thing that a person presents—in person or remotely—to say "this is who I am." The types of credentials issued in an ID system vary along multiple dimensions, depending on whether they are physical (i.e., they must be physically carried by a person in order to use them), or digital (i.e., they are machine readable and therefore can be used in a digital environment).
A credential type essentially maps to partner and data share policy.
auth
: Represents individual's data shared with Online Verification Partners (further used for Authentication and eKYC).
qrcode
: qrcode type is used for qrcode partners to issue qrcode related credential data.
euin
: It is used to issue credential data to partners who wish to download euin card using euin policy.
reprint
: Reprint auth type is used for issuing credential information to reprint partners.
vercred
: To issue verifiable credentials to partners, vercred credential type is used.
New credential types may be defined as per needs of a country.
This service creates request for credential issuance.
Key Manager encrypts/decrypts data.
Auth Adapter integrates with KeyCloak for authentication.
This job will feed the existing UIN/ VID identity information to newly deployed IDA instance.
This is a one-time job that populates salts that are used to hash and encrypt data for Identity and VID services. This job must be executed before deploying these services. The following tables are populated:
uin_hash_salt
in mosip_idrepo
DB.
uin_encrypt_salt
in mosip_idmap
DB.
To know more about the developer setups, read:
This service creates request for credential issuance.
Keymanager encrypts/decrypts data.
Auth Adapter integrates with Keycloak for authentication.
The documentation here will guide you through the prerequisites required for the developer' setup.
Below are a list of tools required in ID Repository (Credential Request Generator Service) setup:
JDK 11
Any IDE (like Eclipse, IntelliJ IDEA)
Apache Maven (zip folder)
pgAdmin
Postman
Git
Notepad++ (optional)
lombok.jar (file)
settings.xml (document)
Follow the steps below to set up ID Repository Services on your local system:
2. Unzip Apache Maven and move the unzipped folder in C:\Program Files
and settings.xml
to "conf" folder C:\Program Files\apache-maven-3.8.4\conf
.
3. Install Eclipse, open the lombok.jar
file and wait for some time until it completes the scan for Eclipse IDE and then click Install/Update
.
4. Check the Eclipse installation folder C:\Users\userName\eclipse\jee-2021-12\eclipse
to see if the lombok.jar
is added. By doing this, you don't have to add the dependency of lombok
in your pom.xml
file separately as it is auto-configured by Eclipse.
5. Configure the JDK (Standard VM) with your Eclipse by traversing through Preferences → Java → Installed JREs
.
Open the project folder where pom.xml
is present.
Open command prompt from the same folder.
Run the command mvn clean install -Dgpg.skip=true
to build the project and wait for the build to complete successfully.
After building of a project, open Eclipse and select Import Projects → Maven → Existing Maven Projects → Next → Browse to project directory → Finish
.
After successful importing of project, update the project by right-click on Project → Maven → Update Project
.
3. Create an empty folder inside the mosip-config
with sandbox-local
name and then copy and paste all config files inside sandbox-local
folder except .gitignore, README and LICENSE
.
Properties to be updated:
application-default.properties
mosip.mosip.resident.client.secret = <current_password>
.
db.dbuser.password=<password>
.
mosip.kernel.xsdstorage-uri=file:///home/user/Desktop/tspl/mosip-config/sandbox-local/
(i.e. sandbox-local
folder location).
Comment this out auth.server.admin.issuer.internal.uri
in application-default.properties
file because you already have this auth.server.admin.issuer.uri
, and hence there is no need of auth.server.admin.issuer.internal.uri
.
mosip.identity.mapping-file=<Path_to_identity_mapping_json_file>
. (For Example: file:///home/user/Desktop/tspl/mosip-config/sandbox-local/identity-mapping.json
)
id-repository-default.properties
mosip.credential.service.database.hostname
mosip.credential.service.database.port
6. Put both the files in the same folder and change the location attribute to sandbox-local
folder in config-server-start.bat
file and also check the version of kernel-config-server.jar
towards the end of the command.
Example:
java -jar -Dspring.profiles.active=native -Dspring.cloud.config.server.native.search-locations=/home/vipul/Desktop/tspl/mosip-config/sandbox-local -Dspring.cloud.config.server.accept-empty=true -Dspring.cloud.config.server.git.force-pull=false -Dspring.cloud.config.server.git.cloneOnStart=false -Dspring.cloud.config.server.git.refreshRate=0 kernel-config-server-1.2.0-20201016.134941-57.jar
.
7. Run the server by opening the config-server-start.bat
file.
The server should now be up and running.
Below are the configurations to be done in Eclipse:
1. Open Eclipse and run the project for one time as Java application
, so that it will create a Java application which you can see in debug configurations and then change its name. (e.g.: project name with environment - "Credential-request-generator-dev").
2. Open the arguments and pass this -Ddomain.url=dev.mosip.net -Dapplication.base.url=http://localhost:8090 -Dspring.profiles.active=default -Dspring.cloud.config.uri=http://localhost:51000/config -Dspring.cloud.config.label=master
in VM arguments.
3. Here, the domain URL represents the environment on which you are working (eg., it can be dev2.mosip.net
or qa3.mosip.net
).
4. Click Apply and then debug it (starts running).
The APIs can be tested with the help of Swagger-UI.
Swagger is an interface description language for describing restful APIs expressed using JSON. You can access Swagger-UI of credential-request-generator-services for localhost from https://localhost:8094/v1/credentialrequest/swagger-ui/index.html?configUrl=/v1/credentialrequest/v3/api-docs/swagger-config#/
.
Authorised and entrusted partners who host module to provide authentication service to various partners. Even MOSIPs IDA module an is an Online Verification Partner.
Providers of compatible integrated in biometric devices
Provider of
1. Download lombok.jar
and settings.xml
from .
For the code setup, clone the repository and follow the guidelines mentioned in the .
1. For the environment setup, you need an external JAR that is available with different versions. (E.g.: You can download kernel-auth-adapter.jar
and add to the project Libraries → Classpath → Add External JARs → Select Downloaded JAR → Add → Apply and Close
).
2. Clone .
4. As ID Authentication is using two property files, id-authentication-default
and application-default
, you will have to configure them according to your environment. The same files are available for reference.
5. To run the server, two files are required- and .
For API documentation, refer .
Also, retrieves and updates status.
encrypts/decrypts data.
stores/retrieves biometrics and demographic documents.
retrieves online verification partners to issue credentials.
publishes events related to UIN updation and auth type status updates.
creates datashare url for sharable attributes.
Default credential types provided as part of are given below:
These types are defined in of .
In MOSIP sandbox, the job is run .
Refer .
.
1. Download lombok.jar
and settings.xml
from .
For the code setup, clone the repository and follow the guidelines mentioned in the .
1. For the environment setup, you need an external JAR that is available with different versions. (E.g.: You can download kernel-auth-adapter.jar
and add to project Libraries → Classpath → Add External JARs → Select Downloaded JAR → Add → Apply and Close
).
2. Clone .
4. As ID Repository is using two properties files, id-repository-default
and application-default
, you will have to configure them according to your environment. The same files are available for reference.
5. To run the server, two files are required- and .
For API documentation, refer .
The ".well-known" folder is a convention used in web development to provide a standardized location for certain files or resources that need to be publicly accessible and discoverable. It typically resides at the root level of a website or web server. The purpose of this folder is to make it easy for web clients (browsers, applications, or services) to find important files or resources related to web services and security.
MOSIP can use the ".well-known" directory to serve the following purposes:
Standardization: To provide a standardized location for specific public files and resources related to web services and security. It makes it easier for developers and web clients using MOSIP to know where to look for important information.
Security: Security-related files and resources can be placed in the ".well-known" directory, such as the public certificate for encryption and signature verification can be placed here.
Interoperability: By following the ".well-known" convention, web developers using MOSIP can ensure interoperability with various web standards and protocols. For example, MOISP shares the context file which contains the structure of its Verifiable Credentials.
Ease of Configuration: Web servers can be configured to serve files from the ".well-known" directory without needing custom configurations for each specific resource. This simplifies the server setup and maintenance process.
Transparency: For matters related to security policies and contact information, such as in the "security.txt" file, placing them in a well-known location makes it transparent and easily accessible to anyone interested in the website's security practices.
MOSIP's ".well-know" directory contains three files,
controller.json
mosip-context.json
public-key.json
The "controller.json" file is used in MOSIP to share the details of the public key using which a MOSIP-issued verifiable credential can be asserted.
The "mosip-context.json" file contains the schema of the subject in the MOSIP-issued verifiable credential.
The "public-key.json" file contains the public key using which the signature of the MOSIP-issued verifiable credential can be asserted.
Identity Service stores, updates, retrieves identity information.
Also, retrieves and updates UIN status.
The documentation here will guide you through the prerequisites required for the developer' setup.
Below are a list of tools required in ID Repository Services (Identity Service) setup:
JDK 11
Any IDE (like Eclipse, IntelliJ IDEA)
Apache Maven (zip folder)
pgAdmin
Postman
Git
Notepad++ (optional)
lombok.jar (file)
settings.xml (document)
Follow the steps below to set up ID Repository- Identity Services on your local system:
1. Download lombok.jar
and settings.xml
from here.
2. Unzip Apache Maven and move the unzipped folder in C:\Program Files
and settings.xml
to "conf" folder C:\Program Files\apache-maven-3.8.4\conf
.
3. Install Eclipse, open the lombok.jar
file and wait for some time until it completes the scan for Eclipse IDE and then click Install/Update
.
4. Check the Eclipse installation folder C:\Users\userName\eclipse\jee-2021-12\eclipse
to see if the lombok.jar
is added. By doing this, you don't have to add the dependency of lombok
in your pom.xml
file separately as it is auto-configured by Eclipse.
5. Configure the JDK (Standard VM) with your Eclipse by traversing through Preferences → Java → Installed JREs
.
For the code setup, clone the repository and follow the guidelines mentioned in the Code Contributions.
Open the project folder where pom.xml
is present.
Open command prompt from the same folder.
Run the command mvn clean install -Dgpg.skip=true
to build the project and wait for the build to complete successfully.
After building of a project, open Eclipse and select Import Projects → Maven → Existing Maven Projects → Next → Browse to project directory → Finish
.
After successful importing of project, update the project by right-click on Project → Maven → Update Project
.
1. For the environment setup, you need an external JAR that is available here with different versions. (E.g.: You can download kernel-auth-adapter.jar
and add to project Libraries → Classpath → Add External JARs → Select Downloaded JAR → Add → Apply and Close
).
2. Clone mosip-config repository.
3. Create an empty folder inside the mosip-config
with sandbox-local
name and then copy and paste all config files inside sandbox-local
folder except .gitignore, README and LICENSE
.
4. As Id Repository is using two properties files, id-repository-default
and application-default
, you will have to configure them according to your environment. The same files are available here for reference.
Properties to be updated:
application-default.properties
mosip.mosip.resident.client.secret = <current_password>
.
db.dbuser.password=<password>
.
mosip.kernel.xsdstorage-uri=file:///home/user/Desktop/tspl/mosip-config/sandbox-local/
(i.e. sandbox-local
folder location).
Comment this out auth.server.admin.issuer.internal.uri
in application-default.properties
file because you already have this auth.server.admin.issuer.uri
, and hence there is no need of auth.server.admin.issuer.internal.uri
.
mosip.identity.mapping-file=<Path_to_identity_mapping_json_file>
. (For Example: file:///home/user/Desktop/tspl/mosip-config/sandbox-local/identity-mapping.json
)
id-repository-default.properties
mosip.idrepo.db.url
mosip.idrepo.db.port
Comment out all the lines containing mosip.biometric.sdk.providers.finger
, mosip.biometric.sdk.providers.face
and mosip.biometric.sdk.providers.iris
.
Comment out this property mosip.kernel.idobjectvalidator.referenceValidator
.
mosip.idrepo.mosip-config-url=file:///home/user/Desktop/tspl/mosip-config/sandbox-local/
(i.e. sandbox-local
folder location).
5. To run the server, two files are required- kernel-config-server.jar and config-server-start.bat.
6. Put both the files in the same folder and change the location attribute to sandbox-local
folder in config-server-start.bat
file and also check the version of kernel-config-server.jar
towards the end of the command.
Example:
java -jar -Dspring.profiles.active=native -Dspring.cloud.config.server.native.search-locations=file:C:\Users\myDell\mosipProject\mosip-config\sandbox-local -Dspring.cloud.config.server.accept-empty=true -Dspring.cloud.config.server.git.force-pull=false -Dspring.cloud.config.server.git.cloneOnStart=false -Dspring.cloud.config.server.git.refreshRate=0 kernel-config-server-1.2.0-20201016.134941-57.jar
.
7. Run the server by opening the config-server-start.bat
file.
The server should now be up and running.
Below are the configurations to be done in Eclipse:
1. Open Eclipse and run the project for one time as Java application
, so that it will create a Java application which you can see in debug configurations and then change its name. (e.g.: project name with environment - "Identity-Service-dev").
2. Open the arguments and pass this -Ddomain.url=dev.mosip.net -Dapplication.base.url=http://localhost:8090 -Dspring.profiles.active=default -Dspring.cloud.config.uri=http://localhost:51000/config -Dspring.cloud.config.label=master
in VM arguments.
3. Here, the domain URL represents the environment on which you are working (eg., it can be dev2.mosip.net
or qa3.mosip.net
).
4. Click Apply and then debug it (starts running).
For API documentation, refer here.
The APIs can be tested with the help of Swagger-UI.
Swagger is an interface description language for describing restful APIs expressed using JSON. You can access Swagger-UI of identity-services for localhost from http://localhost:8090/idrepository/v1/identity/swagger-ui/index.html?configUrl=/idrepository/v1/identity/v3/api-docs/swagger-config#/
.
VID Service provides functionality to create/update Virtual IDs mapped against an UIN. It also provides the facility to update the status of VID. VIDs are created based on the VID policy defined in the configuration.
The documentation here will guide you through the prerequisites required for the developer' setup.
Below are a list of tools required in ID Repository VID Services setup:
JDK 11
Any IDE (like Eclipse, IntelliJ IDEA)
Apache Maven (zip folder)
pgAdmin
Postman
Git
Notepad++ (optional)
lombok.jar (file)
settings.xml (document)
Follow the steps below to set up ID Repository (VID Services) on your local system:
1. Download lombok.jar
and settings.xml
from here.
2. Unzip Apache Maven and move the unzipped folder in C:\Program Files
and settings.xml
to "conf" folder C:\Program Files\apache-maven-3.8.4\conf
.
3. Install Eclipse, open the lombok.jar
file and wait for some time until it completes the scan for Eclipse IDE and then click Install/Update
.
4. Check the Eclipse installation folder C:\Users\userName\eclipse\jee-2021-12\eclipse
to see if the lombok.jar
is added. By doing this, you don't have to add the dependency of lombok
in your pom.xml
file separately as it is auto-configured by Eclipse.
5. Configure the JDK (Standard VM) with your Eclipse by traversing through Preferences → Java → Installed JREs
.
For the code setup, clone the repository and follow the guidelines mentioned in the Code Contributions.
Open the project folder where pom.xml
is present.
Open command prompt from the same folder.
Run the command mvn clean install -Dgpg.skip=true
to build the project and wait for the build to complete successfully.
After building of a project, open Eclipse and select Import Projects → Maven → Existing Maven Projects → Next → Browse to project directory → Finish
.
After successful importing of project, update the project by right-click on Project → Maven → Update Project
.
1. For the environment setup, you need an external JAR that is available here with different versions. (E.g.: You can download kernel-auth-adapter.jar
and add to project Libraries → Classpath → Add External JARs → Select Downloaded JAR → Add → Apply and Close
).
2. Clone mosip-config repository.
3. Create an empty folder inside the mosip-config
with sandbox-local
name and then copy and paste all config files inside sandbox-local
folder except .gitignore, README and LICENSE
.
4. As Id Repository is using two properties files, id-repository-default
and application-default
, you will have to configure them according to your environment. The same files are available here for reference.
Properties to be updated:
application-default.properties
mosip.mosip.resident.client.secret = <current_password>
.
db.dbuser.password=<password>
.
mosip.kernel.xsdstorage-uri=file:///home/user/Desktop/tspl/mosip-config/sandbox-local/
(i.e. sandbox-local
folder location).
Comment this out auth.server.admin.issuer.internal.uri
in application-default.properties
file because you already have this auth.server.admin.issuer.uri
, and hence there is no need of auth.server.admin.issuer.internal.uri
.
mosip.identity.mapping-file=<Path_to_identity_mapping_json_file>
. (For Example: file:///home/user/Desktop/tspl/mosip-config/sandbox-local/identity-mapping.json
)
id-repository-default.properties
mosip.idrepo.db.url
mosip.idrepo.db.port
Comment out all the lines containing mosip.biometric.sdk.providers.finger
, mosip.biometric.sdk.providers.face
and mosip.biometric.sdk.providers.iris
.
mosip.idrepo.mosip-config-url=file:///home/user/Desktop/tspl/mosip-config/sandbox-local/
(i.e. sandbox-local
folder location).
5. To run the server, two files are required- kernel-config-server.jar and config-server-start.bat.
6. Put both the files in the same folder and change the location attribute to sandbox-local
folder in config-server-start.bat
file and also check the version of kernel-config-server.jar
towards the end of the command.
Example:
java -jar -Dspring.profiles.active=native -Dspring.cloud.config.server.native.search-locations=file:C:\Users\myDell\mosipProject\mosip-config\sandbox-local -Dspring.cloud.config.server.accept-empty=true -Dspring.cloud.config.server.git.force-pull=false -Dspring.cloud.config.server.git.cloneOnStart=false -Dspring.cloud.config.server.git.refreshRate=0 kernel-config-server-1.2.0-20201016.134941-57.jar
.
7. Run the server by opening the config-server-start.bat
file.
The server should now be up and running.
Below are the configurations to be done in Eclipse:
1. Open Eclipse and run the project for one time as Java application
, so that it will create a Java application which you can see in debug configurations and then change its name. (e.g.: project name with environment - "vid-service-dev").
2. Open the arguments and pass this -Ddomain.url=dev.mosip.net -Dapplication.base.url=http://localhost:8090 -Dspring.profiles.active=default -Dspring.cloud.config.uri=http://localhost:51000/config -Dspring.cloud.config.label=master
in VM arguments.
3. Here, the domain URL represents the environment on which you are working (eg., it can be dev2.mosip.net
or qa3.mosip.net
).
4. Click Apply and then debug it (starts running).
For API documentation, refer here.
The APIs can be tested with the help of Swagger-UI.
Swagger is an interface description language for describing restful APIs expressed using JSON. You can access Swagger-UI of resident-services for localhost from http://localhost:8091/idrepository/v1/swagger-ui/index.html?configUrl=/idrepository/v1/v3/api-docs/swagger-config#/
.
This a quick-read user guide for Inji (Resident Mobile Application). To refer to detailed Inji documentation, click here.
Prerequisite: Build Inji
code to generate an apk
file. Transfer the apk
file on to a smart phone on which it is to be installed.
On the Home screen, a few tooltips are displayed after the initial setup to guide the user with the next steps.
It is recommended to keep your digital credentials (ID) with you at all times. To get started with the app, Add IDs to your profile.
There are two ways by which one can download their VC:
Downloading VC using the UIN/ VID feature
Downloading VC using the Application ID feature
Below are the steps to download the credentials using the UIN/ VID option:
Click on the ID card generated to view the following details:
At times, when the resident/user does not have their UIN/ VID, but they possess the Registration ID or PRID, they can make use of this feature to download their digital credentials.
After entering the Application ID, click on Get UIN/VID and follow the steps mentioned in the flow above.
Prerequisites:
Two or more devices with the Resident Mobile app installed are required for sharing credentials.
All required permissions like Bluetooth, location and camera access are enabled on both the devices.
The parties involved are mostly likely to be a Resident who would want to share their credentials with a Relying party (can be with a banker/ health worker/ etc).
Let us understand sharing of credentials with an example. Assuming that a Resident is wants to share their credentials with a Relying/ requesting party on the receiver's phone. The steps that both the parties have to follow is illustrated below:
Enable required permissions on both devices to get started.
Connectivity details displayed on the Sharing and Requesting device.
Once the connection is made, the Sharing ID
screen is displayed on the Resident' phone with the name of the Requesting device. Here, the sharing party can enter the reason for sharing their credentials and tap on Accept request and choose ID
.
Incoming VC on requested device. Relying party can tap Accept request and receive ID
to receive the VC.
Success message displayed on the Sharing and Requesting device after successful VC share.
The History
tab on the devices displays the details of the activities performed.
For instance,
On the Resident' phone, the History
tab displays the history of the shared credentials.
On the Relying party' phone, the History
tab shows the history of the received credentials.
Below are the steps for performing face authentication on the resident's phone using Inji.
Let us assume that the:
QR code on requesting phone
Scanner on Sharing Device
Connectivity details on both the devices : Sharing & Requesting device
Sharing screen after establishing the connection
Incoming VC on Requested Device:
Success pop up after successful share with face authentication on both the devices.
History on both the devices.
Below are the steps for performing face authentication on the requesting phone using Inji.
Let us assume that the:
QR code is on the requesting phone
Scanner on Sharing Device
Connectivity details on both the devices: Sharing and Requesting device
Sharing screen after establishing the connection .
On the Requesting Device, following is displayed:
Success pop up after successful share with face auth on both the devices.
History on both the devices.
After successful download of VC, residents will have an option to activate VC for online login.
Resident clicks on Activation pending for online login
in the home page.
Resident will be taken to a detailed view page where an option to Activate
the VC is available.
Further, the resident can click on Activate
button. They will get an alert message after which they can click on Yes, I confirm
button.
The resident needs to enter the OTP sent on their registered number.
After the successful OTP match, VC will be activated for online login with the icon changing to green as shown below.
Also, the same is reflected on the home page.
The Hardware Security Module (HSM) is a highly secure physical device specifically designed and used for cryptographic processing and strong authentication. It encrypts, decrypts, creates, stores, and manages digital keys and is used for signing and authentication. HSMs may be accessed via PKCS11 and JCE interfaces.
To simulate HSM, the default sandbox installation uses . SoftHSM supports PKCS11 but not JCE.
JCE is a Java keystore class implementation that connects to HSMs. HSM vendors should provide JCE support.
MOSIP highly recommends the following specifications for HSM:
Must support cryptographic offloading and acceleration.
Should provide authenticated multi-role access control.
Must have a strong separation of administration and operator roles.
Capability to support client authentication.
Must have secure key wrapping, backup, replication, and recovery.
Must support 2048, 4096-bit RSA private keys, and 256-bit AES keys on the FIPS 140-2 Level 3 Certified Memory of the Cryptographic Module.
Must support at least 10000+ 2048 RSA private keys on FIPS 140-2 Level 3 Certified Memory of the Cryptographic Module.
Must support clustering and load balancing.
Should support the cryptographic separation of application keys using logical partitions.
Must support M of N multi-factor authentication.
PKCS#11, OpenSSL, Java (JCE), Microsoft CAPI, and CNG
Minimum dual Gigabit Ethernet ports (to service two network segments) and, optionally, 10G Fibre ports could be available.
Asymmetric public key algorithms: RSA, Diffie-Hellman, DSA, KCDSA, ECDSA, ECDH, and ECIES
Symmetric algorithms: AES, ARIA, CAST, HMAC, SEED, Triple DES, DUKPT, and BIP32
Hash/message digest: SHA-1, SHA-2 (224, 256, 384, 512 bits).
Full Suite B implementation with fully licensed ECC, including Brainpool, custom curves, and safe curves
Safety and environmental compliance
Compliance with UL, CE, and FCC Part 15 Class B.
Compliance with RoHS2 and WEEE.
Management and monitoring
Support remote administration —including adding applications, updating firmware, and checking the status— from NoC.
Syslog diagnostics support
Command Line Interface (CLI) or Graphical User Interface (GUI)
Support the SNMP monitoring agent.
Physical characteristics
Standard 1U 19-inch rack mount with integrated PIN ENTRY Device or Smart Card or any equivalent security.
Performance
RSA 2048 signing performance: 10,000 per second.
RSA 2048 key generation performance: 10+ per second.
RSA 2048 encryption or decryption performance: 20000+ per second.
RSA 4096 signing performance: 2000+ per second.
RSA 4096 key generation performance: 2+ per second.
RSA 4096 encryption or decryption performance: 20000+ per second.
Should be able to backup keys, replicate keys, and store keys in offline locker facilities for DR. The total capacity is in line with the total number of keys prescribed.
Clustering minimum of 20 HSMs.
Less than 30 seconds for key replication across the cluster.
A minimum of 30 logical partitions and their license should be included in the cost.
Keystore
mosip.kernel.keymanager.hsm.keystore-type
mosip.kernel.keymanager.hsm.config-path
mosip.kernel.keymanager.hsm.keystore-pass
JCE
mosip.kernel.keymanager.hsm.jce.className
mosip.kernel.keymanager.hsm.jce.keyStoreType
mosip.kernel.keymanager.hsm.jce.keyStoreFile
mosip.kernel.keymanager.hsm.jce.<ANY_OTHER_PARAM_01>
mosip.kernel.keymanager.hsm.jce.<ANY_OTHER_PARAM_02>
Introduction
Modular Open Source Identity Platform (MOSIP) integrates a suite of Mock Services designed to emulate key functionalities of MOSIP services within the framework. In the development, testing, and demonstration phases, Mock Services will make available a controlled environment to evaluate MOSIP features. Developers and testers can refer to this documentation to gain a comprehensive understanding of the structure and functionality of each Mock Service within the MOSIP framework.
Note:
The scope of the services are updated as needed. Please refer to the current release documents.
Mock Services do not intend to replace production systems. Mock Services enables evaluation during the development and testing phases.
This document details each of the Mock Services and explains its significance within the MOSIP architecture.
Below mentioned are the current set of mock services available in MOSIP.
Simulates device services for testing, authentication, and delete registration functionalities.
Allows developers to interact with a device-service environment without a physical device.
Mock MV (Manual Verification)
Reproduces the manual verification process for testing and validation purposes.
Enables the testing of manual verification workflows without human intervention.
Simulates the functionality of the Automated Biometric Identification System (ABIS).
Facilitates testing of biometric matching, searching, and integrating with ABIS without accessing production data.
Maintains resident biometric uniqueness through de-duplication.
Interfaces with MOSIP via message queues in JSON format.
Supports 1:N de-duplication and adheres to Specifications.
Replicates MOSIP's Biometric Software Development Kit (SDK) for testing and debugging.
Allows developers to integrate biometric functionalities into applications without connecting to a physical device.
Used for 1:N match, quality, and extraction, etc.
Simulation is available as Mock BioSDK, installed in the MOSIP sandbox.
Exposes REST APIs for 1:1 match and quality check at the MOSIP backend.
Simulates an SMTP server for testing email notifications without sending actual emails.
Enables the testing of communication workflows and email content.
Mock SMTP Server
Is installed as part of the default MOSIP installation.
Mimics real SMTP server behavior for testing and development purposes.
Faster Development and Testing: Enables rapid development and testing cycles without need to access to production systems.
Reduced Costs: Avoids the need for production resources, lowering development and testing costs.
Controlled Environment: Testing with Mock Services provides consistent and predictable behavior, ideal to isolate and troubleshoot.
Data Privacy: Sensitive data remains secure, as development and testing occur with mock data.
MOSIP uses Mock Services in the following modules:
Mentioned below are the services utilized by the Registration Processor module to facilitate the functions.
Mock MV (Manual Verification):
ID Authentication:
To get an overview of Key Manager, refer .
Below is a list of tools required in Key Manager:
JDK 11
Any IDE (like Eclipse, IntelliJ IDEA)
Apache Maven (zip folder)
PostgreSQL
Any DB client (like DBeaver, pgAdmin)
Postman (any HTTP Client)
Git
Any Editor (like Vscode, Notepad++ etc optional)
lombok.jar (jar file)
settings.xml (document)
1. Download and .
2. Unzip Apache Maven and move settings.xml
to "conf" folder <apache maven unzip path>\conf
.
4. Check the Eclipse installation folder to see if the lombok.jar
is added.
5. Configure the JDK (Standard VM) with your Eclipse by traversing through Preferences → Java → Installed JREs
.
Open the project folder where pom.xml
is present.
Open the command prompt from the same folder.
Run the command mvn clean install -Dgpg.skip=true -DskipTests=true
to build the project.
After building, open Eclipse and select Import Projects → Maven → Existing Maven Projects → Next → Browse to project directory → Finish
.
After successful importing of project, update the project by right-click on Project → Maven → Update Project
.
Key Manager uses two property files, kernel-default
and application-default
, configure them accordingly. For instance,
Key Manager needs a Keystore to store keys. Supported Keystore types: PKCS11, PKCS12, Offline, JCE.
Update URL's in property files.(It can be either pointed to any remotely or locally deployed services)
Run the server by opening the config-server-start.bat
file.
To verify the config-server, hit the below URL:
http://localhost:51000/config/{spring.profiles.active}/{spring.cloud.config.name}/{spring.cloud.config.label}
for instance http://localhost:51000/config/kernel/env/master
.
Key Manager REST service consists of bootstrap.properties
file in src/main/resources
.
Below properties needed to be modified in order to connect to the config server:
Services can be run using Run As -> Spring Boot App/Java Application
.
The API's can be tried with the help of Swagger-UI and Postman.
Swagger-UI service can be accessed from (https/http)://(<domain>/<host>:<port>)/<context-path>/swagger-ui/index.html?configUrl=<contect-path>/v3/api-docs/swagger-config
for instance https://dev2.mosip.net/v1/auditmanager/swagger-ui/index.html?configUrl=/v1/keymanager/v3/api-docs/swagger-config
.
The API's can be tried using Postman. URLs and Body structures can be found in swagger or curl command can be copied and imported in Postman.
The Key Manager Service provides secure storage, provisioning and management of secret data. It provides all the cryptographic operations like encryption/decryption and digital signature/verification making one trust store for all partner trust path validation. It manages the lifecycle of encryption/decryption keys, including generation, distribution, administration, and deletion.
This includes keying material such as symmetric keys, asymmetric keys, certificates and algorithm data. It is a web-based key management solution that helps consolidate, control, manage, monitor, all key generation and maintenance of key life cycle required in MOSIP.
Key Manager interfaces with key store like and .
RSA-2048 for all data encryption
The keys are identified as tuple of app_id
and ref_id
.
app_id
(or applicationId
): Typically, module name e.g. REGISTRATION
.
ref_id
(or referenceId
): Specified only for Base keys (except SIGN*). Eg. 10001_110011
* SIGN
: TBD
Root and Module keys are generated by any one of these methods:
Using Key Generator job or
After the deployment, the initial set of pre-requisite keys has to be generated by the Administrator to complete the Key Manager setup. This generation is a one-time activity, and afterwards, the Key Manager will auto-generate all the required Root key and Module master keys upon expiry of key duration.
Base keys are auto-generated (and updaded on expiry) - the administrator is not required to request for generation. The keys reside in the DB. A new key pair is generated if not found in the DB.
Random AES 256-bit key will be generated, generated random key will be used to encrypt the actual registration packet.
Random generated key will be encrypted using the certificate received from server. Certificate contains RSA 2048 bit key.
Certificate Thumbprint will be computed.
Thumbprint will be prepend to encrypted random key for key identification.
Finally, the encrypted random key with prepended thumbprint will be concated with encrypted registration packet using #KEY_SPLITTER# as separator.
Registration packet data will be split to get the encrypted random key, encrypted registration data, certificate thumbprint.
Identifies the respective private key to decryption process.
Identified private key will be decrypted with the mapped master key.
Decrypted private key will be used to decrypt the encrypted random key.
Decrypt the registration packet using the decrypted random key.
Returns the decrypted data to REG_PROC.
Sync Data service requests Key Manager service to provide the reg-client specific certificate. Key identifier will be APP_ID - REGISTRATION, REF_ID - CENTER-ID_MACHINE-ID.
Key Manager service generate a new key pair, encrypts the private key with REGISTRATION master key and creates a new certificate using same master.
Returns the certificate to Sync data service. If key pair is already available and is valid, returns the available certificate.
Sync data service sends the certificate to Registration Client.
The registration packet will be encrypted using the certificate received from the server after collecting all the required data for registration, including adding the digital signatures required to the registration data, and before saving/writing the data on the Registration Client hard-disk.
REG_PROC sends request to decrypt the data to Key Manager service with same app_id and ref_id.
Mock MDS ():
The module interacts with Mock MDS to capture biometric data during the registration.
Mock SDK ():
The module interacts with mock SDK to perform 1: N match for biometrics, extract biometric template, and check the quality of the captured biometrics.
Mock Services help to process packets by providing support to emulate key functionalities such as search for the duplicate biometric data, to perform manual verification, and to check the quality of the captured biometric data.
Mock ABIS ():
module interacts with mock ABIS for testing matching performance and error handling.
module interacts with Mock MV to process the packets that are marked for manual verification.
Mock SDK ():
module interacts with Mock SDK to check the quality of the captured biometrics and for authentication purposes.
module also utilizes the mock services during development, testing, and demonstration phases. It uses Mock SDK to carry out the biometric authentication.
3. Install Eclipse, open the lombok.jar
file and then click Install/Update
.
For the code setup, clone the repository and follow the guidelines mentioned in the .
Download and add to project Libraries → Classpath → Add External JARs → Select Downloaded JAR → Add → Apply and Close
.
Clone .
Refer to deploy local DB.
Secrets can be encrypted using .
Download . For Windows, download , Linux users can run java -jar -Dspring.profiles.active=native -Dspring.cloud.config.server.native.search-locations=file:{mosip-config-mt_folder_path}/config -Dspring.cloud.config.server.accept-empty=true -Dspring.cloud.config.server.git.force-pull=false -Dspring.cloud.config.server.git.cloneOnStart=false -Dspring.cloud.config.server.git.refreshRate=0 {jarName}
.
For API documentation, refer .
AES-256 for
Root and Module keys reside in HSM while Base key pair reside in the DB encrypted by Module keys. All references (aliases) containing metadata of keys are present in table. The table contains encrypted Base keys.
Using Key Manager option in .
The default validity of the keys may be modified by updating before generating keys.
You can revoke Root or Module key by invoking API with force attribute as true. API invalidates existing key and immediately generates new key.
You can revoke Base key by invoking API with the respective applicationId
and referenceId
.
sends request to sync data service for the client configuration data.
To know more about the developer setup, read .
Refer .
.
SNo. | Partners | Application ID | ReferenceID | Partner Domain | Partner Type Code |
---|
S No. | Key | Key type | Objects | Storage | Generated by | Comment |
---|
PK1 | ABIS | PARTNER | mpartner-default-abis (or partner ID) | AUTH | ABIS_Partner |
PK2 | Device Providers | PARTNER | Partner ID | DEVICE | Device_Provider |
PK3 | Print Service Provider | PARTNER | mpartner-default-print (or partner ID) | AUTH | Credential_Partner |
PK4 | Auth Providers or Relying Party | PARTNER | Partner ID | AUTH | Auth_Partner |
PK5 | FTM Providers (per Chip Model) | PARTNER | Partner ID | FTM | FTM_Provider |
PK6 | MISP | PARTNER | Partner ID | AUTH | MISP_Partner |
PK7 | Manual Adjudicator | PARTNER | mpartner-default-manual-adjudication (or partner ID) | AUTH | Manual_Adjudication |
PK8 | IDA system | PARTNER | mpartner-default-auth (or partner ID) | AUTH | Online_Verification_Partner |
PK9 | Resident Services | PARTNER | mpartner-default-resident (or partner ID) | AUTH | Credential_Partner |
Partner Management Services (PMS) module provides the following services:
Partner Management Service
Policy Management Service
For an overview of role of partners in MOSIP, refer here.
Provides various partner services like onboarding partners and providing partner data to other modules.
The diagram below illustrates the relationship of this service to other MOSIP services.
Certificates of partner are uploaded to Key Manager as part of onboarding.
Registration processor fetches ABIS datashare policy from PMS.
PMS sends notification messages to partners via notification service (of Kernel).
Audit logs are logged into Auditmanager.
ID Repository fetches credential data share partners and their polices from PMS.
All PMS data is stored in mosip_pms
DB.
Certificates of Authentication Partners are send to IDA module as IDA runs independently. The certs are shared using Datashare (which futher uses Websub to share data with IDA).
This service manages partner policies.
Audit logs are logged into Auditmanager.
All policies are stored stored in mosip_pms
DB.
Datashare service fetches partner policies and shares data with partners accordingly.
To know more about the partner portal, refer Partner Management portal.
To know more about the developer setup, read Partner Management Services Developers Guide.
Refer API Documentation.
Packet Manager performs the following functions:
Reads/writes registration packets from/to Object Store.
Performs in-memory encryption and decryption of packets.
Performs security checks, checksum, file validations, ID object validations etc. on the registration packet.
Provides packet information to other services via APIs. In case of multiple packets associated with an ID, pulls information from packets based on configured priority. (See packetmanager.default.priority
).
Packet Manager runs as a service and is also available as a library.
The relationship of Packet Manager with other services is explained here. NOTE: The numbers do not signify sequence of operations or control flow. Arrows indicate data flow.
Resident Services uses Packet Manager library to create packet.
Registration Processor reads packet data using Packet Manager service.
Packets are stored and retrieved from Object Store.
Audit logs.
Encryption and decryption of packet.
Registration Client uses Packet Manager library to create packets.
Refer Registration Packet Structure.
Refer API Documentation.
Properties that are shared across all modules are written in the file application-default.properties
.
Module-specific properties are written in respective *.properties
files in mosip-config
Config server is a Spring Cloud Config Server that serves the above properties to modules during run-time. The property files are downloaded when an application starts. The config server is installed as part of sandbox installation.
Some of the important properties that must be reviewed and modified for a specific deployment are listed below.
Configurations of each MOSIP module will be available here:
Administration
Commons
Datashare
ID Repository
Key Manager
Packet Manager
Partner Management
Pre-registration
Resident Services
Resident Mobile Application
WebSub
Refer to Postgres DB configuration.
Refer to HSM configuration.
Mandatory Languages - Languages that the user has to fill (can be auto translated) during the pre-registration & registration.
Optional Languages - Languages that are not mandatory but provided as a choice to the user.
User selected Language - The language that the user selected at the time of login. The choices shown are union of Mandatory and Optional languages. The labels and alerts will be use the user selected language
Prefered Language - During registration of a registrant (user for whom identity is requested), he can choose his prefered language. This preference use used for all further notification (email, SMS or any other notification).
Languages for the entire system are configured in application prorperties file:
The i18n file for the respective language has to be added to the artifactory Artifactory. The language codes are as per ISO 639-2
A registration packet is a zipped, encrypted file containing ID information of an individual. It contains meta information about operator/supervisor, registration center, machine, devices etc.
Example zipped file:
10001100771006920220128223618-10001_10077-20220128223618.zip
Naming convention: appid-refid_timestamp.zip
refid: centerid_machineid
*_id.zip
: Applicant demographic/biometric/document fields which are marked as "pvt" or "kyc" or "none" in ID Schema.
*_id.json
: Meta information (process, encrypted hash, signature, schema version etc.) for *_id.zip
file.
*_evidence.zip
: Applicant's demographic/biometric/document fields which are marked as "evidence" or "none" in ID Schema.
*_evidence.json
: Meta information (process, encrypted hash, signature, schema version etc.) for *_evidence.zip
file.
*_optional.zip
: Applicant demographic/biometric/document fields which are marked as "optional" or "none" ID Schema.
*_optional.json
: Meta information (process, encrypted hash, signature, schema version etc.) for *_optional.zip
file.
Note: this is a sample packet and doesnot mean a particular information will be always available in same packet. The fields are populated based on the fieldCategory set in schema json.
Id
Evidence
Optional
See sample packet.
Key type | Location | Issuer | Purpose | Example | Updation method(on expiry) |
Root | Self signed | Root | Key Generator job or Admin Portal | Automatic | 5 years |
Module | Root | Signing, encryption of Base keys | Key Generator job or Admin Portal | Automatic | 3 years |
Base | Database | Module | Encryption of registration packet etc. | Automatic | Automatic | 2 years |
K1 | Kernel Root | ROOT | - | RSA 2048 | Private key, self signed certificate | HSM-1 | Country | Auto generated by key generator |
K2 | Registration | REGISTRATION | - | RSA 2048 | Private key, certifcate signed by Kernel Root | HSM-1 | Country | Auto generated by key generator |
K3 | PreReg | PRE_REGISTRATION | - | RSA 2048 | Private key, certifcate signed by Kernel Root | HSM-1 | Country | Auto generated by key generator |
K4 | Kernel Sign | KERNEL | SIGN | RSA 2048 | Private key, certifcate signed by Kernel Root | HSM-1 | Country | Auto generated by key generator |
K5 | Registration Processor | REGISTRATION_PROCESSOR | - | RSA 2048 | Private key, certifcate signed by Kernel Root | HSM-1 | Country | Auto generated by key generator |
K6 | PMS | PMS | - | RSA 2048 | Private key, certifcate signed by Kernel Root | HSM-1 | Country | Auto generated by key generator |
K7 | ID Repo | ID_REPO | - | RSA 2048 | Private key, certifcate signed by Kernel Root | HSM-1 | Country | Auto generated by key generator |
K7.1 | ID Repo | ID_REPO | demographic_data | RSA 2048 | Private key, certifcate signed by ID Repo | KeyMgr DB | System | Auto-generated when accessed |
K7.2 | ID Repo | ID_REPO | biometric_data | RSA 2048 | Private key, certifcate signed by ID Repo | KeyMgr DB | System | Auto-generated when accessed |
K7.3 | ID Repo | ID_REPO | identity_data | RSA 2048 | Private key, certifcate signed by ID Repo | KeyMgr DB | System | Auto-generated when accessed |
K7.4 | ID Repo | ID_REPO | uin | RSA 2048 | Private key, certifcate signed by ID Repo | KeyMgr DB | System | Auto-generated when accessed |
K7.5 | ID Repo | ID_REPO | credential_request | RSA 2048 | Private key, certifcate signed by ID Repo | KeyMgr DB | System | Auto-generated when accessed |
K8 | Resident Services | RESIDENT | - | RSA 2048 | Private key, certifcate signed by Kernel Root | HSM-1 | Country | Auto generated by key generator |
K9 | Kernel Identity Cache | KERNEL | IDENTITY_CACHE | AES 256 | Symmetric key | HSM-1 | Country | Auto generated by key generator |
K10 | Registration Client (TPM) | - | - | RSA 2048 | Private key, certificate | Client TPM (private key), Server DB (Certificate) | Registration Client Software | Auto generatde by Registration Client Software in TPM |
K11 | Registration Client Packet Encryption | REGISTRATION | CenterID_MachineID | RSA 2048 | Private key, certificate signed by registration | Server DB (private key), Client DB (Certificate) | System | Auto-generated when accessed |
K12 | Data Share (10000 keys) for zero knowledge encryption | - | - | AES 256 | Symmetric key, encrypted by Kernel Identity Cache | KeyMgr DB | System | Auto generated by key generator |
K13 | CA / Sub-CA certificates | - | - | X.509 | Certificates | PMS DB | CA | Manually Uploaded |
K14 | PARTNER | PartnerID | X.509 | Certificates signed by CA | PMS DB | Partners | Manually Uploaded |
K15 | IDA Root | ROOT | - | RSA 2048 | Private key, self signed certificate | HSM-2 | Country | Auto generated by key generator |
K16 | IDA | IDA | - | RSA 2048 | Private key, certificate signed by IDA Root | HSM-2 | Country/IDA Partner | Auto generated by key generator |
K17 | IDA Sign | IDA | SIGN | RSA 2048 | Private key, certificate signed by IDA Root | HSM-2 | Country | Auto generated by key generator |
K18 | IDA Identity Cache | IDA | IDENTITY_CACHE | AES 256 | Symmetric key | HSM-2 | Country | Auto generated by key generator |
K19 | IDA Internal | IDA | INTERNAL | RSA 2048 | Private key, certificate signed by IDA | IDA DB | System | Auto-generated when accessed |
K20 | IDA Partner | IDA | PARTNER | RSA 2048 | Private key, certificate signed by IDA | IDA DB | System | Auto-generated when accessed |
K21 | IDA FIR | IDA | FIR | RSA 2048 | Private key, certificate signed by IDA | IDA DB | System | Auto-generated when accessed |
K22 | IDA Cred Service | IDA | CRED_SERVICE | RSA 2048 | Private key, certificate signed by IDA | IDA DB | System | Auto-generated when accessed |
DKL0 | RSA 2048 | Private key, self signed certificate | Host machine TPM/key store | Auto generated by SBI Service |
DKL1 | RSA 2048 | Private key, self signed certificate | SBI Service | Auto generated by SBI Service |
FK1 | FTM key | RSA 2048 | Private key, FTM Provider issued certificate | FTM | FTM |
DE1 | Biometric encryption random session key | AES>=256 | No storage, key is created with TRNG/DRBG inside FTM | FTM |
FK2 | Secure boot | RSA>=256 | Private key, self signed certificate | FTM | FTM Provider | Key never leaves FTM |
This guide enables the Foundational Trust providers to use the PMP portal effectively. Below is the workflow:
Partner self-register through the portal.
Partner admin and uploads CA certificate.
Partner admin/ Partner uploads partner certificate.
Partner admin/ Partner creates FTM.
Partner admin/ Partner uploads certificate from the menu before approval/ rejection.
Partner admin approves/ rejects the FTM.
The partner can register themselves on the MOSIP PMP portal by clicking Register on the landing page.
They need to fill up a form with the details below:
First and Last name
Organization Name
Partner type (Device Provider)
Address, e-mail, phone number
Username and password
To view the details entered, click Home to see the dashboard.
The Partner admin needs to upload the CA certificate to enable the partner to use the portal. To do so, the Partner admin:
Clicks Upload CA Certificate option on the left navigation pane of the partner portal.
Selects the Partner Domain as FTM.
Chooses the certificate to upload (only files with extensions such as .cer or .pem).
Clicks Upload.
The uploaded certificates can be viewed by clicking on View Certificates-> View
.
Similarly, the Partner certificates can be added by the Partner admin or partner.
The certificate can be uploaded by clicking Home-> Upload Certificate -> Upload.
The certificate can be viewed by clicking Home-> View Certificate ->View.
The partner can create FTM details by,
Clicking FTM Details -> Create FTM
Fill up the information like Partner Name, Make and Model.
Clicking Save.
The partner can upload FTM certificates by,
Selecting Upload Certificate option from the Actions menu against the FTM created.
Entering the Partner Domain as FTM and choosing the certificate file.
Clicking Upload.
The Partner Admin can choose to approve or reject the FTM certificate uploaded. Below illustrates the workflow:
Finally, you can see the FTM activated.
This guide enables the Device provider partner to use the partner portal effectively. Below is the workflow:
The partner self-registers through the portal.
Partner admin uploads the CA certificate.
Partner admin or Partner uploads the partner certificate.
Partner admin or Partner creates device details.
Partner admin approves or rejects device details.
Partner admin or Partner creates SBI details.
Partner admin approves or rejects SBI details.
Partner admin or Partner maps devices and SBI.
The Device Provider partner can register themselves on the MOSIP PMS portal by clicking Register on the landing page.
They need to fill up a form with the details below:
First and Last name
Organization Name
Partner type (Device Provider)
Address, e-mail, phone number
Username and password
To view the details entered, click Home to see the dashboard.
The Partner admin needs to upload the CA certificate to enable the partner for using the portal. To do so, the Partner admin:
Clicks Upload CA Certificate option on the left navigation pane of the partner portal.
Selects the Partner Domain.
Chooses the certificate to upload (only files with extensions such as .cer or .pem).
Clicks Upload.
The uploaded certificates can be viewed by clicking on View Certificates-> View
.
Similarly, the Partner certificates can be added by the Partner admin/ partner.
The certificate can be uploaded by clicking Home-> Upload Certificate -> Upload.
The certificate can be viewed by clicking Home-> View Certificate ->View.
The partner can add devices to the portal. To do so,
Partner clicks Device details-> Create Device
.
Enters the necessary details to create/add devices like:
Partner Name
Device Type and Sub Type
Make and Model
Click Save.
The Partner Admin can choose to approve/reject the device details entered by the partner.
The Partner can create SBI by filling in the required details.
The Partner Admin can choose to approve/reject the SBI details entered by the partner.
The partner can map the device with an SBI.
Below is the workflow that includes the registration process for an Auth or Credential partner and the steps that need to be followed for using the partner portal.
The partner self-registers through the portal.
Partner selects the relevant Policy Group.
Partner admin uploads the CA certificate.
Partner admin or partner uploads the partner certificate.
Partner admin or Partner maps the Partner Policy.
Partner admin approves or rejects partner policy mapping.
Partner logins after the approval and generates the API key for the approved partner policy mapping using an unique label.
The Auth/ Credential partner can register themselves on MOSIP PMS portal by clicking Register on the landing page.
They need to fill up a form with the details below:
First and Last name
Organization Name
Partner type (Authentication Partner/ Credential Partner)
Address, e-mail, phone number
Username and password
To view the details entered, click Home to see the dashboard.
On successful registration, the partner can see their username displayed on the top right corner.
Partner selects the relevant Policy Group from Map Policy Group dropdown.
Clicks Save.
The Partner admin needs to upload the CA certificate to enable the partner for using the portal. To do so, the Partner admin:
Clicks Upload CA Certificate option on the left navigation pane of the partner portal.
Selects the Partner Domain.
Chooses the certificate to upload (only files with extensions as .cer or .pem).
Clicks Upload.
The uploaded certificates can be viewed by clicking on View Certificates-> View
.
Similarly, the Partner certificates can be added by the Partner admin/ partner.
Once the certificates are uploaded,
Partner maps the policy to the Policy group by clicking on Partner Policy Mapping -> +Map Policy.
Partner enters the Partner Name.
Selects the Auth Policy Name from the dropdown.
Enters a value for the Request Details (unique value) and clicks Save.
Once this is done, you will see a message saying Policy mapping grequest submitted successfully
.
Also, the status is displayed as "In progress" and this means that the partner cannot generate the API key until the request is approved by the Partner admin.
Once the Partner Policy Mapping request is raised by the partner, the Partner admin has the privilege to approve/ reject the mapping. To do so,
Partner admin logs into the PMS portal and clicks on Partner Policy Mapping
in the left navigation pane.
Selects the policy mapping that needs an approval.
From the action menu against the policy mapping, selects Manage Policy.
Clicks Approve.
Once the request is approved, the partner can view the status being updated to Approved
instead of InProgress
.
Partner logins after the Partner Policy Mapping is approved by the Partner admin and generates the API key with an unique label. To do so,
Partner clicks Partner Policy Mapping
on the left navigation pane.
From the actions menu, click Generate API Key.
Partner enters a unique value for the Label
field.
Click Generate.
The API key is generated and can be used by the partner.
The partner can also deactivate a particular API Key by clicking on the cross-mark (X) next to it. Please note, once deactivated, it cannot be activated again. You may need to generate a new API key as per requirement.
Partner policies control the data that needs to be shared with a partner. The policies reside in auth_policy
table of mosip_pms
DB.
Policy type | Partners | Description |
---|---|---|
Policies are not applicable for Device Provider, FTM Provider and MISP Partner as data is not shared with them.
Refer to the default policies loaded while installing MOSIP.
Common policies are grouped example 'Telecom', 'Banking', 'Insurance' etc.
This document defines the APIs specifications for various operations that ABIS can perform to integrate with MOSIP.
API specification version: 0.9
Published Date: February 05, 2021
Publish Date | Revision |
---|
An ABIS system that integrates with MOSIP should support the following operations.
All ABIS operations are via. a message queue and are asynchronous. The data sent in ABIS can be byte array or text based on a configuration in the registration processor.
Common parameters used for all ABIS operations:
The following operations are supported by MOSIP:
referenceId must not be active prior to this operation i.e., it must not have been used before this operation.
De-duplication must not be performed in this operation.
MOSIP will provide biometric data in CBEFF format to ABIS as a response of referenceURL and the data will be encrypted and encoded as mentioned below.
Insert Request
Success Response
Failure Response
The reference URL is MOSIP's datashare URL which is generated based on a policy defined by MOISP's adopter.
The referenceURL is authenticated and authorized; ABIS needs to send a JWT token inside the request header COOKIE.
The referenceURL will be active for a certain time as decided by the MOSIP adopter.
The data sent in the referenceURL will be encrypted.
Authentication Token
As mentioned above in order to access the request URL the ABIS system needs to send a JWT token inside the request header COOKIE. In order to get the token ABIS needs to call MOSIP's AuthN & AuthZ API with Client ID & Secret Key by passing the credentials (clientid, secretkey and appid) which would be provided by the System Integrator (SI).
Sample Request URL
POST https://{base_url}/v1/authmanager/authenticate/clientidsecretkey
Sample Request Body
Sample Response
DataShare URL
Below is the sample API detail for reference URL.
Sample Request URL
GET https://{base_url}/v1/datashare/get/mpolicy-default-abis/mpartner-default-abis/mpartner-default-abismpolicy-default-abis20210205062412BlQo0rJB
Sample Encrypted Response
The structure of the encrypted data downloaded from referenceURL in MOSIP 1.2.0 or later versions
The data downloaded would be URL-safe base64 encoded. Hence, after decoding the data will be in the below format. It will be divided into two Parts after splitting using #KEY_SPLITTER#.
Block 1:
Block 1, i.e. the encrypted key data is again split into three parts,
The 1st part is VER_BYTES (version bytes). The Current version constant is set as VER_R2 and this is present in the first 6 bytes of Block 1.
The 2nd part is the Certificate Thumbprint i.e. the key identifier which is present in the next 32 bytes after VER_BYTES.
The 3rd part is the Encrypted Random AES Key, encrypted with the RSA OAEP - SHA256-MFG1. This constitutes the remaining 256 bytes of Block 1.
Block 2:
Block 2, i.e. the encrypted actual data is again split into two parts,
The 1st part is the random 32 bytes which will be used as AAD in AES encryption(first 32 bytes). From this 32 bytes AAD data, the first 12 bytes is IV/Nonce.
The 2nd part is the encrypted data which is encrypted using AES GCM PKCS5Padding.
Note: In Java 11, PKCS5Padding
serves as a synonym for NoPadding
in GCM mode encryption. Conversely, in Java 17, the synonym PKCS5Padding has been eliminated, and it is now mandatory to exclusively use NoPadding. Consequently, if data is encrypted using PKCS5Padding in Java 11, it will be decrypted with NoPadding in Java 17.
The structure of the encrypted data downloaded from referenceURL in MOSIP 1.1.5.5 or prior versions
The data downloaded would be base64 encoded. Hence, after decoding the data will be in the below format. It will be divided into two Parts after splitting using #KEY_SPLITTER#.
Block 1:
Block 1, i.e. the encrypted key data is again split into two parts,
The first part is the Certificate Thumbprint i.e. the key identifier which is the first 32 bytes in Block 1.
The second part is the Encrypted Random AES Key which is encrypted with RSA OAEP - SHA256-MFG1. This constitutes the remaining 256 bytes of Block 1.
Block 2:
Block 2, i.e. the encrypted actual data is again split into two parts,
The 1st part is the Encrypted data, encrypted using AES GCM PKCS5Padding.
The 2nd part is IV/Nonce i.e. the last 32 bytes appended after encrypted data.
Note: In Java 11, PKCS5Padding
serves as a synonym for NoPadding
in GCM mode encryption. Conversely, in Java 17, the synonym PKCS5Padding has been eliminated, and it is now mandatory to exclusively use NoPadding. Consequently, if data is encrypted using PKCS5Padding in Java 11, it will be decrypted with NoPadding in Java 17.
Sample Response after Decryption
Sample Response in case of Authentication Failure
All Possible Error codes and Messages from Datashare URL
Please note that for all the functional failures MOSIP sends a response code 200.
All Insert requests added to the queue earlier must be serviced by ABIS when performing an Identify request.
Identify request provides a 1:N comparison. The given input is compared either against the gallery passed or if the gallery is not specified the entire database.
The input for comparison can be provided by referenceID or referenceURL.
If the referenceID is given it is used as the preferred option. The given referenceID must be existing in the ABIS database else ABIS will throw an error.
If the referenceID is omitted or NULL and the referenceURL is passed the ABIS retrieves the biometrics provided in the referenceURL and compares the same against either a gallery or its database.
If in case, both referenceID and referenceURL are missing ABIS throws an error.
We are not using the referenceURL in Identify request for our current implementation. Hence, it will be an empty string for Identify request. MOSIP adopters can have customized workflows where the referenceURL can be used.
Identify requests should give to all candidates which are considered as a match based on ABIS thresholds.
This request should not match against referenceID that is not in the reference database.
The response now has a section for analytics that contains key-value pairs. Values can be JSON objects also. The contents of the analytics section will be agreed upon by the MOSIP adopter with the ABIS provider. Scores are also moved to this section and are not mandatory response parameters anymore.
Ordering or ranking of results is not explicitly specified and can be agreed upon between the MOSIP adopter and the ABIS provider.
The flags section of the request can be used to customize or control ABIS behaviour by sending specific key-value pairs.
"targetFPIR" or "maxResults" are examples of such flags that can alter ABIS behaviour. These are optional attributes for MOSIP during an identify request. MOSIP expects the adopters to define these parameters based on the accuracy expectations and workflow requirements. These can be set at the ABIS configuration level and need not be part of the individual request at all.
To give an example, please find the following calculation for targetFPIR - which is the error rate at which identification requests are expected to return a non-empty candidate list.
round (-10 * log10 (target false positive identification rate))
With this calculation:
Removes only the entry referred by the referenceId.
This operation can be used to remove duplicates found by Identify.
A Ping request should respond with a response on the liveness of the ABIS system.
ABIS responds with the count of requests that are still pending.
ABIS will send a count of records in the reference database
are the self-services which are used by the partners themselves via a portal. Partner Management Portal is a web based UI application that provides services to various types of partners.
Partner Management module has two services:
Policy Management service
Partner Management service
The documentation here will guide you through the prerequisites required for the developer's setup.
Below are a list of tools required in Partner Management Services setup:
JDK 11
Any IDE (like Eclipse, IntelliJ IDEA)
Apache Maven (zip folder)
pgAdmin
Postman
Git
Notepad++ (optional)
lombok.jar (file)
settings.xml (document)
Follow the steps below to set up Partner Management Services on your local system:
Unzip Apache Maven and move the unzipped folder in C:\Program Files
and settings.xml
to conf
folder C:\Program Files\apache-maven-3.8.4\conf
.
Install Eclipse, open the lombok.jar
file and wait for some time until it completes the scan for Eclipse IDE and then click Install/Update
.
Check the Eclipse installation folder C:\Users\userName\eclipse\jee-2021-12\eclipse
to see if the lombok.jar
is added. By doing this, you don't have to add the dependency of lombok
in your pom.xml
file separately as it is auto-configured by Eclipse.
Configure the JDK (Standard VM) with your Eclipse by traversing through Preferences → Java → Installed JREs
.
Open the project folder where pom.xml
is present.
Open command prompt from the same folder.
Run the command mvn clean install -Dgpg.skip=true -DskipTests=true
to build the project and wait for the build to complete successfully.
After building of a project, open Eclipse and select Import Projects → Maven → Existing Maven Projects → Next → Browse to project directory → Finish
.
After successful importing of project, update the project by right-click on Project → Maven → Update Project
.
Create an empty folder inside the mosip-config
with sandbox-local
name and then copy and paste all config files inside sandbox-local
folder except .gitignore, README and LICENSE
.
Put both the files in the same folder and change the location attribute to sandbox-local
folder in config-server-start.bat
file and also check the version of kernel-config-server.jar
towards the end of the command.
Example:
java -jar -Dspring.profiles.active=native -Dspring.cloud.config.server.native.search-locations=file:C:\Users\myDell\mosipProject\mosip-config\sandbox-local -Dspring.cloud.config.server.accept-empty=true -Dspring.cloud.config.server.git.force-pull=false -Dspring.cloud.config.server.git.cloneOnStart=false -Dspring.cloud.config.server.git.refreshRate=0 kernel-config-server-1.2.0-20201016.134941-57.jar
.
As mentioned in the steps above, you may have to make some changes in the two properties files as per your environment.
Run the server by opening the config-server-start.bat
file.
The server should now be up and running.
Below are the configurations to be done in Eclipse:
1. Open Eclipse and run the project for one time as Java application
, so that it will create a Java application which you can see in debug configurations and then change its name. (e.g.: project name with environment - "partner-management-dev").
2. Open the arguments and pass this -Ddomain.url=dev.mosip.net -Dapplication.base.url=http://localhost:8090 -Dspring.profiles.active=default -Dspring.cloud.config.uri=http://localhost:51000/config -Dspring.cloud.config.label=master
in VM arguments.
3. Here, the domain URL represents the environment on which you are working (eg., it can be dev2.mosip.net
or qa3.mosip.net
).
4. Click Apply and then debug it (starts running). In the console, you can see a message like "Started PartnerManagementService in 34.078 seconds (JVM running for 38.361)"
.
Policy management service also can run by following the above steps.
The APIs can be tested with the help of Swagger-UI and Postman.
Swagger is an interface description language for describing restful APIs expressed using JSON. Can access Swagger-UI of partner-management-services for dev-environment from https://dev.mosip.net/v1/partnermanager/swagger-ui/index.html?configUrl=/v1/partnermanager/v3/api-docs/swagger-config
and localhost from http://localhost:9109/v1/partnermanager/swagger-ui/index.html?configUrl=/v1/partnermanager/v3/api-docs/swagger-config
.
Can access Swagger-UI of policy-management-services for dev-environment from https://dev.mosip.net/v1/policymanager/swagger-ui/index.html?configUrl=/v1/policymanager/v3/api-docs/swagger-config
and localhost from http://localhost:9107/v1/policymanager/swagger-ui/index.html?configUrl=/v1/policymanager/v3/api-docs/swagger-config
.
Postman is an API platform for building and using APIs. Postman simplifies each step of the API lifecycle and streamlines collaboration so you can create better APIs—faster. It is widely used tool for API testing.
Partner management portal allows the partners to register themselves in MOSIP. With LTS release, the following types of partners can register themselves:
Authentication Partners
Credential Partners(with limited features)
Device Providers
FTM Provider
A Partner Admin can create Policies that are required for Authentication and Credential partners. The section below describes the types of policies that are supported by MOSIP.
To create policies, policy groups should be defined. Policy groups can be considered as the regulatory bodies in a country, examples could be Telecom, Insurance, Banking, etc.
Login as Partner Admin
into the PMS portal.
After successful login, on the left navigation pane, click on Policy -> Policy Group.
The existing policy groups are listed on the screen and the new ones can be created.
To create Policy groups
Click Policy -> Policy Group -> +Create Policy Group
Enter the Policy group Name and Description and click Save.
To search or filter any data pertaining to policy groups, use the filter menu.
You can also change the status of policy group(Deactivate/Re-activate) or edit it using the Action menu as shown below.
On successful creation of Policy groups, polices can be created under that group. MOSIP supports two types of policies, i.e., Auth policy and Datashare policy.
Click Auth Policy -> Create Policy.
Add the Name and Description.
From the dropdown, select the Policy group.
Add the Policies Data.
Click Save.
Note: Once the policy is created, it will be in Inactive state. You have to activate it before using it for a partner.
Select the policy you want to activate or edit.
From the Actions menu, select Activate/Edit.
Use the filter menu.
Data Share policy can be created/edited in the same way as the steps mentioned in the previous section on Auth policy
by using Data Share Policy menu options.
Partners in MOSIP are created in a self-service mode. The partner visits the MOSIP partner management portal and requests for collaborating with MOSIP by providing basic details such as organization name and email-id, purpose of registration (how they want to collaborate with MOSIP - as a device provider, authentication partner, print partner, etc), basic credentials and performing an OTP based verification. Once these details are filled by the partner and a request is sent to MOSIP, the Partner Admin
verifies the details of the partners and allows the partner to integrate with MOSIP.
To know more about each of the partners, click:
Device key
Device key
Name | Description | Restrictions | Type |
---|
Code | Status |
---|
Code | Reason |
---|
ABIS must get biometric data from referenceURL, process it and store it locally within the ABIS reference database. More details about the referenceURL is mentioned in our .
Below are the sample API details for getting the authentication token. More details about the API are available in our .
Encrypted Key Data | KEY_SPLITTER | Encrypted Actual Data |
---|
Encrypted Key Data | KEY_SPLITTER | Encrypted Actual Data |
---|
Error Code | Error Message |
---|
Target False Positive Identification Rate | targetFPIR |
---|
to know about the biometric specification in MOSIP.
to know how MOSIP stores biometric data.
to get the JWT token.
for details about the De-Duplication process in MOSIP.
Download lombok.jar
and settings.xml
from .
For the code setup, clone the repository and follow the guidelines mentioned in the .
For the environment setup, you need an external JAR that is available with different versions. (E.g.: You can download kernel-auth-adapter.jar
and add to project Libraries → Classpath → Add External JARs → Select Downloaded JAR → Add → Apply and Close
).
Clone .
As Partner Management Services is using two properties files, partner-management-default
and application-default
, you will have to configure them according to your environment. The same files are available for reference.
To run the server, two files are required- and .
For API documentation, refer .
Download the and then import it in your postman
.
Partner Type | Associated Role |
---|
By default, on clicking Auth policy, the screen displays the list of existing auth .
Auth policy
AP
Specifies authentication types and KYC fields to be shared during authentication.
Datashare policy
Online Verification Partner, Credential Partner, Manual Adjudiation, ABIS partner
Specifies data to be shared with partners
1 | Success |
2 | Failed |
1 | internal error - Unknown |
2 | aborted |
3 | unexpected error |
4 | unable to serve the request - invalid request structure |
5 | missing referenceId (in request body) |
6 | missing requestId (in request body) |
7 | unable to fetch biometric details (using referenceURL) |
8 | missing reference URL (in request body) |
9 | missing requesttime (in request body) |
10 | referenceId already exists (in ABIS) |
11 | CBEFF has no data |
12 | referenceId not found (in ABIS) |
13 | invalid version |
14 | invalid id |
15 | invalid requesttime format |
16 | invalid CBEFF format |
17 | data share URL has expired |
18 | Biometric Quality check failed |
Block 1 | #KEY_SPLITTER# | Block 2 |
Block 1 | #KEY_SPLITTER# | Block 2 |
DAT-SER-003 | File does not exists or File is empty |
DAT-SER-006 | Data share not found |
DAT-SER-006 | Data share usage expired |
KER-ATH-401 | Authentication Failed |
KER-ATH-403 | Forbidden |
1 in 1,000 | 30 |
1 in 10,000 | 40 |
1 in 100,000 | 50 |
Partner Admin | PARTNER_ADMIN |
Policy Manager | POLICYMANAGER |
Authentication Partner | AUTH_PARTNER |
Credential Partner | CREDENTIAL_PARTNER |
Device Provider | DEVICE_PROVIDER |
FTM Provider | FTM_PROVIDER |
Reference implementations of modules or components are non production grade implementations that are meant to showcase a reference design or architecture. They can be used as references or starting points for customization and actual implementations.
Pre-registration portal
Admin portal
Partner Management portal
Authentication Application
Registration Client
ID object validator (kernel-ref-idobjectvalidator)
Integration with the SMS service provider (kernel-smsserviceprovider-msg91)
Integration with a Virus Scanner (kernel-virusscanner-clamav)
HSM Keystore Implementation (hsm-keystore-impl)
IAM: Keycloak
External Stage
Demo deduplication
Hazelcast Cache Provider
Demographic authentication (normalization)
Child Authentication Filter
Booking Service
Pre-registration module enables a resident to:
enter demographic data and upload supporting documents,
book an appointment for one or many users for registration by choosing a suitable registration center and a convenient time slot,
receive appointment notifications,
reschedule and cancel appointments.
Once the resident completes the above process, their data is downloaded at the respective registration centers prior to their appointment, thus, saving enrollment time. The module supports multiple languages.
MOSIP provides backend APIs for this module along with a reference implementation of pre-registration portal.
User provides consent
User provides demographic information
User uploads supporting documents (Proof of Address, Birth certificate, etc.)
A pre-registration request ID known as AID is generated and provided to the user.
Note: The AID was formerly called PRID in the pre-registration context.
User selects a registration center based on postal code, geo-location, etc.
The available slots are displayed
An option to cancel and re-book the appointment is made available
An acknowledgement is sent via email/SMS
The user can print the acknowledgement containing AID and QR code.
This QR code can be scanned at the in-person registration centers.
User provides the AID/ QR code at the registration center.
The registration form gets pre-filled with the pre-registration data.
The relationship of the pre-registration module with other services is explained here. NOTE: The numbers do not signify sequence of operations or control flow
Fetch ID Schema details with the help of Syncdata service.
Fetch a new OTP for the user on the login page.
Log all events.
Pre-Registration interacts with Keycloak via kernel-auth-adapater
. The Pre-Reg module communicates with endpoints of other MOSIP modules. However, to access these endpoints, a token is required. This token is obtained from Keycloak.
Database used by pre-reg.
Generate a new AID for the application.
Send OTP in the email/SMS to the user.
Registration Processor uses reverse sync to mark the pre-reg application as consumed.
Registration clients use Datasync service to get the pre-reg application details for a given registration center, booking date and AID.
Request data from MasterData service to get data for dropdowns, locations, consent forms etc..
Pre-registration module consists of the following services:
For more details, refer to Pre-registration repo.
MOSIP provides a reference implementation of the pre-registration portal that may be customized as per country needs. The sample implementation is available at reference implementation repository. For getting started with the pre-registration, refer to the Pre-registration user guide
To access the build and read through the deployment instructions, refer to Pre-registration repo.
For details related to Pre-registration configurations, refer to Pre-registration configuration.
To know more about the developer setup, read Pre-registration Developers Guide.
Refer API Documentation.
The documentation here will guide you through the pre-requisites required for Pre-registration developer setup.
JDK 11
Any IDE (Eclipse, IntelliJ IDEA)
Apache Maven (zip folder)
pgAdmin
Postman
Git
lombok.jar (file)
MOSIP Pre-registration specific JARs: The version will depend on which Pre Registration branch you have cloned. If it is "release-1.2.0.1" then you can download 1.2.0.1.B1 or 1.2.0.1.B2 version of below jars whichever is available.
Notepad++ (optional)
Fork the MOSIP Pre-registration repository from Github MOSIP repository to your GitHub account.
Clone the forked repository into your local machine.
git clone https://github.com/${urgithubaccname}/pre-registration.git
git remote add upstream https://github.com/mosip/pre-registration.git
git remote set-url --push upstream no_push
git remote -v
git checkout -b my-release-1.2.0.1
git fetch upstream
git rebase upstream/release-1.2.0.1
Inside settings.xml
change local repository directory to your directory name where .m2 folder
is located. E.g.: <localRepository>C:/Users/username/.m2/repository</localRepository>
Add settings.xml
inside .m2 folder
(Maven Folder). E.g.: C:\Users\username\.m2
Import the project in Eclipse IDE and it starts updating Maven projects configuration, refreshing workspaces, project starts building (downloading sources, javadoc).
Add downloaded lombok.jar
to project, click on downloaded JAR and install specifying Eclipse IDE(eclipse.exe) location.
Configure the JDK (Standard VM) with your Eclipse by traversing through Preferences → Java → Installed JREs
.
Add MOSIP Pre-registration specific JARs from Maven central:
Adding JARs to Build Path: Right click on service -> Build Path -> Configure Build Path -> click on Classpath -> Add External JARs -> Add required JARs -> Apply and close.
Add auth-adapter
, transliteration
, ref-idobjectvalidator
, virusscanner
, lombok
JARs to pre-registration-application-service
, pre-registration-datasync-service
classpath.
Add auth-adapter
, lombok
JARs to pre-registration-core
, pre-registration-batchjob
, pre-registration-captcha-service
, pre-registration-booking-service
classpath.
Run mvn clean install -Dgpg.skip=true
command to build locally and to execute test cases.
Update Maven dependencies: Maven syncs the Eclipse project settings with that of the pom. It downloads dependencies required for the project.
Build and run the Project.
To run the pre-registration-application-service locally without config server, update values in application.properties and bootstrap.properties:
spring.cloud.config.uri=https://localhost:8080
spring.cloud.config.label=develop
spring.cloud.config.name=pre-registration
spring.application.name=pre-registration-application-service
spring.profiles.active=default
Point below urls to a valid env which has MOSIP setup:
mosip.base.url=https://yourenvurl
auth-token-generator.rest.issuerUrl:https://iam.yourenvurl/auth/realms/mosip
javax.persistence.jdbc.password: XXXXXX
javax.persistence.jdbc.url=jdbc:postgresql://yourenvurl:5432/mosip_prereg
mosip.batch.token.authmanager.password: XXXXXX
mosip.iam.adapter.appid=prereg
mosip.iam.adapter.clientsecret=XXXXXX
auth.server.admin.issuer.uri=https://iam.yourenvurl/auth/realms/
Fork the Pre-registration UI repo to your GitHub account.
Clone the forked repository into your local machine.
git clone https://github.com/${urgithubaccname}/pre-registration-ui.git
git remote add upstream https://github.com/mosip/pre-registration-ui.git
git remote set-url --push upstream no_push
git remote -v
git checkout -b my-release-1.2.0.1
git fetch upstream
git rebase upstream/release-1.2.0.1
Install all dependencies with npm install
.
Install Angular JS npm install -g @angular/cli
.
Start the Angular JS server ng serve
.
Open http://localhost:4200
to access the application.
You will face CORS issue since API Services are hosted on https://{env}
.
Update the API services BASE_URL
in config.json
:
config.json
is found inside assets directory.
E.g.: C:\MOSIP\pre-registration-ui\pre-registration-ui\src\assets\config.json
Create a new file named proxy.conf.json
:
Location should be in C:\MOSIP\pre-registration-ui\pre-registration-ui\proxy.conf.json
project folder.
Start the server by executing ng serve --proxy-config proxy.conf.json --ssl true
.
Open the browser, load the app with https://localhost:4200
.
For API documentation, refer here.
The APIs can be tested with the help of Swagger-UI and Postman.
Swagger is an interface description language for describing restful APIs expressed using JSON. You can access Swagger-UI of pre-registration here:
Pre-registration Application service : https://{env}/preregistration/v1/application-service/swagger-ui.html
Pre-registration Datasync Service : https://{env}/preregistration/v1/sync/datasync-service/swagger-ui.html
Pre-registration Captcha service : https://{env}/preregistration/v1/captcha/swagger-ui.html
Pre-registration Booking service : https://{env}/preregistration/v1/appointment/booking-service/swagger-ui.html
May 07, 2020 | This is the first formal publication of the interface as a version-ed specification. Earlier draft are superseded by this document. The interface is revamped to make it friendlier to programmers and also has a new method for conversion. |
June 09, 2020 | A note related to targetFPIR was added. |
June 26, 2020 |
August 04, 2020 |
November 19, 2020 | Note on encryption of biometric data share using referenceURL has been added. |
February 05, 2021 |
March 23, 2021 |
May 3, 2021 | The logic for encryption has been updated for ABIS Datashare URL. |
September 8, 2021 | All possible error codes for DataShare URL has been added. |
requestID | ID that is associated with each request sent to ABIS | ABIS should not use this ID in any other context outside the request | UUID |
referenceID | ID of a single registration record. Registration record is maintained in MOSIP. This ID is the mapping between MOSIP and ABIS | None | UUID |
referenceURL | URL to the biometrics data stored in MOSIP. This URL will have read only access | None | HTTPS URL |
biometricType | Type of biometric data sent in the request | FID/FIR/IIR | String |
returnValue | Code for response | String |
failureReason | Code for failure reason | String |
The Registration Client is a thick Java-based client where the resident's demographic and biometric details are captured along with the supporting documents in online or offline mode. Data is captured in the form of registration packets and is cryptographically secured to ensure that there is no tampering. The captured information is packaged and sent to the server for further processing.
MOSIP provides a reference implementation of a Java-based Registration Client. The code, build files for the Registration Client are available in the Registration Client repo.
Registration Client is featured to allow an operator to choose the operation language. Option to select their preferred language is provided on the login screen.
Data collection during registration client supports more than one language at a time.
Before starting any registration process, the operator can choose the languages amongst the configured ones.
To know more about setting up the reference registration client, refer to Registration Client Installation Guide.
To know more about the features present in the Registration Client, refer to Registration Client User Guide.
The Registration Client can be operated by an operator who can be either a Supervisor or an Officer. They can login to the client application and perform various activities. The Supervisor and the Officer can perform tasks like Onboarding, Synchronize Data, Upgrade software, Export packet, Upload packets, View Re-registration packets, Correction process, Exception authentication, etc. In addition to this, the Supervisor has exclusive authority to Approve/reject registrations.
To know more about the onboarding process of an operator, refer to Operator onboarding.
The relationship of Registration Client with other services is explained here. NOTE: The numbers do not signify sequence of operations or control flow.
Registration Client connects to the Upgrade Server to check on upgrades and patch downloads.
All the masterdata and configurations are downloaded from SyncData-service.
Registration Client always connects to external biometric devices through SBI.
Registration Client scans the document proofs from any document scanner.
Acknowledgement receipt print request is raised to any connected printers.
Packets ready to be uploaded meta-info are synced to Sync Status service. Also, the status of already uploaded packets are synced back to Registration Client.
All the synced packets are uploaded to Packet Receiver service one by one.
The image below shows the setup of Registration Client Host machine.
Registration Client comprises of JavaFX UI, Registration-services libaries and any third party biometric-SDK.
SBI is allowed to run on loopback IP and should listen on any port within 4501-4600 range. More than one SBI can run on the host machine. Registration Client scans the allowed port range to identify the available SBI.
Registration Client connects to local Derby database. This is used to store all the data synced.
All the completed registration packets are stored under packetstore directory.
.mosipkeys
directory is used to store sensitive files. db.conf
under this directory stores encrypted DB password. This is created on the start of registration client for the first time.
TPM - is the hardware security module used to get machine identifier, to secure DB password, prove the client authenticity on auth requests and packets created in the host machine.
The registration packets and synced data are stored in the client machine.
Most of the synced data are stored in the Derby DB. Derby DB is encrypted with the bootpassword.
Derby DB boot password is encrypted with machine TPM key and stored under .mosipkeys/db.conf
.
Synced UI-SPEC/script files are saved in plain text under registration client working directory. During sync, SPEC/script file hash is stored in derby and then the files are saved in the current working directory. Everytime the file is accessed by the client performs the hash check.
Synced pre-registration packets are encrypted with TPM key and stored under configured directory.
Directory to store the registration packets and related registration acknowledgments is configurable.
Registration packet is an signed and encrypted ZIP.
Registration acknowledgment is also signed and encrypted with TPM key.
Registration Client can be customized as per a country' requirements. For details related to Registration Client configurations, refer to Registration Client configuration.
Blueprint of registration forms to be displayed in registration client are created as json called as UI-SPEC.
Every process ( NEW / LOST / UPDATE UIN / CORRECTION ) has its own UI-SPEC json.
Kernel-masterdata-service exposes API's to create and publish UI-SPEC.
Published UI-SPEC json are versioned.
Only published UI-SPEC are synced into registration-client.
UI-SPEC json files are tamper proof, client checks the stored file hash everytime it tries to load registration UI.
UI-SPEC json will fail to load if tampered.
Default UI Specifications loaded with Sandbox installation is available here
To know more about the developer setup, read Registration Client Developers Guide.
UI specs of Pre-Registration module are used to configure the form fields in the Demographic Details and Document Upload functionality. UI specs are saved as a JSON file with a list of fields. Each field has a set of attributes/ properties that can be configured which affects the look and feel along with the functionality of the field. Below is the list of all the properties available for each field in the UI specs:
Property Name | Details | Sample Value |
---|---|---|
This guide helps in understanding the pre-registration sample UI implementation. The pre-registration portal can be used in self-service as well as in assisted mode.
Self-service mode- In this mode, residents can pre-register themselves by accessing the pre-registration portal. They can login with their email address or phone number and fill up the demographic form, upload relevant documents to book an appointment for themselves and their family/friends. Finally, they would receive an acknowledgement along with a pre-registration ID that can be used at the registration center.
Assisted mode- When used in an assisted mode, the operator could be handling the portal and helping other residents in filling up the details and creating an application on their behalf. The languages that the operator and the resident understands, may or may not be the same. If we consider a country with linguistic diversity, the possibilities increase. In such cases, the operator might log in with a language that they are familiar with, and also select a language (data capture language) familiar to the resident for filling up the demographic form and other details.
The key steps in this process are:
Login/create a user account
Create an application
Book an appointment
Receive appointment acknowledgement
To create an application, the resident/operator can follow the steps below:
Open the browser and visit the pre-registration portal.
Select the language of your preference from the dropdown.
Enter your valid email address or phone number in the text box.
Select the Captcha field.
Click Send OTP to receive a One Time Password (OTP) on your provided email address or mobile number.
Enter the OTP and click Verify.
Note: If you have not received the One-Time Password (OTP), please click on Send to request another OTP. Enter the newly received OTP to proceed. Once your OTP has been successfully verified, you will be able to create, view, or update your pre-registration application.
Once the OTP is verified, you will see a pop up for selecting the languages for data entry.
Select the languages and click Submit.
Note:
This choice will be available only if the ID issuer has configured the usage of optional languages.
Countries will have multiple languages some of which will be mandatory while others could be optional(regional languages). MOSIP provides a configuration using which a country can mandate the capture of demographic data in required number of languages (combination of mandatory and optional).
On the Demographic details page, read the Terms and Conditions and select the check box to agree. This agreement is to provide consent for the storage and processing of your personal information.
Click Accept and proceed.
Note: User consent is mandatory for creating/updating applications. The contents on this page will be displayed in all data capture languages selected.
Enter your demographic details, which includes Name, Age/DOB, Gender, Residential Status, Address, Mobile Number, Email Id, etc.
You can also change or verify your demographic details in the other selected language.
After you have filled and verified your demographic details, click Continue.
Note: The mandatory fields/labels have a *
mark. Field and button labels, error and information messages will be displayed in the user preferred language selected in the login screen. The fields displayed on this screen are configurable based on the ID schema defined by the country.
UI specs of Pre-registration module are used to configure the form fields in the Demographic Details and Document Upload functionality pages. These specs are saved as a JSON file with a list of fields.
Select the document (e.g. Passport, Reference Identity Number, etc.) from the document drop-down list.
Click Browse to locate the scanned document on your machine.
Select the file that you want to upload.
When the file is uploaded successfully, the document will appear on the right side. Verify that you have uploaded the correct document.
Repeat the steps above to upload document(s) for each applicable document category.
When adding an applicant, if a newly added applicant’s Proof of Address (POA) document is same as that of the existing user’s POA, which has been already uploaded, click Same As option and select the name of the applicant.
Click Continue to preview your application.
To change the demographic details (Name, Age, etc.), click modify at the top-right corner adjacent to the Demographic details section.
To modify the uploaded documents, click modify at the bottom-right corner adjacent to the Documents Uploaded section and make changes.
To add a new applicant, click Add Applicant. On clicking the Add Applicant option, you will be navigated to the Demographic details page to provide Consent and proceed with providing the required demographic data and upload of documents.
Click Continue.
On Your Applications page, click Create New Application to generate a new application.
Once the application is created, there could be multiple statuses depending on the data filled by the user/resident or the actions performed by them. The user can view all the pre-registration applications created by them in the Dashboard. The different statuses with a brief explanation are mentioned below:
The applications are sorted and displayed by the order of creation of application. The last application created appears first in the list.
If the user visits the registration center and consumes the appointment, then the application will be removed from the list.
If the appointment date has passed, the status changes to "Expired" and is retained on the dashboard for further rebooking/modification as required.
The recommended registration centers are automatically displayed based on your demographic details (Postal Code)
On the Book Appointment page, you can find a registration center through the three options as follows:
Click Nearby centers to view the registration centers based on your geographical location.
Use the search box to find the registration center based on your search criteria.
Click Recommended Centers to view registration centers based on your demographic details. (Postal Code)
Click Continue.
Note: The default display of registration centers will be based on the Postal Code of the user. To modify this setting, please update the location hierarchy in the pre-registration-default.properties
file using the property: preregistration.recommended.centers.locCode
.
Select your preferred date from the list of available calendar days and the number of available bookings.
The list of available time slots for your selected date is categorized between Morning and Afternoon.
Select your preferred time slot from the list.
Select the particular applicant name to book an appointment (click + to add the applicant). Note: On clicking the Add Applicant option, you will be navigated to the Demographic Details page to provide Consent and proceed with providing the required demographic data/documents.
Verify the time slot(s) as selected against the applicant name(s).
Click Continue.
On the confirmation pop-up, click Confirm.
Click OK.
After successful completion of the Pre-registration application, you will receive an acknowledgment on the registered phone number (SMS) or email address as per details provided in the demographic form.
The acknowledgement contains the following information: name, pre-registration ID, age/DOB, mobile number, email id and registration center details, appointment date, appointment time)
A QR code containing the pre-registration ID is generated. This QR code can be scanned at the registration center to fetch the details to be used during the registration process.
You can print, download, email or SMS your acknowledgment.
To print your acknowledgement, click Print.
To download your acknowledgement, click Download PDF.
To add the additional recipient(s) to receive the acknowledgment of your application, follow these steps:
Click Send Email/SMS.
Enter the mobile number and/or enter the email ID.
Click Send to receive the acknowledgement on your provided e-mail address or mobile number.
On Your Applications page, select the check box for the applicable applicant.
Click Book/Modify Appointment to re-book an appointment (on the top right corner)..
The user can select any appointment date available and the appointment slot available
A user cannot re-book the appointment if the appointment booking is less than 48 hours (configurable) from the time of booking
On Your Applications page, click on delete icon against pre-registration application of an applicant, a pop-up window appears on the screen.
Select the Discard entire application option in the pop-up window.
Click SUBMIT to discard your application.
On Your Applications page, click on delete icon against pre-registration application of an applicant, a pop-up window appears on the screen.
Select Cancel appointment and save the details option in the pop-up window.
Click SUBMIT to cancel an appointment.
Following a successful appointment cancellation, the system unlocks the time slot of the registration center to ensure that someone else can book it.
This guide contains all the details you may want to know about the operator onboarding.
To generate the first operator in MOSIP eco-system, refer to the steps below.
The Admin needs to:
Create the role Default in KeyCloak with all the other roles.
Create the operator' user account in KeyCloak.
Assign the operator user account with the Default role.
Perform Zone and Center mapping for the operator using the Admin Portal.
Onboard the operator machine using the Admin Portal. Machine' details can be extracted using the
The operator will need to:
Download the latest registration client and login with the credentials set in KeyCloak. The operator will automatically skip Operator/Supervisor onboarding and reaches the home page of the registration client.
Register themselves in MOSIP and get a RID and UIN.
Once the operator is registered:
The Admin changes the role of the operator to either REGISTRATION_OFFICER or REGISTRATION_SUPERVISOR.
Deletes the role Default from KeyCloak so that no other user has the role Default.
This operator can now register and onboard other Supervisors and Officers.
Admin needs to map the operator' UIN in KeyCloak under Attributes with attribute name as individualId
.
Admin needs to remove the "Default" role mapping for the operator' user account if it exists.
The operator needs to login (password based) to the Registration Client using Keycloak credentials.
The operator needs to ensure that the Registration Client machine is online.
The operator will land into the below page and needs to click on Get Onboarded
The operator needs to provide their biometrics and click Save.
All the biometric modalities displayed in the Operator biometrics page must be captured before clicking on Save.
Captured biometrics quality must be greater than or equal to the threshold displayed in the UI.
Note- The threshold values are configurable and can be set as per the ID issuer.
Note:
After successful onboarding of the operator, the templates are extracted from the captured biometrics using configured Bio-SDK. The extracted templates are stored in Derby DB. This can be used later for operator' biometric-authentication and also for local de-duplication checks during registration.
After the first login and successful on-boarding, the registration client would mandate the operator to login with the configured authentication mode decided by the administrator.
Any number of operators can login to a registration client machine but they need to be mapped to the same center where the machine is onboarded.
Login operator' user ID is case-insensitive.
Summarizing, on-boarding of an operator is successful only if,
The operator is active and not block listed.
The operator and the machine belongs to the same center.
The operator's User ID is mapped to their UIN.
The operator's biometric authentication is successful during on-boarding.
The system is online during on-boarding.
Operator logs into Registration Client for the first time and is redirected to Onboarding screen. Here, they need to capture all their biometrics and then click SAVE button.
Success/Failure response sent back to Registration Processor based on the authentication result.
Registration Processor sends back this response to Registration Client.
After successful authentication, the captured biometrics are sent to configured Bio-SDK to extract templates.
The extracted templates are stored in local Derby DB.
These templates stored in local DB can be used later for operator's biometric-authentication and also for local de-duplication checks during registration.
MOSIP supports single factor and multi factor login including password, iris, fingerprint, and face authentication for registration client. An administrative configuration setting determines the mode of authentication login.
The registration client can authenticate an operator in offline mode using the locally stored biometrics templates (face/finger/iris) and password hash.
The registration client temporarily locks the operator’s account in case they provides an invalid password/fingerprint/iris/face for X times continuously to login (X is configurable). The temporary account lock lasts for X minutes (X is again configurable).
An Operator can logout of the registration client by:
Clicking on the Logout button,
Closing the registration client,
Being in-active on the registration client for configured amount of time after which they are automatically logged out.
Upon logout, any unsaved data will be lost.
Data will not be automatically saved in the database and will not be retained in memory though transaction details which is used for auditing will be captured and stored (except for PII data).
Note- Registration client provides an alerts to the operator ‘X’ minutes before reaching the auto logout time limit. Registration client displays a countdown timer in the alert. The operator can choose to dismiss the alert and continue working. This will also reset the timer to zero.
Default settings schema is configured as below:
All the connected devices are listed in this page.
Option to scan for SBI for any specific port range is available in this page.
If more than one device is idenitied, operator can choose amongst the listed devices to set the default device for the current login session.
Access control on this page is controlled via the settings-schema.
All the Registration Client related configuration key-value pairs are listed in this page.
Operator can set the local preference on the server configraton value, applicable only for permitted configuration keys.
Access control on this page is controlled via the settings-schema.
All the available background jobs are listed here along with their cron expression.
Every job's next and previous trigger time is listed along with the job name.
Privileged operator can update the cron expression of any job.
Synchornize Data
in home page will trigger all of these listed jobs with just one click.
If the operator needs to trigger specific job, the same can be handled in this page.
Access control on this page is controlled via the settings-schema.
This guide describes the various functions provided in the Home page of the reference UI implementation of the registration client.
The Registration Client menu bar displays the following:
MOSIP logo
Home button
Logged in username
Center name
Machine name
Server connection status symbol (shows if the client is online or offline)
Breadcrumbs (User Guide/Reset Password/Logout)
Synchronize data is the data required to make the Registration Client(RC) functional. Full sync is performed initially during the launch of the RC for the first time. Post this, the RC syncs only the changes from sever and this is called as the delta sync. Synchronize data is automated and can be triggered manually. This happens automatically while launching the RC and is also manually initiated by the operator.
Configuration sync: Sync of properties which drives in deciding the RC UI functionality. For example: Invalid login attempts, idle timeout, thresholds, etc.
Masterdata sync : As a part of this sync, supporting information like Dynamic fields data, templates, locations, screen authorization, blocklisted words, etc. are pulled in.
UserDetails sync: userID, along with their status is synced. Only the user details belonging to machine mapped center will be synced.
Certificate sync: Certificates used to validate the server signatures, device CA certificates, public key (specific to a center and machine, also called as policy key) used to encrypt the registration packet will be synced.
Packet sync:
All the approved/rejected Registration IDs(RIDs) will be synced to the server.
All the synced RID packets will be uploaded to the server.
All the uploaded packet status will be synced from server.
An operator can download the pre-registration data of a machine mapped center while being online and store it locally in the registration machine for offline use. Now, when the system is offline and the pre-registration data is already downloaded, the operator can enter the pre-registration ID to auto-populate the registration form. Provided the machine is online, any pre-registration application can be downloaded irrespective of the center booked by the resident.
Note- Date Range of pre-registration data to be downloaded and storage location of pre-registration data in the registration machine is configurable. Also, this is synced as a part of configuration sync.
Using this option, the operators can onboard themselves anytime.
Application upload refers to upload of supervisor reviewed registration packets (approved and rejected). From this page, the operator can export the packets to any location on their machine on clicking Save to Device button.
Upload of registration packet from this page internally performs two operations in sequence:
Sync registration metadata to server
On successful sync of metadata, actual registration packets are uploaded to the server.
Client Status: This column displays the status of a registration packet based on the above mentioned operation.
APPROVED
REJECTED
SYNCED
EXPORTED
Server Status:
On success,
PUSHED
PROCESSED
ACCEPTED
On failure,
REREGISTER
REJECTED
RESEND
This column displays the type of registration packet (New packet, Lost packet, Update packet, Correction packet)
When a machine is re-mapped from one center to another center, all the pending activities in the machine related to the former center needs to be completed.
On successful completion of pending tasks, the former center's details will be deleted from the local Derby DB and a full sync will be initiated to pull in the new center details.
Packet Approvals/rejections
Packet upload
Server confirmation on receiving a packet
Deletion of packets after receiving a confirmation
Deletion of pre-registration packets
Deletion of center specific data like the public/policy key
Note- After completing the above tasks, a restart will be prompted to initiate the full sync with new center details.
Clicking on this button, triggers a check for any new client version availability in the upgrade server.
The machine must be online to be able to check updates.
If there is any new version available, a confirmation pop-up is displayed to the operator for starting the upgrade or a reminder is displayed.
The operator can initiate any task from amongst- New registrations, Update UIN, Lost UIN, Correction flow. To get started, the operator needs to select a language for data entry. The number of languages displayed in the UI is configurable and depends on the country.
An operator can initiate the process of registering a new applicant in the MOSIP ecosystem by filling the new registration form with the resident.
Below are few of the processes that needs to be completed for a new registration.
Capture consent- For every registration, the registration client provides an option for the operator to mark an individual's consent for data storage and utilization.
Capture biometrics of a resident The capture of biometrics is governed by the country, i.e. capture of each modality (fingerprint, iris or face) can be controlled by the country using the global configuration. When the operator clicks on the Capture button and tries to capture the biometrics of the resident, the device needs to make the capture when the quality of the biometrics is more than the threshold configured by the country. The device will try to capture the biometrics until the quality threshold has crossed or the device capture timeout has crossed which is also configurable.
After the timeout has occurred and the captured quality of biometrics is less than the threshold, registration client provides an option to re-try capture of biometrics but for a particular number of times which is also configurable. If the resident has a biometric exception (resident is missing a finger/iris or quality of finger/iris is very poor) the operator can mark that particular biometrics as exception but the operator has to capture the resident's exception photo.
What is the difference between an adult' and an infant' biometric capture?
For an adult, all the configured biometrics can be captured.
For an infant, by default, only the face biometrics is allowed to be captured.
Concept of biometric exception
Permanent or temporary missing / defective fingers or irises can be marked as exception during registration process.
Marked exception finger / iris names are sent as part of rcapture
request to SBI.
A photo of resident is captured highlighting his/her biometric exceptions called as Proof of exception (POE).
Biometric exception photo is captured by the biometric face camera device.
Until 1.2.0, POE was collected only as documentType
field. From 1.2.0.1, Captured biometric exception photo is stored in the biometric data file (CBEFF xml file).
When a resident visits the registration center to update their demographic or biometric details, the operator captures the updated data as provided by the resident in the registration client. The resident needs to give their UIN and also select the field(s) that needs an update. The image below gives an idea of the update UIN process Flow in the registration client.
The UIN update feature is configurable by a country. The Admin can turn ON or OFF, the UIN update feature using the configuration.
There might be a situation when a resident might have lost his UIN and visits the registration center for retrieving the same. The operator then captures the biometrics and the demographic details of the individual and processes a request to retrieve the lost UIN. As biometrics play a crucial in identifying a person's identity, it is mandated to provide the biometrics as a part of the Lost UIN flow. Other details like demographic data, uploading documents are optional.
As a part of Lost UIN flow, the contact details like the phone number/ e-mail address of the UIN holder can also be updated and stored in the ID Repository based on the value provided for the property (uingenerator.lost.packet.update.information) which is specified in the Registration Processor properties.
If the value already exists for the above mentioned property and once the lost UIN is found, details corresponding to the property value can be fetched from Packet manager and updated in ID Repository.
Example: uingenerator.lost.packet.update.information= phone,email
For a resident whose UIN is yet not generated, they can get a request intimation to re-provide their details with a RequestId. The same AddtionalInfo RequestId must be provided to the operator during the correction flow.
Note- The above mentioned Registration tasks are completely configurable through UI Specs<>
The operator can preview the data filled and navigate back to the respective tabs in case of corrections.
Once the resident and the operator are satisfied with the data being captured, the operator can proceed to the Authenticate tab and provide his valid credentials to mark the complete of the registration task.
Once the registration process (new registration, UIN update or lost UIN, correction) is completed, the registration client generates an acknowledgement receipt. This receipt contains a QR code of the Registration application ID, captured demographic data in the selected language, photograph of the resident and ranking of each finger from 1 to 10 (1 being the finger with the best quality). This receipt is print friendly and can be used for printing using any printer.
The image below is that of a sample acknowledgement receipt.
The Supervisor has the exclusive authority to approve/reject packets. The supervisor is supposed to manually re-verify the registrations before uploading them to the server. This page enables them to perform this activity.
Steps to approve/reject packets
Click on any of the registrations listed in the left pane. The registration details are displayed on the right pane.
Supervisor needs to manually verify all the details in the right pane.
Supervisor can click Approve/Reject button based on their verification.
To mark the completion of this approval process, they need to click on Authenticate and provide their credentials.
On successful authentication, approved/rejected packets will be removed from here and be seen on the Application Upload page.
All the registrations which are being marked with the RE-REGISTER status is listed here.
On clicking Dashboard, the Registration client dashboard HTML template is rendered. Default dashboard displays information about the operator, Packets and the Sync Activities.
This section has been reserved for the countries to be able to display their live news and updates. This can be implemented as per a country's requirements.
The guide here lists down some of the important properties that may be customised for a given installation. Note that the listing here is not exhaustive, but a checklist to review properties that are likely to be different from default. If you would like to see all the properites, then refer to the files listed below.
IMPORTANT : From the LTS version, All the properties are synced to the registration-client only from registration-default.properties
file.
See for location of these files.
Registration Client reaches SBI on 127.0.0.1 within the below configured port range. As per SBI spec, allowed port range is from 4501 to 4600.
Timeouts in milliseconds set during any http calls to SBI.
Quality score threshold based on modality, Possible values 1 to 100
Retry attemps, Possible values 1 to 10
Quality score threshold based on modality for operator authentication, Possible values 1 to 100
Registration client can be integrated with more than one bio-sdks. Possible values for "modality-name" are "finger", "iris" or "face".
SDK implementation class full name
mosip.biometric.sdk.providers.<modality-name>.<vendor-name>.classname
SDK API version
mosip.biometric.sdk.providers.<modality-name>.<vendor-name>.version
SDK implementation class consturctor args - comma separated
mosip.biometric.sdk.providers.<modality-name>.<vendor-name>.args
SDK initialization args, this will be passed as initparams
mosip.biometric.sdk.provider.<modality-name>.<vendor-name>.<key1>=<value1>
Quality threshold used by SDK to match modality
mosip.biometric.sdk.providers.<modality-name>.<vendor-name>.threshold
Example configurations shown below for MOCK SDK named as mockvendor:
On every successful biometric capture during registration, Quality of the biometrics is computed by bio-sdk if below config is enabled. Possible values are Y/N.
mosip.registration.quality_check_with_sdk=Y
Jobs like RID sync, packet upload, status sync is carried out in batches, number of registration records to be executed in a batch on every trigger.
Only the modalities configured will be collected during operator onboarding.
On every Pre-Registration application fetch in registration page, clears all the captured data prior to Pre-Registration application fetch. Set the field id's which should not be cleared after Pre-Registration application fetch. It is comma separated list of field ids as per UI-SPEC.
Storage Location to store the downloaded Pre-Registration Packets in local system
Pre-registration applications fetch time span, No. of days before appointment date.
Comma separated list of offline job ids. Offline jobs are the jobs which are not part of manual sync.
Comma separated list of untagged job ids. Untagged jobs, which will be not part of manual sync but only from scheduler.
Comma separated list of job ids which needs Registration Client restart.
Registration batch jobs scheduler.
Default CRON expression for scheduling the Jobs.
All the identified scanner implementations will be used to list the identified devices. For each device dpi, width and height can be configured. If it is not configured, it defaults to 0.
Values in the this config mosip.registration.docscanner.id
map supports regex.
Enable GPS device for capturing the geo-location. If y, GPS device will be enabled. If n, GPS device will be disabled.
mosip.registration.gps_device_enable_flag=N
Model of the GPS Device
mosip.registration.gps_device_model=GPSBU343Connector
Timeout for connecting to GPS device
mosip.registration.gps_port_timeout=1000
GPS Serial Port in Linux machine
mosip.registration.gps_serial_port_linux=/dev/ttyusb0
GPS Serial Port in Windows machine
mosip.registration.gps_serial_port_windows=
The Distance Parameter for GPS Verification
mosip.registration.distance.from.machine.to.center=900000
To Reset Password in Registration Client
On clicking Reset password from the Actions menu present in the top right corner of Home page, it redirects the operator to the URL below which is configurable.
Enables / disables reviewer authentication on any biometric exception during registration
If enabled cross-checks of residents biometrics with locally stored operator biometric templates.
Minimum disk space required to create a packet - in MB
Registration packet store location
Number of days allowed to start Registration Client without upgrade when software upgrade is available.
mosip.registration.softwareUpdateCheck_configured_frequency=60
Time in Seconds for forced log-out of operator, if operator is idle for the specified duration
mosip.registration.idle_time=900
Time in Seconds to diplay the warning message pop-up to operator, if operator is idle for the specified duration
mosip.registration.refreshed_login_time=600
Maximum no. of days for approved packet pending to be synced to server beyond which Registration Client is frozen for registration
mosip.registration.last_export_registration_config_time=100
Maximum no. of packets pending EOD approval beyond which Registration Client is frozen for registration
mosip.registration.reg_pak_max_cnt_apprv_limit=100
Enable supervisor authentication feature. If y, supervisor approval will be enabled, else, will be disbaled
mosip.registration.supervisor_approval_config_flag=Y
No. of days beyond audit creation date to delete audits
mosip.registration.audit_log_deletion_configured_days=10
No. of days beyond registration date to delete synced and uploaded registration packet
mosip.registration.reg_deletion_configured_days=1
No. of days beyond appointment date to delete unconsumed pre-registration application data
mosip.registration.pre_reg_deletion_configured_days=1
Maximum duration to which registration is permitted without sync of master data
mosip.registration.sync_transaction_no_of_days_limit=5
Allowed number of invalid login attempts
mosip.registration.invalid_login_count=50
Used to configure the time (in minutes) for locking account after crossing configured invalid login count
mosip.registration.invalid_login_time=2
Configuration used to check if any sync job is missed / failed beyond expected days, this configuration is checked everytime operator clicks on any registration process. We follow below convention to create this config key.
mosip.registration.job api name as in sync_job_def table.frequency=value in days
#Maximum no. of days without running the Master Sync Job beyond which Registration Client is frozen for registration
mosip.registration.masterSyncJob.frequency=190
#Maximum no. of days without running the Pre-Registration Sync Job beyond which Registration Client is frozen for registration
mosip.registration.preRegistrationDataSyncJob.frequency=190
#Maximum no. of days without running the Packet Sync Status Job beyond which Registration Client is frozen for registration
mosip.registration.packetSyncStatusJob.frequency=190
#Maximum no. of days without running the Key Policy Sync Job beyond which Registration Client is frozen for registration
mosip.registration.keyPolicySyncJob.frequency=190
#Maximum no. of days without running the Registration Deletion Job beyond which Registration Client is frozen for registration
mosip.registration.registrationDeletionJob.frequency=190
#Maximum no. of days without running the Configuration Sync Job beyond which Registration Client is frozen for registration
mosip.registration.synchConfigDataJob.frequency=190
#Maximum no. of days without running the Audit Logs Deletion Job beyond which Registration Client is frozen for registration
mosip.registration.deleteAuditLogsJob.frequency=190
Date format to be displayed on acknowledgement slip, default value - dd/MM/yyyy hh:mm a
Date format to be displayed on Registration Client dashboard, default format - dd MMM hh:mm a
The registration UI forms are rendered using respective UI specification JSON. This is derived from the defined by a country. Here, we would be discussing about the properties used in the UI specification of Registration Client.
In the Registration Client, currently, Registration Tasks(process) forms are configurable using the UI specifications.
Each process has multiple screens and each screen is rendered with one or more fields.
This guide helps the operator in setting up the registration client.
A Trusted Platform Module (TPM) is a specialized chip on a local machines that stores RSA encryption keys specific to the host system for hardware authentication.The pair is maintained inside the chip and cannot be accessed by software. By leveraging this security feature every individual machine would be uniquely registered and identified by the MOSIP server component with it's TPM public key.
To onboard the machine and the operator, the Admin needs to:
Create and activate the registration client machine using Admin portal.
Create a user/operator account in Keycloak
Assign the operator a role of either the Supervisor or Officer using the Admin portal.
Finally, perform the User Zone mapping and User Center mapping in the Admin portal.
System prerequisites:
CPU - Dual Core Processor - 2GHZ
RAM - 16 GB
Local Storage Disk Space - 500 GB
USB 2.0 ports or equivalent hub.
Physical machine with TPM 2.0 facility.
Windows OS [10 v]
To setup the registration client:
Download the reg-client.zip
from the hosted server.
Unzip the client. You can see the directory structure below.
Click run.bat
to launch registration client.
The client always launches with the pre-loader screen which displays the information about the network status, build status verification, online status, etc.
First time launch
After the pre-loader, the login screen is displayed.
Any valid operator credentials can be provided to authenticate and start the initial sync.
On successful intial sync, the operator will be prompted to restart the application.
After the first launch, the operator can notice .mosipkeys and db folders created under the registration client setup folder.
Note: Deletion of either the .mopsipkeys or the db folder makes the application get into an invalid state and hence will fail to launch. To be able to launch the client again, the operator should make sure that both the folders are removed and then re-launch the client.
On the next launch after the initial sync,
The registration client login page provides the operator an option to select the language for viewing the registration client UI.
After successful login, the operator either lands into the operator onboard page or the home page.
Offline- Operator can use the registration client in offline mode to only do the registrations and EOD process. During offline mode, the operator authentication will be based on locally saved password hash. An operator can work in offline mode only if they have logged into to the registration client being online atleast once.
Online- Machine must be online for the registration client first launch. For any server-client sync or vice-versa, the registration client must be online. In the online mode, the client reaches out to the server for password authentication.
Note: On successful onboard of the operator, biometric templates of the operator are stored locally. Biometric authentication does not reach out to the server everytime, instead it is validated based on the locally stored templates on the registration client machine.
1. Incorrect username/password
2. Configuration / masterdata Sync failed
New (code - 6, 8, 9, 10, 11, 12) for ABIS have been added.
Analytics section has been added to the overall response for Identify and the have been updated.
Note on and was added for Insert Request.
New (code - 17) for ABIS has been added.
Status | Description | User Action |
---|---|---|
After successful onboarding, the operator is automatically re-directed to the .
Request from Registration Client goes to for operator authentication.
Registration Processor passes this request to where it checks whether the user is mapped to a valid UIN and then matches the biometrics sent in the request with the biometrics of the mapped UIN.
Extracted templates are sent back from .
Clicking on in the home page opens a pop-up displaying configured Settings page as per the above sample settings schema.
For more details, refer .
Enter demographic data and upload documents If the resident has a pre-registration application ID, the operator can auto-populate the demographic data and the documents by entering the pre-registration ID. If the resident does not have a pre-registration ID, the operator can enter the resident’s demographic details (such as Name, Gender, DOB, Residential Address, etc.) and upload the documents (such as Proof of Address, Proof of Identity, Proof of Birth) based on the defined by the country.
To extract the TPM public key, use the .
For more details on operator onboarding, refer to .
For more details on Home page, refer to .
In the development environment, Registration client can be tested using mock SBI. Find the instructions to build and run the mock SBI, click .
id
The id property is the unique id provided to a field to uniquely identify it. The id can be alpha-numeric without any spaces between them.
"id":"zone"
description
This is a non-mandatory property used to describe the field.
"description": "zone"
labelName
This property defines label name for the field. This property has sub-attributes as the language code (eng, fra, ara) to store data in different languages based on the country's configuration.
"labelName": { "eng": "Zone", "ara": "منطقة", "fra": "Zone"}
controlType
This property defines the kind of UI component to be used to capture data in UI. Currently the values that can be used are: • textbox (creates multiple textboxes for each field to capture input in all the languages configured for the setup) • dropdown • fileupload • date (creates a date picker) • ageDate (creates a date picker along with number toggle to provide age directly) • checkbox (creates a toggle checkbox for the field which can be checked or unchecked) • button (creates dropdown options as buttons, which user can select easily)
inputRequired
This property decides if the field is to be displayed in the UI form or not. It is useful for some internal fields which do not need any input from the user.
required
This is a mandatory property which decides if the field is a required form field or not. If true, user must provide some value for the field.
type
This property defines the data type of the value corresponding to this field. The data types supported are “number”, “string” and “simpleType”. The type “simpleType” means that language specific value will be saved along with the language code.
fieldType
This property is relevant when control type is “dropdown” or “button”. It defines if the field is of type “default” or “dynamic”. If it is “dynamic” then all the options for the dropdown are populated from the “master.dynamic_field” table otherwise they are populated from corresponding table example “master.gender”
subType
This is relevant for 2 cases: 1. When control type is “dropdown”/ “button” and the type is “dynamic” then “subtype” can be used to populate the options for the field with the data available in “master.dynamic_field” table. 2. When the control type is “fileupload”, then the property ”subtype” is used to map the field to a “code” in the “master.doc_category” table.
validators
* This property enables us to add the list of language specific validators for the field. * Each validator can have the below fields, “langCode”, “type”, “validator”, “arguments”, “errorMessageCode” * The “type” defines the validation engine type. * The “validator” gives the pattern/ methodName/ scriptName/ expression * The “arguments” array to is used to hold parameter or dependent field ids required for validation * The “errorMessageCode” can be given to add custom error message which will be shown to the user when the validation fails. The error message corresponding to this code will be picked from language specific i18n translation files. In case “errorMessageCode” is not given then generic error message will be displayed to the user when validation fails. Currently, regex is supported by MOSIP. If “langCode” is not added then same “validator” is used for all languages of the field.
"validators": [{ "langCode": "eng", "type": "regex", "validator": "^(?=.{0,50}$).*", "arguments": [], "errorMessageCode": "UI_1000" },{ "langCode": "ara", "type": "regex", "validator": "^[A-Z]+$", "arguments": [] },{ "langCode": "fra", "type": "regex", "validator": "^[A-Z]+$", "arguments": [] }]
transliteration
If enabled, then value entered by user in one language is transliterated into other.
locationHierarchyLevel
This attribute is mandatory for the location dropdown fields. The value will be as per corresponding location hierarchy level from the master.loc_hierarchy_list table.
{ "id":"region", "controlType":"dropdown", "fieldType":"default", "type":"simpleType", "parentLocCode":"MOR", "locationHierarchyLevel":1 ..}
parentLocCode
This attribute is to be used only for location dropdown fields and it is optional. The corresponding location dropdown will be pre populated in UI based on the value in “parentLocCode”. If this attribute is NOT mentioned in UI specs, then the dropdown will be populated based on selection in its parent dropdown, as before. For the first dropdown, in case this attribute is not mentioned in UI specs then the value from “mosip.country.code” configuration will be used for backward compatibility.
{ "id":"region", "controlType":"dropdown", "fieldType":"default", "type":"simpleType", "parentLocCode":"MOR", "locationHierarchyLevel":1 ..}
visibleCondition
The property is used to define an expression using json-rules-engine which will decide the visibility condition for the form field. Refer https://www.npmjs.com/package/json-rules-engine to get the syntax for the conditions.
"visibleCondition": { "all": [{ "fact": "identity", "operator":"lessThanInclusive", "value": 10, "path": "$.age" }]} This condition will make the field visible only when the “age” field value <= 10.
requiredCondition
The property is used to define an expression using json-rules-engine which will decide the required condition for the form field. Refer https://www.npmjs.com/package/json-rules-engine to get the syntax for the conditions.
"requiredCondition": { "all": [{ "fact": "identity", "operator": "equal", "value": "MLE", "path": "$.gender.0.value" },{ "fact": "identity", "operator": "equal", "value": "102", "path": "$.maritalStatus.0.value" }]} This condition will make the field required only when the “gender” field value = ‘MLE’ and “maritalStatus” field value is 102” .
alignmentGroup
* This property is used to group the fields on the screen. * If it is skipped, then all the fields will appear in same sequence (horizontally layout) as they appear in UI specs. * If you want the first and fifth field to be in same row in the screen, you can add this attribute with same group name. * The UI is responsive so it will accommodate as many fields in one row as they will fit comfortably.
containerStyle
This is used to optionally apply some CSS styles to the UI field container.
"containerStyle": { "width": "600px", "margin-right": "10px" }
headerStyle
This is used to optionally apply some CSS styles to the UI header of the field container.
"headerStyle": { "width": "600px", "margin-right": "10px" }
changeAction
This property can be added to define interaction between the 2 or more form fields when user changes value in one of them. Currently the “changeAction” supported are “copy&disable” and “copyto” ONLY.
1. "changeAction": "copyto:permanentZipcode, presentZipcode,addressCopy" When the checkbox “addressCopy” is checked the value of the field “permanentZipcode” will be copied to “presentZipcode”. 2. "changeAction":"copy&disable: permanentCountry=presentCountry, permanentAddressLine1=presentAddressLine1" When the checkbox “addressCopy” is checked the value of the field “permanentCountry” will be copied to “presentCountry” and the field “presentCountry” will be disabled. Same with “permanentAddressLine1” and “presentAddressLine1”.
Incomplete
Filled only demographic details
Upload documents and book an appointment
Pending appointment
Filled demographic details and uploaded documents
Book an appointment
Booked
Filled demographic details, uploaded documents, and booked appointment
Visit the registration center on the appointment date and time
Expired
Appointment date has passed
Re-book an appointment
Cancelled
Appointment has been cancelled
Re-book an appointment
MOSIP provides a reporting framework for real-time streaming of data and visualization. The dashboards give a visual display of metrics and important data to track the status of various pre and post-enrollment processes. This data helps ID issuers with improving efficiency, forecasting, and better decision-making. The framework has been used to create a set of default dashboards using Kibana.
Details of reporting framework can be found here.
The following dashboards are configured on Kibana. The NDJSON source files are available here.
The reporting pipeline and dashboards may be customized according to an adopter's requirements.
Setup a data pipeline for populating data from database to Elasticsearch as given here.
After data is populated in Elasticsearch, add/customize dashboards in Kibana as given here.
Following are the metrics that are collected in the client application using the micrometer library:
JVM Memory Metrics
JVM Thread Metrics
JVM GC Metrics
JVM Heap Pressure Metrics
Processor Metrics
Class Loader Metrics
Disk Metrics
Packet Metrics based on client and server status
All the metrics collected are appended to metrics.log
file. Rolling policy of the metrics.log
is defined in registration-services logback.xml
.
Below are the challenges faced in exporting the collected metrics from client application to the server for further analysis:
Unreliable network conditions on field.
Metrics files are mostly large files, and cannot afford retries on failed attempts.
Required HTTP based metrics export.
To overcome the above challenges, Registration Client is built with tus-java-client
(version: 0.4.3) . Tusd server URL and the upload chunk-size are made configurable in the client application.
mosip.registration.tus.server.url
: This is the server URL config which specifies to which URL the metrics files are to be uploaded.
mosip.registration.tus.server.upload.chunksize
: This config defines the chunk size, which means, how much size of the file is to be uploaded at once. By default, this is given as 1024, which means 1KB
Note:
The Tus protocol
is designed to enable resumable uploads of large files over HTTP, which can be useful for web applications that need to handle file uploads in unreliable network conditions or with large files that might take a long time to upload. For more information on TUS, refer here.
Tusd
is a popular implementation of the Tus protocol that can be used as a standalone server. It is a part of the MOSIP deployment.
Each metric json logged into metrics.log
file is tagged with machine name. Refer the below log lines with the machine names.
A job is scheduled to upload collected metrics to server from the client application.
Job runs with a fixed delay of 15 minutes.
Resumable file URLs are stored under .metrics
folder of registration client. Once the complete file is uploaded to server, the metrics file is deleted locally.
Registration Client is a thick Java-based client where the resident's demographic and biometric details are captured along with the supporting documents in online or offline mode.
The documentation here will guide you through the prerequisites required for the developer' setup.
Below are a list of tools required in Registration Client:
JDK 11
Any IDE (like Eclipse, IntelliJ IDEA)
Apache Maven (zip folder)
Git
Follow the steps below to set up Registration Client on your local system:
For the code setup, clone the Registration Client repository and follow the guidelines mentioned in the Code Contributions.
Open the project folder where pom.xml
is present.
Open command prompt from the same folder.
Run the command mvn clean install -Dgpg.skip=true -DskipTests=true
to build the project and wait for the build to complete successfully.
After building of a project, open Eclipse and select Import Projects → Maven → Existing Maven Projects → Next → Browse to project directory → Finish
.
After successful importing of project, update the project by right-click on Project → Maven → Update Project
.
1. For the environment setup, you need an external dependency that is available here with different versions. (E.g.: You can download mock-sdk.jar
and add to registration-services project Libraries → Classpath → Add External JARs → Select Downloaded JAR → Add → Apply and Close
).
2. Registration Client UI is developed using JavaFX and the UI pages are fxml files which can be opened using a tool called Scene Builder
. The JavaFX integration with the Eclipse IDE is provided with the e(fx)clipse tool. Go to Help → Install New Software → Work with → Add
. Give Name and Location as mentioned in below image.
Once the software is installed, you will be prompted to restart your IDE.
3. Download openjfx-11.0.2_windows-x64_bin-sdk.zip
from here, unzip and place it in your local file system. This folder contains list of javafx related jars that are necessary for running Registration Client UI.
4. We can change the application environment in the file registration-services\src\main\resources\props\mosip-application.properties
by modifying the property mosip.hostname
Below are the configurations to be done in Eclipse:
1. Open Eclipse and run the project for one time as Java application
, so that it creates a Java application which you can see in debug configurations.
2. Open the arguments and pass this --module-path C:\Users\<USER_NAME>\Downloads\openjfx-11.0.2_windows-x64_bin-sdk\javafx-sdk-11.0.2\lib --add-modules=javafx.controls,javafx.fxml,javafx.base,javafx.web,javafx.swing,javafx.graphics --add-exports javafx.graphics/com.sun.javafx.application=ALL-UNNAMED
in VM arguments.
3. Click Apply and then debug it (starts running). You can see a popup which shows informative messages of what is happening in background while Registration Client UI is loading and the application will be launched.
Dockerfile is available under registration-client repository.
The concept here is to run nginx in the docker container from which registration-client.zip is hosted and also registration-client on the field will connect to this nginx to check for new updates or changes.
Note: We refer this nginx server as registration-client download and upgrade server.
While running the registration-client docker container we need to pass below environment variables:
Required
client_version_env
: pom version of the registration client build
client_upgrade_server_env
: public URL of this nginx server
reg_client_sdk_url_env
: URL to download sdk zip. Make sure to have SDK jar and its dependent jars in the zip.
artifactory_url_env : artifactory URL to download below mentioned runtime dependencies
“${artifactory_url}/artifactory/libs-release-local/icu4j/icu4j.jar”
“${artifactory_url}/artifactory/libs-release-local/icu4j/kernel-transliteration-icu4j.jar”
“${artifactory_url}/artifactory/libs-release-local/clamav/clamav.jar”
“${artifactory_url}/artifactory/libs-release-local/clamav/kernel-virusscanner-clamav.jar”
keystore_secret_env
: password of the keystore.p12 file
host_name_env
: syncdata-service public domain name
signer_timestamp_url_env
: URL of timestamp server to timestamp registration-client jar files.
Need to mount a volume to “/home/mosip/build_files” with below files
logback.xml
Client.crt : Signer certificate to be added in the registration-client build for jar sign verification.
keystore.p12 : Store private key used to sign the registation-client and registration-services jar files with “CodeSigning” alias.
maven.metadata.xml : Holds the current version of registration-client hosted in the upgrade server.
Optional
reg_client_custom_impls_url_env
: URL to download scanner custom implementation jars(runtime dependency).
Finally, you can download the registration client zip from below URL. {registratiionclient download server domain name/ip}/registration-client/{client_version}/reg-client.zip}
References
Run (https://github.com/mosip/mosip-infra/blob/develop/deployment/v3/mosip/regclient/create-signing-certs.sh) to generate Client.crt
and keystore.p12
.
To get the content of maven-metadata.xml
and logback.xml
check (https://github.com/mosip/mosip-helm/blob/develop/charts/regclient/templates/configmap.yaml)
Registration Processor(Regproc) is a backend processing engine to enable the ID Lifecycle management. It has several stages and services, registration packet flows through various stages depending on the type of flow.
The documentation here will guide you through the prerequisites required for the developer's setup.
Below are a list of tools required in Registration Processor:
JDK 11
Any IDE (like Eclipse, IntelliJ IDEA)
Apache Maven (zip folder)
pgAdmin
Git
Notepad++ (optional)
lombok.jar (file)
settings.xml (document)
Follow the steps below to set up Registration Processor on your local system:
1. Download lombok.jar
and settings.xml
from here.
2. Unzip Apache Maven and move the unzipped folder in C:\Program Files
and settings.xml
to conf
folder C:\Program Files\apache-maven-3.8.4\conf
.
3. Install Eclipse, open the lombok.jar
file and wait for some time until it completes the scan for Eclipse IDE and then click Install/Update
.
4. Check the Eclipse installation folder C:\Users\userName\eclipse\jee-2021-12\eclipse
to see if the lombok.jar
is added. By doing this, you don't have to add the dependency of lombok
in your pom.xml
file separately as it is auto-configured by Eclipse.
5. Configure the JDK (Standard VM) with your Eclipse by traversing through Preferences → Java → Installed JREs
.
For the code setup, clone the registration repository from and follow the guidelines mentioned in the Code Contributions.
Open the project folder where pom.xml
is present.
Open command prompt from the same folder.
Run the command mvn clean install -Dgpg.skip=true -DskipTests=true
to build the project and wait for the build to complete successfully.
After building of a project, open Eclipse and select Import Projects → Maven → Existing Maven Projects → Next → Browse to project directory → Finish
.
After successful importing of project, update the project by right-click on Project → Maven → Update Project
.
1. For the environment setup, you need an external JAR that is available here with different versions. (E.g.: You can download kernel-auth-adapter.jar
and add to project Libraries → Classpath → Add External JARs → Select Downloaded JAR → Add → Apply and Close
).
2. Clone mosip-config repository.
3. Create an empty folder inside the mosip-config
with sandbox-local
name and then copy and paste all the config files inside sandbox-local
folder except .gitignore, README and LICENSE
.
4. As Registration Processor is using two properties files, registration-processor-default
and application-default
, you will have to configure them according to your environment. The same files are available here for reference.
5. To run the server, two files are required- kernel-config-server.jar and config-server-start.bat.
6. Put both the files in the same folder and change the location attribute to sandbox-local
folder in config-server-start.bat
file and also check the version of kernel-config-server.jar
towards the end of the command.
Example:
java -jar -Dspring.profiles.active=native -Dspring.cloud.config.server.native.search-locations=file:C:\Users\myDell\mosipProject\mosip-config\sandbox-local -Dspring.cloud.config.server.accept-empty=true -Dspring.cloud.config.server.git.force-pull=false -Dspring.cloud.config.server.git.cloneOnStart=false -Dspring.cloud.config.server.git.refreshRate=0 kernel-config-server-1.2.0-20201016.134941-57.jar
.
7. Run the server by opening the config-server-start.bat
file.
The server should now be up and running.
8. Before running any application of Registration Processor, replace the below properties in bootstrap.properties
:
spring.cloud.config.uri=http://localhost:51000/config
spring.cloud.config.label=master
spring.profiles.active=default
spring.profiles.active can be dmz or mz(default)
9. An alternative approach to start the application is to place the dependency of application to be executed in the pom of MOSIP stage executor
update maven and place above mentioned properties in the bootstrap.properties
and then start MOSIP stage executor
.
Registration Processor (Regproc) is a backend processing engine to enable the ID Lifecycle management. The diagram below shows the Registration Processor along with the other modules that contribute in issuing a Unique Identification Number(UIN) for an individual. Internally, Regproc follows the SEDA architecture where data flows via multiple stages till the UIN is issued.
The relationship of Regproc with other services is explained here. NOTE: The numbers do not signify sequence of operations or control flow
Registration packets are uploaded by Registration Client to Packet Receiver.
After packet validation is done Regproc notifies pre-registration application using datasync service.
Quality of biometrics is checked using an external biometric SDK. This is done in Regproc's Quality Classifier stage.
The above data is shared by providing a URL that partners use to fetch data. This URL is obtained from Datashare service.
Regproc's ABIS Middleware stage communicates with ABIS via Activemq. The ABIS performs deduplication and sends back result to the Queue.
Regproc stores and updates applicant demographic and biometric information in ID Repository. Also Activates or deactivates applicant UIN.
Regproc calls IDA Internal Authentication Service to authenticate Applicant(for update flow), introducer, operator and supervisor(when bio auth mode is used to create packet).
After the UIN is processed the Printing Stage calls Credential Service to create credential for print. This credential will be pushed to websub and the Printing systems will consume same.
The Notification Service is used to send email/sms notification to the applicant after the request processing is completed in server.
Regproc connects to external "Manual Adjudication System" via queue. Regproc sends applicant information required for adjudication in queue and external adjudication system consumes it. The data is shared from mosip to external adjudication system based on policy.
Regproc calls Key Manager for decrypting packet and for encrypting information.
Regproc uses Masterdata Service to validate center, machine, user etc.
Regproc connects to Virus Scanner for scanning packets in Packet Receiver Stage and Packet Uploader Stage
Each Stage in regproc calls Packet Manager to read information from packet.
The Registration Processor contains several stages and services.
The registration packet flows through the various stages depending on the type of flow. See Registration Flows and Stage Sequence.
Note: The Print Stage has been renamed as the Credential Requestor Stage. For further information, please click here.
Refer to repo.
Refer to Configuration Guide.
To know more about the developer setup, read Registration Processor Developers Guide.
Refer API Documentation.
When biometric duplicates are found in ABIS, the MOSIP system sends a request for manual adjudication to the Manual Adjudication System via a queue.
The system integrator can build the Manual Adjudication System, which would be listening to the MOSIP-to-ManualAdjudication
queue for any manual adjudication requests and sends a response back in the ManualAdjudication-to-MOSIP
system after verifying the data.
The data sent to the Manual Adjudication System is driven by a policy defined in MOSIP and the specification is similar to ABIS identify request.
The manual adjudication stage in registration processor performs the following functions:
Sends the applicant's demographic and biometric information (via queue + Datashare) to Manual Adjudication System (MAS).
Receives decision from MAS and accordingly forwards the packets.
If rejected, notifies the applicant.
requestId
: request_id that is associated with each request present in reg_manual_verification
table.
referenceId
: reg_id that is associated with each request present in reg_manual_verification
table.
referenceURL
: Datashare url of biometrics, demographics(identity), audits, metainfo, documents of reg_id
gallery
: contains the matched ref_id and referenceURL of matched ref_id.
returnValue
: 1-Success, 2-Failure
candidateList
: It contains matched candidate referenceIds, count and analytics.
Datashare contains biometrics, identity documents, metainfo, audits related to particular rid and the datashare URL contains encrypted form of this data.
Note: Datashare encryption using partner key and decryption in MAS is using private key of that partner.
GET https://datashare-service/v1/datashare/get/mpolicy-default-adjudication/mpartner-default-adjudication/mpartner-default-adjudicationmpolicy-default-adjudication202011110619201EpLEjvD
The structure of the encrypted data downloaded from referenceURL in MOSIP 1.2.0 or later versions. The data downloaded would be URL-safe base64 encoded. Hence, after decoding the data will be in the below format. It will be divided into two parts after splitting using #KEY_SPLITTER#.
Block 1:
Block 1, i.e. the encrypted key data is again split into three parts, • The 1st part is VER_BYTES (version bytes). The Current version constant is set as VER_R2 and this is present in the first 6 bytes of Block 1.
• The 2nd part is the Certificate Thumbprint i.e. the key identifier which is present in the next 32 bytes after VER_BYTES.
• The 3rd part is the Encrypted Random AES Key, encrypted with the RSA OAEP - SHA256-MFG1. This constitutes the remaining 256 bytes of Block 1.
Block 2:
Block 2, i.e. the encrypted actual data is again split into two parts,
• The 1st part is the random 32 bytes which will be used as AAD in AES encryption (first 32 bytes). From this 32 bytes AAD data, the first 12 bytes is IV/Nonce.
• The 2nd part is the encrypted data which is encrypted using AES GCM PKCS5Padding.
The structure of the encrypted data downloaded from referenceURL in MOSIP 1.1.5.5 or prior versions.
The data downloaded would be base64 encoded. Hence, after decoding the data will be in the below format. It will be divided into two parts after splitting using #KEY_SPLITTER#.
Block 1:
Block 1, i.e. the encrypted key data is again split into two parts,
The first part is the Certificate Thumbprint i.e. the key identifier which is the first 32 bytes in Block 1.
The second part is the Encrypted Random AES Key which is encrypted with RSA OAEP - SHA256-MFG1. This constitutes the remaining 256 bytes of Block 1.
Block 2:
Block 2, i.e. the encrypted actual data is again split into two parts,
The 1st part is the Encrypted data, encrypted using AES GCM PKCS5Padding.
The 2nd part is IV/Nonce i.e. the last 32 bytes appended after encrypted data.
partner Id
: mpartner-default-adjudication
policy Id
: mpolicy-default-adjudication
In registration-processor-default.properties
, the possible Error codes are as follows:
This stage is applicable only if required biometrics are not present in the packet as per country configuration.
Sends applicant's demographic documents (via Queue + Datashare) to external Verification System (VS).
Receives decision from VS and accordingly forwards the packets.
If rejected, notifies the applicant.
requestId
: verification_request_id that is associated with each request present in reg_verification table
.
referenceId
: reg_id that is associated with each request present in reg_verification
table.
referenceURL
: Datashare URL of biometrics, demographics(identity) ,audits, metainfo, documents of reg_id .
Response parameters
returnValue
: 1-success, 2-failure
Datashare contains biometrics, identity, documents, metainfo, audits related to particular rid
and datashare URL contains encrypted form of this data.
Note: Datashare encryption using partner key and decryption in MAS is using private key of that partner.
GET https://datashare-service/v1/datashare/get/mpolicy-default-adjudication/mpartner-default-adjudication/mpartner-default-adjudicationmpolicy-default-adjudication202011110619201EpLEjvD
The structure of the encrypted data downloaded from referenceURL in MOSIP 1.2.0 or later versions. The data downloaded would be URL-safe base64 encoded. Hence, after decoding the data will be in the below format. It will be divided into two parts after splitting using #KEY_SPLITTER#.
Block 1:
Block 1, i.e. the encrypted key data is again split into three parts, • The 1st part is VER_BYTES (version bytes). The Current version constant is set as VER_R2 and this is present in the first 6 bytes of Block 1.
• The 2nd part is the Certificate Thumbprint i.e. the key identifier which is present in the next 32 bytes after VER_BYTES.
• The 3rd part is the Encrypted Random AES Key, encrypted with the RSA OAEP - SHA256-MFG1. This constitutes the remaining 256 bytes of Block 1.
Block 2:
Block 2, i.e. the encrypted actual data is again split into two parts,
• The 1st part is the random 32 bytes which will be used as AAD in AES encryption (first 32 bytes). From this 32 bytes AAD data, the first 12 bytes is IV/Nonce.
• The 2nd part is the encrypted data which is encrypted using AES GCM PKCS5Padding.
The structure of the encrypted data downloaded from referenceURL in MOSIP 1.1.5.5 or prior versions.
The data downloaded would be base64 encoded. Hence, after decoding the data will be in the below format. It will be divided into two parts after splitting using #KEY_SPLITTER#.
Block 1:
Block 1, i.e. the encrypted key data is again split into two parts,
The first part is the Certificate Thumbprint i.e. the key identifier which is the first 32 bytes in Block 1.
The second part is the Encrypted Random AES Key which is encrypted with RSA OAEP - SHA256-MFG1. This constitutes the remaining 256 bytes of Block 1.
Block 2:
Block 2, i.e. the encrypted actual data is again split into two parts,
The 1st part is the Encrypted data, encrypted using AES GCM PKCS5Padding.
The 2nd part is IV/Nonce i.e. the last 32 bytes appended after encrypted data.
Possible Error codes and Messages from Datashare URL
partner Id
: mpartner-default-adjudication policy Id
: mpolicy-default-adjudication
This guide contains all the information required for successful deployment and running of Resident Portal. It includes information about the Database and template scripts, seed data, roles, OIDC client setup, etc.
Resident Service DB Scripts to be run:
The master-data templates required for the Resident portal are added to the and DML excel files in repository. These scripts need to be applied to the corresponding table.
mosip-resident-client
needs to have below roles in keycloak:
RESIDENT
SUBSCRIBE_AUTH_TYPE_STATUS_UPDATE_ACK_GENERAL
SUBSCRIBE_AUTHENTICATION_TRANSACTION_STATUS_GENERAL
SUBSCRIBE_CREDENTIAL_STATUS_UPDATE_GENERAL
CREDENTIAL_REQUEST
Here is the document which explains how resident-oidc
partner is onboarded through after deployment.
This contains the UI code for Resident portal. To know more about the features and functions present on the portal, refer .
Note: The code is written in Angular JS.
Install node.js
- To build the angular code using angular cli that runs on node, recommended Node: 14.17.3, Package Manager: npm 6.14.13
Install angular cli
– To install angular cli for building the code into deployable artifacts. Follow the following steps to install angular cli on your system.
npm install -g @angular/cli
(to install angular cli)
ng --version
(to verify angular is installed in system) We recommend Angular CLI: 13.3.2
Check out the source code from GIT – To download the source code from git, follow the steps below to download source code on your local system.
git clone https://github.com/mosip/resident-ui.git (to clone the source code repository from git)
Build the code
Follow the steps below to build the source code on your system.
Navigate to the resident-ui directory inside the cloned repository.
Run the command ng build "--prod" "--base-href" "." "--output-path=dist"
in that directory to build the code.
Build Docker image
Follow the steps below to build the docker image on your system.
docker build -t name .
(replace name
with the name of the image you want, "." signifies the current directory from where the docker file has to be read.)
Example: docker build -t residentui .
Run the Docker image
Follow the steps to build docker image on your system.
docker run –d –p 80:80 --name container-name image-name
(to run the docker image created with the previous step,-d
signifies to run the container in detached mode, -p
signifies the port mapping left side of the":" is the external port that will be exposed to the outside world and right side is the internal port of the container that is mapped with the external port. Replace container-name
with the name of your choice for the container, replace image-name
with the name of the image specified in the previous step)
Example: docker run -d -p 8080:8080 --name nginx residentui
Now you can access the user interface over the internet via browser.
Example: http://localhost:8080/#/dashboard
Build & deploy the code locally
Follow the steps below to build the source code on your system.
Navigate to the resident-ui directory inside the cloned repository. Then, run the following command in that directory:
npm install
ng serve
Now, you can access the user interface via browser.
Example: http://localhost:4200
Resident services are the self-services which are used by the residents themselves via a portal. is a web-based UI application that provides residents of a country the services related to their Unique Identification Number (UIN). The residents can perform various operations related to their UIN/ VID and can also raise concerns if any through the portal.
The key features provided on the Resident portal are:
Avail UIN services using UIN/ VID (through ):
View My History
Manage My VID
Secure My ID
Track My Requests
Get Personalised Card
Share My Data
Logout
Get Information
About Registration Centers
List of supporting documents
Get My UIN (using UIN/ VID/ AID)
Verify email ID and/ or phone number
Responsive UI support- Support for the application to work seamlessly on various resolutions.
Book an appointment for new enrolment (via the pre-registration portal)
Ancillary features
Font size
Get Notifications (email and bell notifications)
View profile details of the logged in user (name, photo, and last login details)
Below is an image summarizing the features provided in Resident portal.
The relationship of Resident services with other services is listed below.
Note: The numbers do not signify sequence of operations or the control flow.
Audit Manager: Resident services sends all the audit logs to the Audit Manager.
Digital card service: Resident services uses this service to download the PDF of the UIN card or VID card.
Credential Request Generator Service: This service is used to share the credential with various partners like print partners, authentication partners, and digital card partners.
ID Repository Identity Service: Resident services uses this service to retrieve the identity information of a credential and to lock/unlock authentication types.
ID Repository VID service: This service is used to generate/revoke various types of VIDs.
ID Authentication: This service is used by Resident services to authenticate users.
MOSIP e-Signet: This is used to authenticate and authorize the users in an event of login using UIN/ VID.
Resident UI: This is the interface through which users can interact with the Resident services.
WebSub: This is used to get asynchronous notification from IDA for acknowledgment purposes.
Registration Processor: This is used to sync and upload packets for features pertaining to changes in identity data.
Packet Manager: Resident services uses this service to create packets.
Partner Management Service: Resident services uses this service to get information about various partners and policies.
Keycloak: Resident services uses this to authenticate in order to access the MOSIP internal APIs. The Resident services communicates with endpoints of other MOSIP modules via a token obtained from Keycloak.
Master data service: Resident services invokes the Master Data services to get various templates and machine details.
Notification service: Resident services uses this service to send various notifications through email or SMS.
Key Manager: Resident services uses Key Manager to encrypt or decrypt the data used across features.
The design of the Resident portal embodies the following principles:
One-stop solution: The Resident portal is designed to have components that aims to solve all the queries, issues, or discrepancies of the residents and acts as a one-stop solution for all the requirements.
Self-Sovereign: Once the ID is issued by an authority, the user/resident/citizen chooses to control and manage their data in their choice of devices.
Inclusive: The Resident portal aims to be available in all the browsers while also catering to the needs of visually impaired, dyslexic and colour-blind folks.
Presence assurance: This web-based UI application would put in all its efforts to ensure easy access to all the residents with high availability.
Works Remote: The Resident portal should be able to share credentials when data needs to be shared remotely without physical presence.
Trusted: The identity verification process on the device should be trusted so that it can be used in service delivery without any concerns.
Grievance redressal: The Resident portal ensures that in case of any concerns or grievance, the issue is raised and resolved through the portal itself.
are the self-services which are used by the residents themselves via a portal. Resident Portal is a web based UI application that provides residents of a country the services related to their Unique Identification Number (UIN).
The documentation here will guide you through the pre-requisites required for the developer' setup.
Below are a list of tools required in Resident Services:
JDK 11
Any IDE (like Eclipse, IntelliJ IDEA)
Apache Maven (zip folder)
pgAdmin
Postman
Git
Notepad++ (optional)
lombok.jar (file)
settings.xml (document)
Follow the steps below to set up Resident Services on your local system:
Install Apache Maven.
Copy the settings.xml
to ".m2" folder C:\Users\<username>\.m2
.
Install Eclipse.
Open the lombok.jar
file and wait for some time until it completes the scan for Eclipse IDE and then click Install/Update
. Specify the eclipse installation location if required by clicking the ‘Specify location…’ button. Then, click Install/Update
button to proceed.
Check the Eclipse installation folder C:\Users\userName\eclipse\jee-2021-12\eclipse
to see if the lombok.jar
is added. By doing this, you will not have to add the dependency of lombok
in your pom.xml
file separately as it is auto-configured by Eclipse.
Configure the JDK (Standard VM) with your Eclipse by traversing through Preferences → Java → Installed JREs
.
Open the project folder where pom.xml
is present.
Open command prompt from the same folder.
Run the command mvn clean install -Dgpg.skip=true -DskipTests=true
to build the project and wait for the build to complete successfully.
After building of a project, open Eclipse and select Import Projects → Maven → Existing Maven Projects → Next → Browse to project directory → Finish
.
After successful importing of project, update the project by right-click on Project → Maven → Update Project
.
a. icu4j.jar
b. kernel-auth-adapter.jar
c. kernel-ref-idobjectvalidator.jar
d. kernel-transliteration-icu4j.jar
For e.g.: You can download kernel-auth-adapter.jar
and add to project Libraries → Classpath → Add External JARs → Select Downloaded JAR → Add → Apply and Close
).
a. As Resident Services is using two properties files- resident-default.properties
and application-default.properties
. But for local running of the application, you need to provide additional/overriding properties such as secrets, passwords and properties passed by the environment which can be added in new files application-dev-default.properties
(common properties for all modules) and resident-dev-default.properties
(Resident service specific properties).
b. You will have to create both the property files according to your environment and put them in mosip-config folder
(cloned). The same files are available below for reference.
These two files are loaded by application by specifying the application names in the Application VM arguments like- Dspring.cloud.config.name=application,resident,application-dev
, resident-dev
(also detailed in later section).
To run the server, two files are required- kernel-config-server.jar
and config-server-start.bat
.
Put both the files in the same folder and point the property- Dspring.cloud.config.server.native.search-locations
to mosip-config
folder in config-server-start.bat
file and also check the version of kernel-config-server.jar
towards the end of the command.
Example:
As mentioned earlier, you will have to create property files according to your environment like resident-env-default
and application-env-default
(here env represents environment name). Both files will contain different configurations such as resident-env-default
will have config properties (e.g., secrets, passcodes, etc) used for resident-services module only and application-env-default
is used for environment specific changes and can be used for other modules as well.
In this example, currently, these two files are created for dev environment and hence the files have suffix of -dev
. If you want to run it for a different environment such as qa, create these two files with -qa
suffix and then you will also need to provide the appropriate VM argument for that referring to qa environment.
For instance,
Add mosip.resident.client.secret=***********
property to be able to use a decrypted passcode and run it in your local machine.
If you check the URLs present in application-default
file, they are set to module specific URLs, but you need to use internal/external environment URLs to access the APIs by using application-dev-default file.
In application-dev-default
file, assign environment domain URL to mosipbox.public.url
and change all other URLs with ${mosipbox.public.url}.
It results in mosipbox.public.url=internal/externalAPI
(e.g., mosipbox.public.url=https://api-internal.dev.mosip.net) and it will connect with the Development environment.
Run the server by opening the config-server-start.bat
file.
Open Eclipse and run the project for one time as Java application, so that it will create a Java application which you can see in debug configurations and then change its name. (e.g.: project name with environment - "Resident-dev").
Open the Arguments tab and specify Application VM arguments: For example, for dev environment:
Save this run configuration as ‘Resident-dev’ .
For qa
environment, you can create Resident-qa
run configuration with VM argument as below.
Example:
Click Apply
and then debug it (starts running). In the console, you can see a message like Started ResidentBootApplication in 34.078 seconds (JVM running for 38.361)
.
The APIs can be tested with the help of Postman or Swagger-UI.
Postman is an API platform for building and using APIs. Postman simplifies each step of the API lifecycle and streamlines collaboration so you can create better APIs—faster. It is widely used tool for API testing. Below you will find the APIs postman collection of resident-services.
Swagger is an interface description language for describing restful APIs expressed using JSON. You can access Swagger-UI of resident-services for dev-environment from https://api-internal.dev.mosip.net/resident/v1/swagger-ui.html
and localhost from http://localhost:8099/resident/v1/swagger-ui.html
.
Create an environment as shown in the image below.
This environment is created for dev. Give the variable name as url
and set both the values as https://api-internal.dev.mosip.net
.
In the similar way, create another environment as shown below.
This environment is created for localhost. Give the variable name as url
and set both the values as http://localhost:8099
.
Encrypted Key Data | KEY_SPLITTER | Encrypted Actual Data |
---|---|---|
Encrypted Key Data | KEY_SPLITTER | Encrypted Actual Data |
---|---|---|
Error Code | Error Message |
---|---|
Error Code | Error Message |
---|---|
Encrypted Key Data | KEY_SPLITTER | Encrypted Actual Data |
---|---|---|
Encrypted Key Data | KEY_SPLITTER | Encrypted Actual Data |
---|---|---|
Error Code | Error Message |
---|---|
Error Code | Error Message |
---|---|
For more details on how to configure the Resident OIDC client, refer .
Download lombok.jar
and settings.xml
from .
For the code setup, clone the repository and follow the guidelines mentioned in the .
For the environment setup, you need an external JAR that is available with different versions. Download below mentioned JARs with appropriate latest/appropriate versions. You will need to input appropriate artifact id and version and other inputs.
Clone .
For API documentation, refer .
Download the JSON collection available below and import in your postman. .
Block 1
#KEY_SPLITTER#
Block 2
Block 1
#KEY_SPLITTER#
Block 2
DAT-SER-003
File does not exist or File is empty
DAT-SER-006
Data share not found
DAT-SER-006
Data share usage expired
KER-ATH-401
Authentication failed
KER-ATH-403
Forbidden
RPR-MVS-000
manual verification failed
RPR-MVS-001
Registration Id should not empty or null
RPR-MVS-002
No matched reference id found for given RID
RPR-MVS-025
Manual adjudication failed
RPR-MVS-022
Registration Id should not empty or null
RPR-MVS-022
TablenotAccessibleException
in Manual verification
RPR-MVS-021
Manual verification rejected
RPR-MVS-025
Manual verification resend to queue
RPR-SYS-012
IO EXCEPTION
Block 1
#KEY_SPLITTER#
Block 2
Block 1
#KEY_SPLITTER#
Block 2
DAT-SER-003
File does not exists or File is empty
DAT-SER-006
Data share not found
DAT-SER-006
Data share usage expired
KER-ATH-401
Authentication Failed
KER-ATH-403
Forbidden
RPR-MVS-004
No Assigned Record Found
RPR-MVS-025
Multiple rids found for a reference id
RPR-MVS-022
TablenotAccessibleException in Manual verification
RPR-VER-002
Verification failed
RPR-VER-004
Resend for verification
RPR-MVS-016
Reg Id should not be null or empty
RPR-MVS-021
Manual verification rejected
Below are the steps to create the Resident OIDC client as standard steps in DevOps after e-Signet and Resident deployment.
Have a user created in keycloak with the below roles as needed for the Authorization token in the API requests:
i. ZONAL_ADMIN,
ii. PARTNER_ADMIN,
iii. POLICY_MANAGER,
iv. MISP_PARTNER,
v. PMS_ADMIN
Authenticating user to take the token and use it in all APIs invoked in further steps:
Swagger URL - https://api-internal.dev2.mosip.net/v1/authmanager/swagger-ui/index.html?configUrl=/v1/authmanager/v3/api-docs/swagger-config#/authmanager/getAllAuthTokens
Request Body:
Step 1: Creating a policy group for resident OIDC Client
Note: Since policymanager service swagger does not work, you can use postman for APIs in it.
POST - https://api-internal.dev2.mosip.net/v1/policymanager/policies/group/new
Request Body:
Make note of the prolicyGroupId
from the response.
Step 2: Creating a policy for Resident OIDC client
POST - https://api-internal.dev2.mosip.net/v1/policymanager/policies
Request Body:
Step 3: Publishing policy
POST - https://api-internal.dev2.mosip.net/v1/policymanager/policies/{{policyId}}/group/{{policyGroupId}}/publish Path params: * policyId
- resident-oidc-client-policy * policyGroupId
- from previous response
Step 4: Resident OIDC Client Partner self registration
Swagger URL: https://api-internal.dev2.mosip.net/v1/partnermanager/swagger-ui/index.html?configUrl=/v1/partnermanager/v3/api-docs/swagger-config#/partner-service-controller/partnerSelfRegistration
Request Body:
Step 5: Upload ROOT Certificate as CA certificate
i. Get certificate from keymanager with below parameters:
Swagger URL: https://api-internal.dev2.mosip.net/v1/keymanager/swagger-ui/index.html?configUrl=/v1/keymanager/v3/api-docs/swagger-config#/keymanager/getCertificate
AppID: "ROOT", refID: ""
ii. Uploaded it as CA certificate:
Swagger URL - https://api-internal.dev2.mosip.net/v1/partnermanager/swagger-ui/index.html?configUrl=/v1/partnermanager/v3/api-docs/swagger-config#/partner-service-controller/uploadCACertificate
Request Body (Example only):
Step 6: Upload RESIDENT certificate as CA certificate
i. Get certificate from keymanager with below parameters:
Swagger URL: https://api-internal.dev2.mosip.net/v1/keymanager/swagger-ui/index.html?configUrl=/v1/keymanager/v3/api-docs/swagger-config#/keymanager/getCertificate
AppID: "RESIDENT", refID: ""
ii. Uploaded it as CA certificate:
Swagger URL - https://api-internal.dev2.mosip.net/v1/partnermanager/swagger-ui/index.html?configUrl=/v1/partnermanager/v3/api-docs/swagger-config#/partner-service-controller/uploadCACertificate
Request Body (Example only):
Step 7: Upload RESIDENT : IDP_USER_INFO
certificate as Partner certificate
i. Get certificate from keymanager with below parameters:
Swagger URL: https://api-internal.dev2.mosip.net/v1/keymanager/swagger-ui/index.html?configUrl=/v1/keymanager/v3/api-docs/swagger-config#/keymanager/getCertificate
AppID: "RESIDENT", refID: "IDP_USER_INFO"
ii. Uploaded it as Partner certificate:
Swagger URL - https://api-internal.dev2.mosip.net/v1/partnermanager/swagger-ui/index.html?configUrl=/v1/partnermanager/v3/api-docs/swagger-config#/partner-service-controller/uploadPartnerCertificate
Request Body (Example only):
Step 8: Create policy Mapping request:
Swagger URL: https://api-internal.dev2.mosip.net/v1/partnermanager/swagger-ui/index.html?configUrl=/v1/partnermanager/v3/api-docs/swagger-config#/partner-service-controller/mapPolicyToPartner
Path param:
partnerId
: resident-oidc-client-partner
Request Body:
Output:
Make not of the mappingKey
.
Step 9: Approve policy mapping:
Swagger URL - https://api-internal.dev2.mosip.net/v1/partnermanager/partners/policy/{{mapping key}}
Note: This mapping key will be returned as an output from policy mapping request.
Request Body:
Step 1: Prepare the RESIDENT JWKS public key JSON.
i. Get certificate from keymanager with below parameters
Swagger URL: https://api-internal.dev2.mosip.net/v1/keymanager/swagger-ui/index.html?configUrl=/v1/keymanager/v3/api-docs/swagger-config#/keymanager/getCertificate
AppID: "RESIDENT", refID: ""
ii. Store the certificate as resident-oidc.cer
file. Make sure to replace chars with line breaks and save it*
iii. Get the KeyID of the above certificate using Get All Certificates API
Swagger URL: https://api-internal.dev2.mosip.net/v1/keymanager/swagger-ui/index.html?configUrl=/v1/keymanager/v3/api-docs/swagger-config#/keymanager/getAllCertificates
AppID: "RESIDENT", refID: ""
From the response get the keyId
. This will be the kid
attribute in the OIDC client creation step.
iv. Get JWKS public key JSON from certificate
Use the certpem2jwksjson.jar
with below command to get the JWKS of that. (Attached the Java code of that for creating automted step of this)
In the console, the JSON text of the public key of the certificate will be printed. Copy it.
v. Correct the kid
in JWKS public key JSON
In the JSON public key, replace the kid
value with the keyId
in the earlier step.
Step 2: Create the OIDC client in PMS
Swagger URL: https://api-internal.dev2.mosip.net/v1/partnermanager/swagger-ui/index.html?configUrl=/v1/partnermanager/v3/api-docs/swagger-config#/client-management-controller/createClient
In the request body, make sure to replace thebelow attributes:
publicKey
- the JWKS public key JSON from earlier step
logoUri
- Correct hostname for the Resident UI
redirectUris
- Correct the hostname for Resident Service
Request Body (Example only):
The response will contain the Resident OIDC client ID in clientId
attribute.
Step 3: Configure the Resident OIDC client in resident-default.properties
.
Configure the above obtained Resident OIDC client ID resident-default.properties
with property name mosip.iam.module.clientID
.
Note: This will need a restart of the resident service if it is already deployed.
The provided guide presents a list of essential properties that can be customised according to a specific installation. Please note that this list is not exhaustive but rather acts as a checklist to review properties that are expected to differ from their default settings. If you require access to all properties, please refer to the files mentioned below.
Resident Service uses the following configuration files:
Properties used for configuring the database.
These are the authentication types allowed for a resident and default unlock duration.
Templates type codes for authentication types:
Below are the properties used for validation purpose:
Property used to get the identity mapping json
Properties used for machine specification and center:
This is an exclusion list of URL patterns that should not be a part of authentication and authorization.
Properties used to define the endpoints that should not be part of authentication.
These properties (comma separated values) are used to define the keys of the properties to be exposed to UI.
This property will directly apply the certs URL without the need for constructing the path from issuer URL. This is useful for keeping different certs URL for integrating with MOSIP e-Signet for offline token validation.
Used in open-id-connect
based login with UIN/VID in MOSIP e-Signet(IDP)
Used for login purpose:
Properties used to define application and reference id.
For Minio: object.store.s3.url=http://minio.minio:9000
For AWS: object.store.s3.url=s3.${s3.region}.amazonaws.com
Property used to enable virus scanner flag
Property used to get the vid policy json:
Property used to get the UI schema json
This property is used to get the data format from MVEL file:
Below websub properties are used for authentication type status event:
Below websub properties used for authentication transaction status event:
Below websub properties used for credential status event:
Properties used to get the data format from MVEL file.
The Registration centers will be searched based on the distance value in meters from the Geo location identified
Configure Time limit for OTP Flooding scenario (in minutes).
Define property name in below format-
resident..template.property.attribute.list
Commenting or removing this property will disable reference validator.
For validating request time as per before and after time limit (in seconds) in contact-details/update API.
The java.time.format.FormatStyle enum to use for date time formatting based on locale. Allowed values with examples are:
FULL ('Tuesday, April 12, 1952 AD' or '3:30:42pm PST')
LONG('January 12, 1952')
MEDIUM ('Jan 12, 1952')
SHORT ('12.13.52' or '3:30pm')
Current value is MEDUIM. For more details refer to the enum.
URL pattern for logging filter. For example, "/callback/" .Defaults to "/".
This will print the request details such as URL, headers and body for debugging purpose. Default is false.
The Resident Portal is a user-friendly web-based platform designed to assist residents in accessing various services associated with their Unique Identification Number (UIN). This portal offers a range of essential services such as:
UIN services using UIN/VID (through e-Signet):
View My History
Manage My VID
Secure My ID
Track My Requests
Get Personalised Card
Share My Data
Logout
Get Information
About Registration Centers
List of supporting documents
Get My UIN (using UIN/ VID/ AID)
Verify email ID and/ or phone number
Responsive UI support- Support for the application to work seamlessly on various resolutions.
Book an appointment for new enrolment (via the pre-registration portal)
Ancillary features
Font size
Get Notifications (email and bell notifications)
View profile details of the logged in user (name, photo, and last login details)
Below is the detailed explanation of each of the features mentioned above.
Residents can use these services to view, update, change, manage or share their data. They can also report an issue in case of a grievance.
Pre-requisites: To login into the Resident Portal, the resident should have their unique virtual ID (VID) or Unique Identification Number (UIN) and also have access to the registered email ID/ phone number to be able to receive the OTP.
Resident accesses the Resident Portal dashboard page.
Resident clicks on UIN Services
.
The login screen appears and the resident can choose one of the options to log in.
To login with OTP authentication, the resident clicks on Log in here> More ways to login > Login with OTP
.
Resident needs to enter valid VID in the Enter Your VID
text field and check the box I'm not a robot
.
Next, the resident clicks on the Get OTP
button.
The resident receives the OTP on the registered phone number and email ID.
The resident needs to enter the valid OTP received within stipulated time and click the Verify
button.
The resident is then navigated to the Consent page. On this page, the Essential and Voluntary claims are displayed. Additionally, they will also have to allow access to their data in Authorize Scope section to avail various services. Based on the consent provided by the resident, the services will be made available for modification.
The resident has the choice to select from the list of Voluntary claims while the Essential claims are mandatory.
The resident should now click the Continue
button. The system navigates the resident to the UIN Services menu page from where they can avail various services.
Note: Consent page will be available only for first time login.
The residents can view the history of all the transactions associated with their logged-in UIN/ VID. They can also view their details and if any unaccounted entry is found, a report can be raised against the same.
The residents can perform the following:
Search: The residents can enter an Event ID to search a particular event.
Filter based on date (From date and To date): The Residents can put a “from” and “to” date in order to get the list of all the events performed in the chosen date range.
Filter based on status (Success/ In Progress/ Failure): The Residents can filter based on the status of the event. E.g.: If they want to view all “In Progress” events, they can choose the status as “In Progress”. Additionally, they can also select any combination of the above three options.
Filter based on History Type (Authentication, ID Management, Data Update, Data Share, Service Requests): The Residents can filter based on the type of event. Additionally, they can also choose any combination of the above five options.
Authentication Request: This includes all the authentication and e-KYC requests.
ID Management Request: This includes the below services:
Manage My VID (Generate/Revoke VID)
Verify phone number/email ID
Secure My ID (Lock/unlock various authentication types)
Data Update Request: This includes the below services:
Update my UIN (demographic data and contact data)
Data Share Request: This includes the below services:
Share with a partner
Service Request: This includes the below services:
Download configured card
Physical card
Get my UIN
Book an appointment (lost UIN, Update UIN, Pre-registration, other)
Go button: Residents can click on the Go
button once they are done selecting all the required filters.
Download the PDF of the results: The residents can download the PDF version of the search result populated.
Clicking on the accordion/ the caret of a particular event, the following options will appear:
a. View Details: The residents can view the details about an event by clicking on View Details
. They will be redirected to Track My Request
page with pre-filled EID where they can see further details about the event.
b. Pin Event to the top: The residents can pin the events to the top of the list based on their preference. Currently, this is configured for up to 3 events but it can be customized as per country’s requirements. Also, the resident can unpin the pinned events by clicking Unpin from Top
.
c. Report a grievance: The residents can report a grievance in case of fraud or for any event not initiated by them. On clicking Report an Issue
, the resident will be redirected to the Grievance Redressal Form
page where they will see a set of pre-filled data as well as a set of data to be filled.
Pre-filled data:
Name
Event ID (EID)
Registered Email ID
Registered Mobile Number
Data to be filled:
Alternate Email ID
Alternate Mobile Number
Comments
Once the event is completed, a message is displayed containing the grievance tracking ID.
Below are the images with different filters on this page.
On clicking Manage My VID
, the resident will be taken to a page where they can view details of the existing VIDs, generate new VID, revoke existing VID or download a VID card.
The following types of VIDs can be seen based on the VID policy:
Perpetual VID
Temporary VID
One-time VID
Note: The resident can get to know about the features of a particular VID by hovering over the “i” symbol.
The residents can perform the following:
Create a new VID : The residents can click on the Create
button against any of the VID type selected. They can click on Yes
to proceed. Once the event is completed, a message is displayed containing the Event ID along with a link to track the service.
Revoke an existing VID: The residents can click on the Delete icon to revoke an existing VID. They can click on Yes
to proceed. Once the event is completed, a message is displayed containing the Event ID along with a link to track the service.
Download a VID card:
a. The residents can click on the Download icon to initiate the download process. They can click on Download
to proceed. Once the event is completed, a message is displayed containing the Event ID, a link to track the service and the password combination.
b. Once the card is ready to download, they will receive a notification for the same under the bell icon displayed on the top right corner of the screen and as an Email notification.
c. On clicking on the notification, the resident will be taken to Track My Request
page with pre-filled EID.
d. On this screen, the resident will be able to download the card by clicking on Download My VID card
button on the bottom left corner of the screen.
e. The downloaded card will be a password protected PDF. The residents can view the downloaded VID card by entering the password combination displayed on the screen.
View VID number: All the VID numbers will be masked by default. The residents can view the unmasked version of VID by clicking on eye icon next to the VID number.
On clicking Secure My ID
, the residents can view the status of all the authentication types. They can choose to lock or unlock authentication types like the following:
Email authentication
Mobile authentication
Demographic authentication
Fingerprint authentication
Iris authentication
Face authentication
The residents can perform the following,
View the current status of authentication types: The lock icon on each card indicates the current status of the authentication type. E.g.: If the lock is open, the authentication type is unlocked which means the residents can authenticate themselves using that particular authentication type and vice versa.
Lock/ unlock the authentication types: To lock/ unlock a particular authentication type, the residents can click on lock/ unlock button. Once the preferences of each authentication type is selected, the residents can click on Submit
to save the changes and click Yes
to proceed. Once the event is completed, a message is displayed containing the Event ID along with a link to track the service.
On clicking Track My Requests
, the residents can track the status of an EID associated with the logged-in UIN/ VID. They can also view and download the detailed information about the entered EID like:
Event ID- This is the unique ID provided against each event
Event Type- This is the feature that is being availed. E.g.: Lock/unlock authentication types
Event Status- This is the status of the event which can hold values like Success, Failure or In-Progress
Individual ID- This is the type of individual ID that was used to login. E.g.: VID or UIN
Summary- This the the detailed description of the event.
Timestamp- This the time when the event occurred.
Authentication Mode- This is the authentication mode used to login. E.g.: OTP or Biometric or QR code
Partner Logo- This is the logo of the registered partner.
Partner Name- This is the name of the registered partner.
Attribute List- This is the list of attributes shared with the registered partner.
Purpose- This is the purpose of sharing data with the registered partner as input by the resident.
The resident can reach Track My Requests
page by the following ways:
UIN services > View history > Click on the event tile > View details
By clicking the bell icon
UIN services > Track My Requests
Note:
Residents can download their updated UIN /VID card.
Report a grievance: The residents can report a grievance in case of fraud or for any event not initiated by them. On clicking Report an Issue
, the resident will be redirected to the Grievance Redressal Form
page where they will see a set of pre-filled data as well as a set of data to be filled.
On clicking Get Personalised Card
, the residents can select the data to be added to their credential. They can preview the chosen data and download it. To be able to download the card, residents should select at least 3 attributes from the list mentioned below:
Name- Name of the resident. They can choose the format in which they want the name to be displayed.
Date of Birth- Date of birth of the resident. They can choose the format in which they want the date of birth to be displayed.
UIN- Unique Identification Number. They can choose to mask or unmask the UIN.
Perpetual VID- Perpetual Virtual ID. They can choose to mask or unmask the VID.
Phone number- Registered phone number of the resident. They can choose to mask or unmask the phone number.
Email ID- Registered email ID of the resident. They can choose to mask or unmask the email ID.
Address- Address of the resident. They can choose the format in which they want the address to be displayed.
Gender
Photo
These details can be previewed as and when the attributes are chosen.
Once the event is completed, a message is displayed containing the Event ID along with a link to track the service.
On clicking Share My Data
, the residents can choose the data to be shared with any of the registered partners to avail various third party services.
To share the data, residents should select at least 3 attributes from the list mentioned below:
Name- Name of the resident. They can choose the format in which they want the name to be displayed.
Date of Birth- Date of birth of the resident. They can choose the format in which they want the date of birth to be displayed.
UIN- Unique Identification Number. They can choose to mask or unmask the UIN.
Perpetual VID- Perpetual Virtual ID. They can choose to mask or unmask the VID.
Phone number- Registered phone number of the resident. They can choose to mask or unmask the phone number.
Email ID- Registered email ID of the resident. They can choose to mask or unmask the email ID.
Address- Address of the resident. They can choose the format in which they want the address to be displayed.
Gender
Photo
These details can be previewed as and when the attributes are chosen.
Additionally, the residents have to:
Select the partner with whom they want to share their data from a dropdown list of registered partners.
Enter the purpose of sharing the data with the registered partner.
On clicking the Share
button, the resident will have to provide consent to share their data with the external partner.
Once the event is completed, a message is displayed containing the Event ID along with a link to track the service.
The Resident Portal menu bar contains the following:
Font Size- Residents can alter the size of the font based on their preferences.
Language- Residents can select the language of preference.
Bell icon Notification- Residents can view the notifications of all the asynchronous events in chronological order.
Profile Icon- Residents can view the following:
Name of the logged in user
Photo of the logged in user
Last login details
Logout option
The residents can book an appointment for registration using the pre-registration portal. To do so, they can click on Book an appointment
tile which will redirect them to the pre-registration portal. To know more about pre-registration portal, refer to this link Pre-registration.
The residents can use this feature to verify their registered email ID or phone number.
Steps to verify email ID/ phone number:
Resident clicks either on Verify email ID or Verify phone number option
Enter the UIN/VID.
Select I’m not a robot
against the captcha and click on Send OTP
.
Resident enters the OTP received on the requested channel and clicks on Submit
.
Based on the scenario, any of the below three messages will be displayed:
a. Email ID/ phone number successfully verified: On successful verification, a message is displayed on the screen saying that the phone number/ email ID has been successfully verified.
b. Email ID/ phone number was already verified: If the verification has been previously completed, a message is displayed saying the email ID/ phone number was already verified.
c. Email ID/ phone number does not exist: If there is no email ID/ phone number linked to the UIN/VID, a message is displayed saying no email ID/ phone number was found associated to this UIN/VID.
The residents can use this feature for one of the following:
Download their UIN card
Check the status of their Application ID (AID)
Steps to download the UIN:
Resident clicks on Get My UIN
Enter the AID/UIN/VID.
Select I’m not a robot
against the captcha and click on Send OTP
.
Resident enters the OTP received on the registered email ID/ phone number and clicks on Submit
.
The default PDF of UIN card will be downloaded and a success message is seen stating that the UIN has been successfully downloaded.
Steps to check the status of the AID:
Note: If the UIN is not ready, then the AID is used to get status else UIN card will be downloaded using AID too.
Resident clicks onGet My UIN
.
Enter the AID.
Select I’m not a robot
against the captcha and click on Send OTP
.
Resident enters the OTP received on the registered email ID/ phone number and clicks on Submit
.
The status of the AID will be shown.
Residents can view the list of supported documents in the PDF format and download the same. Also, some sample documents are available for reference.
Residents can search for Registration Centres on the basis of below two mechanisms:
Nearby centers: The resident will be asked to allow permission for location access in order to enable the system to suggest the nearest Registration Centres.
Manually look for centers: If the Resident wants to manually look for a center, they can do so by choosing a level in location hierarchy from the drop-down (e.g.: Region, Province, Postal Code) and entering the value against the same.
They can also download the PDF version of the result displayed on the screen for reference.
Object Store is a storage module for MOSIP named as Khazana. The module is an abstraction of storage layer used across Registration Client, Packet Manager, Datashare or Durian for packets and biometric data.
Khazana provides following adapters to store objects
POSIX - Supports storage of packets on a filesystem. Its typically used by registration client to store packets locally on the machine. This adapter is not receommended for usage in low latency environments like packet manager.
S3 - S3 is one of the well known API for object stores. AWS Java S3 Client is used in Khazana to support any S3 compliant object storage solutions.
Object Store is used for following purpose wihin mosip
Registration Client - Encrypted packets
Pre-registration - Uploaded Documents
Idrepo - Individual's biometrics and documents
Datashare - On demand individual's biometrics, documents and other information.
As part of our sandbox deployment we have provided an example use case with minio for on-prem deployment and AWS S3 with AWS deployment. Object Store is installed as part of default sandbox deployment.
Note: Please note its important to choose the right partner for object storage and work with them to scale acordingly. Please follow the hardware estimate for Object Store based on respective Object store products.
The below is the list of S3 Java API's used by MOSIP. This can be used to understand the vendor compatibility. Khazana does not use any internal business logic and is purely an storage abstraction layer.
Java API Used by MOSIP | S3 Documentation URL |
---|---|
Version: 1.1.5.5
Release date: May 31st, 2022
Note: This method of installation is deprecated for the newer releases of MOSIP + 1.2.x
WebSub provides a common mechanism for communication between publishers of any kind of Web content and their subscribers, based on HTTP web hooks. Subscription requests are relayed through hubs, which validate and verify the request. Hubs then distribute new and updated content to subscribers when it becomes available. WebSub was previously known as PubSubHubbub. For more information, read .
In MOSIP, WebSub is used to share data with services and partners. message broker has been used to implement the WebSub APIs. Message brokers are a natural fit for the implementation of WebSub hubs as they serve a similar purpose.
The relationship of WebSub with other services is explained here. NOTE: The numbers do not signify sequence of operations or control flow.
Topic is registered and published or subscribed by MOSIP Services.
Content delivery and Intent Verification is done by WebSub to Mosip Services.
Content delivery and Intent Verification is done by WebSub to Partners.
Topic is registered and published or subscribed by the Partners.
Data and metadata needed for delivery, delivery reports and other functionalities are stored in Kafka.
module provides a common mechanism for communication between publishers of any kind of Web content and their subscribers, based on HTTP webhooks.
Below is a list of tools required in WebSub:
Ballerina (Swan-Lake)
Any IDE (like Vs Code)
Kafka
Postman (any HTTP Client)
Git
Download and install it as per instructions. (Use bal -v
to ensure installation and version).
Download and install it.
For the code setup, clone the repository and follow the guidelines mentioned in the .
Open the hub and consolidator folders where Ballerina.toml
is present.
Open the command prompt from the same folder.
Run the command bal build
to build the hub and consolidator.
Open the project in VS Code either by open with vs code
or from File -> Open Folder
.
Run Configure and run Kafka, update KAFKA_BOOTSTRAP_NODE
in Config.toml
to point to your Kafka.
WebSub consists of consolidator and hub.
Consolidator should be started first, Got to consolidator -> java -jar target/bin/<Jarname>
. (Config.toml should be in the same place where you are running this command).
Start WebSub with the same approach.
MOSIP is deployed as a combination of microservices. MOSIP deployment as mircroservices adds more scalability, robustness, resilience and fault tolerance. MOSIP services are grouped as multiple modules like Kernel, prereg, regproc etc. These modules can be deployed independently with all its dependencies as per the business needs.
MOSIP releases the below mentioned artifacts in open source:
: Artifacts released into Maven Central Repository after successful compilation of tagged code from all Mosip repositories.
: Docker images for all MOSIP services.
MOSIP helm: Packaged charts for every MOSIP service.
helm repo add mosip https://mosip.github.io/mosip-helm
: Scripts to create cluster and configure it for MOSIP.
Deployment scripts: These scripts are a part of to install the helm charts in a desired sequence.
MOSIP by default supports two ways of installation:
- A simple sandbox-based implementation with very low resources to help understand the working modules (deprecated).
- A scalable deployment with a service mesh, HA and other security protection (supported from v1.2.0.1-B1).
Apart from these installation models, countries can adopt a model of their choice. We recommend using Kubernetes or equivalent container orchestration tools for better management of the microservices.
MOSIP uses DB for all relational data storage. The DB creation SQL scripts are located under /db_scripts
the folder of the module repository. In , Postgres is installed as a docker inside the cluster. However, in production deployment, typically, Postgres will be installed external to the cluster.
Entity relationships diagrams for all databases used in MOSIP are given below.
Connection details
{module_name
}_database_url
{module_name
}_database_username
{module_name
}_database_password
Hibernate configurations
javax.persistence.jdbc.driver
hibernate.dialect
hibernate.jdbc.lob.non_contextual_creation
hibernate.hbm2ddl.auto
hibernate.show_sql
hibernate.format_sql
hibernate.connection.charSet
hibernate.cache.use_second_level_cache
hibernate.cache.use_query_cache
hibernate.cache.use_structured_entries
hibernate.generate_statistics
logging.level.org.hibernate.SQL
logging.level.org.hibernate.type
These are some of the reference settings of a production database. It is expected that these are reviewed and finalized for a given deployment.
This page lists all the technologies used in building MOSIP. Free and open-source software with clear long-term support availability has been chosen. For a deployment, certain choices can be replaced with other free or commercial options.
Domain | Tools/Technologies | Version | Licence Type | Commercial | Production | Cost |
---|
Refer for further details.
To know more about the developer setup, read .
Copy websub-service.toml
and websub-consolidator.toml
file from to hub and consolidator folder respectively and rename both of them as Config.toml
(case-sensitive).
The APIs can be tried with the help of .
getConnection(bucketName).getObject(bucketName, finalObjectName)
getConnection(bucketName).getObjectMetadata(bucketName, finalObjectName)
doesBucketExistV2(bucketName)
createBucket(bucketName)
getObjectMetadata()
getObjectMetadata().getUserMetadata()
addUserMetadata(m.getKey(), m.getValue())
PutObjectRequest(bucketName, finalObjectName, s3Object.getObjectContent(), objectMetadata)
getRequestClientOptions()
setReadLimit(readlimit)
putObject(putObjectRequest)
deleteObject(bucketName, objectName)
listObjects(account, searchPattern)
getObjectSummaries()
listObjects(searchPattern)
getObjectSummaries()
doesObjectExist(bucketName, finalObjectName)
GetObjectTaggingRequest(bucketName,finalObjectName)
getObjectTagging(getObjectTaggingRequest)
SetObjectTaggingRequest(bucketName,finalObjectName,objectTagging)
setObjectTagging(setObjectTaggingRequest)
Version: 1.2.0.1-B3 (Latest stable release)
Release date: 14th April 2023
Version: 1.2.0.1-B2
Release date: 8-Jan 2023
Version: 1.2.0.1-B1
Release date: 14-Oct 2022
The Authdemo service is utilized to execute the IDA APIs that are employed by API-testrig and DSLrig.
The purpose of the Authdemo Service is to showcase the functionality of authentication.
It can be considered as a simplified iteration of an authentication service, serving as a mock or prototype for the purpose of testing.
When prompted, input the NFS host, it's pem-key, ssh login user of NFS server.
Install script will create the NFS directories i.e., /srv/nfs/mosip/packetcreator-authdemo-authcerts
to store the certificates generated by Authdemo service.
These certificates will be used by API-testrig, orchestrator and packetcreator.
API-testRig tests the working of APIs of the MOSIP modules.
MOSIP’s successful deployment can be verified by comparing the results of API-testrig with the testrig benchmark.
When prompted, input the hour of the day to execute the API-testrig.
Daily API-testrig cron job will be executed at the very opted hour of the day.
The reports will move to the object store ( i.e., s3/minio) under automationtests
bucket.
Packetcreator will create packets for DSL orchestrator.
Note: It is recommended to deploy the packetcreator on a separate server/ cluster from where the other DSL orchestrators can access this service.
When prompted, input the NFS host, it's pem-key, ssh login user of NFS server.
Install script will create two NFS directories i.e., /srv/nfs/mosip/packetcreator_data
, /srv/nfs/mosip/packetcreator-authdemo-authcerts
.
Packetcreator_data
contains biometric data which is used to create packets.
The default packetcreator_data
can be accessible from here.
Copy the packetcreator_data
from the link mentioned above to the NFS directory /srv/nfs/mosip/packetcreator_data
.
Ensure to use the same NFS host and path i.e., /srv/nfs/mosip/packetcreator-authdemo-authcerts
for Authdemo and packetcreator service.
When prompted, input the kubernetes ingress type (i.e., Ingress/Istio) and DNS as required if you are using the Ingress-nginx.
DSLrig will test end-to-end functional flows involving multiple MOSIP modules.
The Orchestrator utilizes the Packet Creator to generate packets according to the defined specifications. It then communicates with the Authdemo Service making REST API calls to perform authentication-related actions or retrieve the necessary information.
When prompted, input the NFS host, it's pem-key, ssh login user of NFS server.
Install script will create NFS directories, i.e., /srv/nfs/mosip/dsl-scenarios/sandbox.xyz.net
to store the DSL scenario sheet.
The Default template for DSL scenario sheet can be accessible from here.
Copy the scenario csv from the above link to the NFS directory /srv/nfs/mosip/dsl-scenarios/sandbox.xyz.net
. Make sure to rename the csv files by replacing env
with your domain ex: sandbox.xyz.net
.
To run the dslorchestrator for sanity only, update the dslorchestrator
configmap TESTLEVEL
key to sanity
.
The reports will move to object store (i.e., s3/minio) under dslreports
bucket.
Operating System | CentOS | 7.7 | MIT License | Yes | Yes | NA - Part of Azure |
Operating System | Ubuntu Server | 20.04 | Free | No | No | NA |
Infrastructure | Cloud - Azure/AWS | NA - Cloud tool | Commercial | Yes | Depends on Deployment Arch. | Depends on Deployment Arch. |
Development - Language Runtime | Java SE 11 | OpenJDK 11 | Oracle Binary Code License | No | Yes | NA |
Development - Expression language | mvel2 | 2.4.7.Final |
Development - Scheduling | quartz | 2.2.1 |
Development - File Server | tus-java-client | 0.4.3 |
Development - Internalization | nv-i18n | 1.29 |
Development - Cryptography | TPM | 2.0 |
Development - UI Application framework | JavaFx | OpenJFX 11 | GPL v2 + Classpath | No | Yes | NA |
Development - Application Framework | Vert.x | 3.9.1 | Apache License 2.0 | No | Yes | NA |
Development - Application Framework | Spring | 5 | Apache License 2.0 | No | Yes | NA |
Development - Utilities | Apache commons(60+ to be considered) | Latest version | Apache License 2.0 | No | Yes | NA |
Development - Data Grid | Apache Ignite | 2.4.0 | Apache License 2.0 | No | Yes | NA |
Development - Object Mapper | Orika | 1.5.2 | Apache License 2.0 | No | Yes | NA |
Development - validator | Hibernate validator | 5.4.2 | Apache Software License 2.0 | No | Yes | NA |
Development - Encryption | BouncyCastle | 1.59 | Adaptation of MIT X11 License | No | Yes | NA |
Development - JSON marshal/unmarshal | Jackson | 2.9.5 | Apache License 2.0 | No | Yes | NA |
Development - Device Driver | RXTX | RXTX-2-2-20081207 | LGPL v 2.1 | No | Yes | NA |
Development - Unit Testing | Junit | 5.x and above | Common Public License - v 1.0 | No | No | NA |
Development - Unit Testing | Junit | 4.x and above | Common Public License - v 1.0 | No | No | NA |
Development - Log | logback | 1.2.3 | GNU Lesser GPL Version 2.1 | No | Yes | NA |
Development - Templating | velocity | 1.7 | Apache License 2.0 | No | Yes | NA |
Development - Tools | Open street view | NA - Cloud tool | Open Database License (ODbL) | No | Yes | NA |
Development - IDE | Eclipse Oxygen | 4.7.3 | Eclipse Public License Version 2.0 | No | No | NA |
Development - Unit Testing | Karma | 2.0.x | MIT License | No | No | NA |
Development - Unit Testing | Jasmine | 2.6.1 | MIT License | No | No | NA |
Development - API Documentation | Swagger | 3.13.2 | Apache License 2.0 | No | No | NA |
Development - Application Server | Tomcat server | 8 | Apache License 2.0 | No | Yes | NA |
Development - Orchestration | Apache Camel | 2.19.3 | Apache License 2.0 | No | Yes | NA |
Development - WebSub | Ballerina Websub | slbeta2 | Apache License 2.0 | No | Yes | NA |
Development - Database | H2 DB | 1.4.197 | No | Yes | NA |
Development - Database | PostgreSQL | Server: 10 | Postgres License BSD 2-clause "Simplified License" | Yes | No | NA |
Development - Database | Derby DB | 10.13.1.1 |
Development - Database Modeling tool | PG Data Modeler | 0.9.2 | Commercial | No | Yes | Nominal |
Development - Scanner library | 7 | Commercial |
Development - Code quality | Sonar | 7.2 | Open Source License | No | No | NA |
Development - UI Designs | Pencil Project | 3.0.4 | GNU Public License version 2 | No | No | NA |
Development - TPM Java client | TSS.Java | 0.3.0 |
Testing tools | Rest-assured | 3.0.0 and above | Apache License 2.0 |
Testing tools | WireMock or Citrus framework | 2.16.0 or respectively | Apache License 2.0 | No | No | NA |
Testing tools | JMeter | 5.3.x | Apache License 2.0 | No | No | NA |
Testing tools | Burp suite Professional + | 9.0.3.7 | PortSwigger - Burp suite Professional + / V1.7.33 | No | No | NA |
Testing tools | TestNG | 6.11 | Apache License 2.0 | No | No | NA |
Testing tools | awaitility | 4.0.3 | Apache License 2.0 | No | No | NA |
Testing tools | testfx | 4.0.16-alpha | EUPL1.1 | No | No | NA |
Testing tools | extentreports | 3.1.5 | Apache License 2.0 | No | No | NA |
Testing tools | selenium-java | 3.141.59 | Apache License 2.0 | No | No | NA |
DevOps tools | Jira | 6.4 and above | Not Open source |
Testing tools | 12.x | No | NA | 12.0.3 | Open Source License |
DevOps tools | SonarLint | v3.5 | GNU GPL |
DevOps tools | GitHub | 2.7.x | Commercial - Github |
DevOps tools | SonarQube | 6.7.3 LTS | GNU GPL |
DevOps tools | Maven | 3.53.x | Apache License 2.0 |
DevOps tools | Docker | 18.03.x CE | Apache 2.0 |
DevOps tools | Ansible | 2.2 | GNU GPL v3.0 |
DevOps tools | Github actions | NA - Cloud tool |
DevOps tools | Travis | NA - Cloud tool | MIT License |
DevOps tools | Glowroot | Apache License 2.0 |
DevOps tools | Prometheus | Apache License 2.0 |
DevOps tools | Grafana | Apache License 2.0 |
DevOps tools | Python | 3.x |
Messaging | ActiveMQ | Apache License 2.0 |
Messaging | Apache Kafka | Apache License 2.0 |
Caching | Hazelcast | Apache License 2.0 |
Object Store | MinIO | GNU AGPL v3 |
Secure Code Scanning | SonarQube with OWASP plugin will be used |
Web Server/HTTP proxy server | Nginx | NA - Cloud tool |
IAM | KeyCloak |
MOSIP modules are deployed in the form of microservices in a Kubernetes cluster.
is used as a trust network extension to access the admin, control, and observation pane
It is also used for on-the-field registrations.
MOSIP uses AWS load balancers for:
SSL termination
Reverse Proxy
CDN/Cache management
Loadbalancing
Kubernetes cluster is administered using the and
In V3, we have two Kubernetes clusters:
Observation Cluster - This cluster is a part of the observation plane and it helps in administrative tasks. By design, this is kept independent of the actual cluster as a good security practice and to ensure clear segregation of roles and responsibilities. As a best practice, this cluster or its services should be internal and should never be exposed to the external world.
is used for managing the Mosip cluster.
in this cluster is used for cluster user access management.
It is recommended to configure log monitoring and network monitoring in this cluster.
In case you have a internal container registry, then it should run here.
MOSIP Cluster - This cluster runs all the MOSIP components and certain third party components to secure the cluster, API’s and Data.
VM’s required have any Operating System and can be selected as per convenience.
In this installation guide, we are referring to Ubuntu OS
throughout.
All the VM's should be able to communicate with each other.
Need stable Intra network connectivity between these VM's.
All the VM's should have stable internet connectivity for docker image download (in case of local setup ensure to have a locally accesible docker registry).
During the process, we will be creating two loadbalancers as mentioned in the first table below:
Server Interface requirement as mentioned in the second table:
Note:
Only proceed to DNS mapping after the ingressgateways are installed and the load balancer is already configured.
The above table is just a placeholder for hostnames, the actual name itself varies from organisation to organisation.
As only secured https
connections are allowed via nginx server, you will need the below mentioned valid ssl certificates:
AWS account and credentials with permissions to create EKS cluster.
Save ~/.kube/config
file with another name. (IMPORTANT. As in this process your existing ~/.kube/config
file will be overridden).
Save .pem
file from AWS console and store it in ~/.ssh/
folder. (Generate a new one if you do not have this key file).
Create a directory as mosip in your PC and
clone k8’s infra repo with tag : 1.2.0.1-B2 inside mosip directory.
git clone https://github.com/mosip/k8s-infra -b v1.2.0.1-B2
clone mosip-infra with tag : 1.2.0.1-B2 inside mosip directory
git clone https://github.com/mosip/mosip-infra -b v1.2.0.1-B2
Set below mentioned variables in bashrc
source .bashrc
Note:
Above mentioned environment variables will be used throughout the installation to move between one directory to other to run install scripts.
A Wireguard bastion host (Wireguard server) provides secure private channel to access MOSIP cluster. The host restricts public access, and enables access to only those clients who have their public key listed in Wireguard server. Wireguard listens on UDP port 51820.
Setup Wirguard VM and wireguard bastion server:
Create a Wireguard server VM in aws console with above mentioned Hardware and Network requirements.
Edit the security group and add the following inbound rules in aws console
type ‘custom TCP', port range ‘51820’ and source '0.0.0.0/0’
type ‘custom UDP', port range ‘51820’ and source '0.0.0.0/0’
Setup Wireguard server
SSH to wireguard VM
Create directory for storing wireguard config files.
mkdir -p wireguard/config
Install and start wireguard server using docker as given below:
Note:
* Increase the no of peers above in case needed more than 30 wireguard client confs. (-e PEERS=30
)
* Change the directory to be mounted to wireguard docker in case needed.
All your wireguard confs will be generated in the mounted directory. (-v /home/ubuntu/wireguard/config:/config
)
Setup Wireguard Client in your PC
Assign wireguard.conf
:
SSH to the wireguard server VM.
cd /home/ubuntu/wireguard/config
assign one of the PR for yourself and use the same from the PC to connect to the server.
create assigned.txt
file to assign the keep track of peer files allocated and update everytime some peer is allocated to someone.
Use ls
cmd to see the list of peers.
get inside your selected peer directory, and add mentioned changes in peer.conf:
cd peer1
nano peer1.conf
Delete the DNS IP.
Update the allowed IP's to subnets CIDR ip . e.g. 10.10.20.0/23
Share the updated peer.conf
with respective peer to connect to wireguard server from Personel PC.
add peer.conf
in your PC’s /etc/wireguard
directory as wg0.conf
.
start the wireguard client and check the status:
Once Connected to wireguard you should be now able to login using private ip’s.
Observation K8s Cluster setup
Setup rancher cluster,
cd $K8_ROOT/rancher/aws
Copy rancher.cluster.config.sample
to rancher.cluster.config
.
Review and update the below mentioned parameters of rancher.cluster.config
carefully.
name
region
version: “1.24“
instance related details
instanceName
instanceType
desiredcapacity
volumeSize
volumeType
publicKeyName.
update the details of the subnets to be used from vpc
Install
eksctl create cluster -f rancher.cluster.config
Wait for the cluster creation to complete, generally it takes around 30 minutes to create or update cluster.
Once EKS K8 cluster is ready below mentioned output will be displayed in the console screen.
EKS cluster "my-cluster" in "region-code" region is ready
The config file for the new cluster will be created on ~/.kube/config
Make sure to backup and store the ~/.kube/config
with new name. e.g. ~/.kube/obs-cluster.config
.
Change file permission using below command:
chmod 400 ~/.kube/obs-cluster.config
Set the KUBECONFIG
properly so that you can access the cluster.
export KUBECONFIG=~/.kube/obs-cluster.config
Test cluster access:
kubect get nodes
Command will result in details of the nodes of the rancher cluster.
Once the rancher cluster is ready we need ingress and storage class to be set for other applications to be installed.
Check the following on AWS console:
An NLB has been created. You may also see the DNS of NLB with
Edit listner "443". Select "TLS".
Note, the target group name of listner 80. Set target group of 443 to target group of 80. Basically, we want TLS termination at the LB and it must forward HTTP traffic (not HTTPS) to port 80 of ingress controller. So
Input of LB: HTTPS
Output of LB: HTTP --> port 80 of ingress nginx controller
Enable "Proxy Protocol v2" in the target group settings
Make sure all subnets are selected in LB -->Description-->Edit subnets.
Check health check of target groups.
Remove listner 80 from LB as we will receive traffic only on 443.
Storage class setup:
Default storage class on EKS is gp2
. GP2
by default is in Delete
mode which means if PVC is deleted, the underlying storage PV is also deleted.
To enable volume expansion for the existing gp2
storage class, modify the YAML configuration by adding allowVolumeExpansion: true
to the gp2
storage class configuration.
kubectl edit sc gp2
: to edit the yaml configuration.
Create storage class gp2-retain
by running sc.yaml
for PV in Retain mode. Set the storage class as gp2-retain in case you want to retain PV.
Create the following domain names:
Rancher: rancher.xyz.net
Keycloak: keycloak.xyz.net
Rancher UI : Rancher provides full CRUD capability of creating and managing kubernetes cluster.
Install rancher using Helm, update hostname
in rancher-values.yaml
and run the following command to install.
Login:
Open Rancher page https://rancher.org.net
.
Get Bootstrap password using
Assign a password. IMPORTANT: makes sure this password is securely saved and retrievable by Admin.
keycloak_client.json
: Used to create SAML client on Keycloak for Rancher integration.
Keycloak - Rancher Integration
Login as admin
user in Keycloak and make sure an email id, and first name field is populated for admin
user. This is important for Rancher authentication as given below.
In Keycloak add another Mapper for the rancher client (in Master realm) with following fields:
Protocol: saml
Name: username
Mapper Type: User Property
Property: username
Friendly Name: username
SAML Attribute Name: username
SAML Attribute NameFormat: Basic
Specify the following mappings in Rancher's Authentication Keycloak form:
Display Name Field: givenName
User Name Field: email
UID Field: username
Groups Field: member
RBAC :
For users in Keycloak assign roles in Rancher - cluster and project roles. Under default
project add all the namespaces. Then, to a non-admin user you may provide Read-Only role (under projects).
Add a member to cluster/project in Rancher:
Give member name exactly as username
in Keycloak
Assign appropriate role like Cluster Owner, Cluster Viewer etc.
You may create new role with fine grained acccess control.
Certificates expiry
In case you see certificate expiry message while adding users, on local cluster run these commands:
Setup mosip cluster
cd $K8_ROOT/mosip/aws
Copy cluster.config.sample
to mosip.cluster.config
.
Review and update the below mentioned parameters of cluster.config.sample
carefully.
name
region
version: “1.24“
instance related details
instanceName
instanceType
desiredcapacity
volumeSize
volumeType
publicKeyName.
update the details of the subnets to be used from vpc
Install
eksctl create cluster -f mosip.cluster.config
Wait for the cluster creation to complete, generally it takes around 30 minutes to create or update cluster.
Once EKS K8 cluster is ready below mentioned output will be displayed in the console screen.
EKS cluster "my-cluster" in "region-code" region is ready
The config file for the new cluster will be created on ~/.kube/config
Make sure to backup and store the ~/.kube/config
with new name. e.g. ~/.kube/mosip-cluster.config
.
Change file permission using below command:
chmod 400 ~/.kube/mosip-cluster.config
Set the KUBECONFIG
properly so that you can access the cluster.
export KUBECONFIG=~/.kube/mosip-cluster.config
Test cluster access:
kubect get nodes
Command will result in details of the nodes of the MOSIP cluster.
Login as admin in Rancher console
Select Import Existing
for cluster addition.
Select the Generic
as cluster type to add.
Fill the Cluster Name
field with unique cluster name and select Create
.
You will get the kubectl commands to be executed in the kubernetes cluster. Copy the command and execute from your PC. (make sure your kube-config file is correctly set to Mosip cluster)
Wait for few seconds after executing the command for the cluster to get verified.
Your cluster is now added to the rancher management server.
Global configmap: Global configmap contains list of necesary details to be used throughout the namespaces of the cluster for common details.
cd $K8_ROOT/mosip
Copy global_configmap.yaml.sample
to global_configmap.yaml.
Update the domain names in global_configmap.yaml
and run.
kubectl apply -f global_configmap.yaml
Storage class setup:
Default storage class on EKS is gp2
. GP2
by default is in Delete
mode which means if PVC is deleted, the underlying storage PV is also deleted.
To enable volume expansion for the existing gp2
storage class, modify the YAML configuration by adding allowVolumeExpansion: true
to the gp2
storage class configuration.
kubectl edit sc gp2
: to edit the yaml configuration.
Create storage class gp2-retain
by running sc.yaml
for PV in Retain mode. Set the storage class as gp2-retain in case you want to retain PV.
Ingress and load balancer (LB) :
Install ingresses as given here:
Load Balancers setup for istio-ingress.
These may be also seen with
You may view them on AWS console in Loadbalancer section.
TLS termination is supposed to be on LB. So all our traffic coming to ingress controller shall be HTTP.
Add the certificates and 443 access to the LB listener.
Update listener TCP->443 to TLS->443 and point to the certificate of domain name that belongs to your cluster.
Forward TLS->443 listner traffic to target group that corresponds to listener on port 80 of respective Loadbalancers. This is because after TLS termination the protocol is HTTP so we must point LB to HTTP port of ingress controller.
Update health check ports of LB target groups to node port corresponding to port 15021. You can see the node ports with
Enable Proxy Protocol v2 on target groups.
Make sure all subnets are included in Availability Zones for the LB. Description --> Availability Zones --> Edit Subnets
Make sure to delete the listeners for port 80 and 15021 from each of the loadbalancers as we restrict unsecured port 80 access over http.
DNS mapping:
Initially all the services will be accesible only over the internal channel.
Point all your domain names to internal LoadBalancers DNS/IP intially till testing is done.
On AWS this may be done on Route 53 console.
Check Overall if nginx and istio wiring is set correctly
Install httpbin: This utility docker returns http headers received inside the cluster. You may use it for general debugging - to check ingress, headers etc.
To see what's reaching httpbin (example, replace with your domain name):
Prometheus and Grafana and Alertmanager tools are used for cluster monitoring.
Select 'Monitoring' App from Rancher console -> Apps & Marketplaces.
In Helm options, open the YAML file and disable Nginx Ingress.
Click on Install
.
Alerting is part of cluster monitoring, where alert notifications are sent to the configured email or slack channel.
Monitoring should be deployed which includes deployment of prometheus, grafana and alertmanager.
After setting slack incoming webhook update slack_api_url
and slack_channel_name
in alertmanager.yml
.
cd $K8_ROOT/monitoring/alerting/
nano alertmanager.yml
Update
Update Cluster_name
in patch-cluster-name.yaml
.
cd $K8_ROOT/monitoring/alerting/
nano patch-cluster-name.yaml
Update
Install Default alerts along some of the defined custom alerts:
Alerting is Installed.
Mosip uses Rancher Fluentd and elasticsearch to collect logs from all services and reflect the same in Kibana Dashboard.
Install Rancher FluentD system : for screpping logs outs of all the microservices from Mosip k8 cluster.
Install Logging from Apps and marketplace within the Rancher UI.
Select Chart Version 100.1.3+up3.17.7
from Rancher console -> Apps & Marketplaces.
Configure Rancher FluentD
Create clusteroutput
:
kubectl apply -f clusteroutput-elasticsearch.yaml
start clusterFlow
kubectl apply -f clusterflow-elasticsearch.yaml
Install elasticsearch, kibana and Istio addons
cd $K8_ROOT/logging
./intall.sh
set min_age
in elasticsearch-ilm-script.sh
and execute the same. min_age
: is the min no. of days for which indices will be stored in elasticsearch.
cd $K8_ROOT/logging
./elasticsearch-ilm-script.sh
Mosip provides set of Kibana Dashboards for checking logs and throughputs .
Brief description of these dashboards are as follows:
Import dashboards:
cd K8_ROOT/logging/dashboard
./load_kibana_dashboards.sh ./dashboards <cluster-kube-config-file>
View dashbords
Open kibana dashboard from: https://kibana.sandbox.xyz.net
.
Kibana --> Menu (on top left) --> Dashboard --> Select the dashboard.
External Dependencies: are set of external requirements needed for funtioning of MOSIP’s core services like DB, object store, hsm etc.
Now that all the Kubernetes cluster and external dependencies are already installed, you can continue with MOSIP service deployment.
MOSIP’s successfull deployment can be verified by comparing the results of api testrig with testrig benchmark.
When prompted input the hour of the day to execute the api-testrig.
Daily api testrig cron jon will be executed at the very opted hour of the day.
MOSIP modules are deployed in the form of microservices in kubernetes cluster.
is used as a trust network extension to access the admin, control, and observation pane.
It is also used for the on-the-field registrations.
MOSIP uses server for:
SSL termination
Reverse Proxy
CDN/Cache management
Loadbalancing
Kubernetes cluster is administered using the and tools.
In V3, we have two Kubernetes clusters:
Observation cluster - This cluster is a part of the observation plane and it helps in administrative tasks. By design, this is kept independent of the actual cluster as a good security practice and to ensure clear segregation of roles and responsibilities. As a best practice, this cluster or it's services should be internal and should never be exposed to the external world.
is used for managing the MOSIP cluster.
in this cluster is used for cluster user access management.
It is recommended to configure log monitoring and network monitoring in this cluster.
In case you have an internal container registry, then it should run here.
MOSIP cluster - This cluster runs all the MOSIP components and certain third party components to secure the cluster, API’s and data.
VM’s required can be with any OS as per convenience.
Here, we are referring to Ubuntu OS throughout this installation guide.
All the VM's should be able to communicate with each other.
Need stable Intra network connectivity between these VM's.
All the VM's should have stable internet connectivity for docker image download (in case of local setup ensure to have a locally accessible docker registry).
Server Interface requirement as mentioned in below table:
As only secured https connections are allowed via nginx server will need below mentioned valid ssl certificates:
One valid wildcard ssl certificate related to domain used for accessing Observation cluster, this needs to be stored inside the nginx server VM for Observation cluster. In above e.g.: *.org.net is the similar example domain.
One valid wildcard ssl certificate related to domain used for accesing Mosip cluster, this needs to be stored inside the nginx server VM for mosip cluster. In above e.g.: *.sandbox.xyz.net is the similar example domain.
Tools to be installed in Personel Computers for complete deployment
[Ansible](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html: version > 2.12.4
Create a directory as MOSIP in your PC and:
clone k8’s infra repo with tag : 1.2.0.1-B2 (whichever is the latest version) inside mosip directory.
git clone https://github.com/mosip/k8s-infra -b v1.2.0.1-B2
clone mosip-infra with tag : 1.2.0.1-B2 (whichever is the latest version) inside mosip directory.
git clone https://github.com/mosip/mosip-infra -b v1.2.0.1-B2
Set below mentioned variables in bashrc
source .bashrc
Note: Above mentioned environment variables will be used throughout the installation to move between one directory to other to run install scripts.
A Wireguard bastion host (Wireguard server) provides secure private channel to access MOSIP cluster. The host restricts public access, and enables access to only those clients who have their public key listed in Wireguard server. Wireguard listens on UDP port51820.
Create a Wireguard server VM with above mentioned Hardware and Network requirements.
Open ports and Install docker on Wireguard VM.
cd $K8_ROOT/wireguard/
create copy of hosts.ini.sample
as hosts.ini
and update the required details for wireguard VM\
cp hosts.ini.sample hosts.ini
execute ports.yml to enable ports on VM level using ufw:
ansible-playbook -i hosts.ini ports.yaml
Note:
Permission of the pem files to access nodes should have 400 permission.
sudo chmod 400 ~/.ssh/privkey.pem
These ports are only needed to be opened for sharing packets over UDP.
Take necessary measure on firewall level so that the Wireguard server can be reachable on 51820/udp.
Setup Wireguard server
SSH to wireguard VM
Create directory for storing wireguard config files.
mkdir -p wireguard/config
Install and start wireguard server using docker as given below:
Note:
Increase the no. of peers above in case more than 30 wireguard client confs (-e PEERS=30) are needed.
Change the directory to be mounted to wireguard docker as per need. All your wireguard confs will be generated in the mounted directory (
-v /home/ubuntu/wireguard/config:/config
).
Install Wireguard client in your PC.
Assign wireguard.conf
:
SSH to the wireguard server VM.
cd /home/ubuntu/wireguard/config
assign one of the PR for yourself and use the same from the PC to connect to the server.
create assigned.txt
file to assign the keep track of peer files allocated and update every time some peer is allocated to someone.
use ls
cmd to see the list of peers.
get inside your selected peer directory, and add mentioned changes in peer.conf
:
cd peer1
nano peer1.conf
Delete the DNS IP.
Update the allowed IP's to subnets CIDR ip . e.g. 10.10.20.0/23
Share the updated peer.conf
with respective peer to connect to wireguard server from Personel PC.
add peer.conf
in your PC’s /etc/wireguard
directory as wg0.conf
.
start the wireguard client and check the status:
Once connected to wireguard, you should be now able to login using private IP’s.
Observation K8s Cluster setup
Install all the required tools mentioned in pre-requisites for PC.
kubectl
helm
ansible
rke (version 1.3.10)
Setup Observation Cluster node VM’s as per the hardware and network requirements as mentioned above.
Setup passwordless SSH into the cluster nodes via pem keys. (Ignore if VM’s are accessible via pem’s).
Generate keys on your PC ssh-keygen -t rsa
Copy the keys to remote observation node VM’s ssh-copy-id <remote-user>@<remote-ip>
SSH into the node to check password-less SSH ssh -i ~/.ssh/<your private key> <remote-user>@<remote-ip>
Note:
Make sure the permission for
privkey.pem
for ssh is set to 400.
Run env-check.yaml
to check if cluster nodes are fine and do not have known issues in it.
cd $K8_ROOT/rancher/on-prem
create copy of hosts.ini.sample
as hosts.ini
and update the required details for Observation k8 cluster nodes.
cp hosts.ini.sample hosts.ini
ansible-playbook -i hosts.ini env-check.yaml
This ansible checks if localhost mapping is already present in /etc/hosts file in all cluster nodes, if not it adds the same.
Open ports and install docker on Observation K8 Cluster node VM’s.
cd $K8_ROOT/rancher/on-prem
Ensure that hosts.ini
is updated with nodal details.
Update vpc_ip variable in ports.yaml
with vpc CIDR ip to allow access only from machines inside same vpc.
Execute ports.yml
to enable ports on VM level using ufw:
ansible-playbook -i hosts.ini ports.yaml
Disable swap in cluster nodes. (Ignore if swap is already disabled)
ansible-playbook -i hosts.ini swap.yaml
execute docker.yml
to install docker and add user to docker group:
ansible-playbook -i hosts.ini docker.yaml
Creating RKE Cluster Configuration file
rke config
Command will prompt for nodal details related to cluster, provide inputs w.r.t below mentioned points:
SSH Private Key Path
:
Number of Hosts
:
SSH Address of host
:
SSH User of host
:
Make all the nodes Worker host
by default.
To create an HA cluster, specify more than one host with role Control Plane
and etcd host
.
Network Plugin Type
: Continue with canal as default network plugin.
For rest of other configurations, opt the required or default value.
As result of rke config
command cluster.yml
file will be generated inside same directory, update the below mentioned fields:
nano cluster.yml
Remove the default Ingress install
Add the name of the kubernetes cluster
cluster_name: sandbox-name
Setup up the cluster:
Once cluster.yml
is ready, you can bring up the kubernetes cluster using simple command.
This command assumes the cluster.yml
file is in the same directory as where you are running the command.
rke up
As part of the Kubernetes creation process, a kubeconfig
file has been created and written at kube_config_cluster.yml
, which can be used to start interacting with your Kubernetes cluster.
Copy the kubeconfig files
To access the cluster using kubeconfig
file use any one of the below method:
cp $HOME/.kube/<cluster_name>_config $HOME/.kube/config
Alternatively
export KUBECONFIG="$HOME/.kube/<cluster_name>_config
Test cluster access:
kubect get nodes
Command will result in details of the nodes of the Observation cluster.
Save your files
Save a copy of the following files in a secure location, they are needed to maintain, troubleshoot and upgrade your cluster.
cluster.yml
: The RKE cluster configuration file.
In case not having Public DNS system add the custom DNS configuration for the cluster.
Check whether coredns pods are up and running in your cluster via the below command:
Update the IP address and domain name in the below DNS hosts template and add it in the coredns configmap Corefile key in the kube-system namespace.
Update coredns configmap via below command.
Check whether the DNS changes are correctly updated in coredns configmap.
Restart the coredns
pod in the kube-system
namespace.
Check status of coredns restart.
Once the rancher cluster is ready, we need ingress and storage class to be set for other applications to be installed.
this will install ingress in ingress-nginx namespace of rancher cluster.
Pre-requisites:
Install Longhorn via helm
./install.sh
Note: Values of below mentioned parameters are set as by default Longhorn installation script:
PV replica count is set to 1. Set the replicas for the storage class appropriately.
Total available node CPU allocated to each instance-manager
pod in the longhorn-system
namespace.
The value "5" means 5% of the total available node CPU.
This value should be fine for sandbox and pilot but you may have to increase the default to "12" for production.
The value can be updated on Longhorn UI after installation.
Access the Longhorn dashboard from Rancher UI once installed.
For Nginx server setup we need ssl certificate, add the same into Nginx server.
SSL certificates can be generated in multiple ways. Either via lets encrypt if you have public DNS or via openssl certs when you don't have Public DNS.
Letsencrypt: Generate wildcard ssl certificate having 3 months validity when you have public DNS system using below steps.
SSH into the nginx server node.
Install Pre-requisites
Generate wildcard SSL certificates for your domain name.
sudo certbot certonly --agree-tos --manual --preferred-challenges=dns -d *.org.net
replace org.net
with your domain.
The default challenge HTTP is changed to DNS challenge, as we require wildcard certificates.
Create a DNS record in your DNS service of type TXT with host _acme-challenge.org.net
, with the string prompted by the script.
Wait for a few minutes for the above entry to get into effect. Verify: host -t TXT _acme-challenge.org.net
Press enter in the certbot
prompt to proceed.
Certificates are created in /etc/letsencrypt
on your machine.
Certificates created are valid for 3 months only.
Openssl : Generate wildcard ssl certificate using openssl in case you don't have public DNS using below steps. (Ensure to use this only in development env, not suggested for Production env).
Install docker on nginx node.
Generate a self-signed certificate for your domain, such as *.sandbox.xyz.net.
Execute the following command to generate a self-signed SSL certificate. Prior to execution, kindly ensure to update environmental variables & rancher domain passed to openssl command:
Above command will generate certs in below specified location. Use it when prompted during nginx installation.
fullChain path: /etc/ssl/certs/tls.crt
.
privKey path: /etc/ssl/private/tls.key
.
Install nginx:
Login to nginx server node.
Provide below mentioned inputs as and when promted
Rancher nginx ip : internal ip of the nginx server VM.
SSL cert path : path of the ssl certificate to be used for ssl termination.
SSL key path : path of the ssl key to be used for ssl termination.
Cluster node ip's : ip’s of the rancher cluster node
Restart nginx service.
Post installation check:
sudo systemctl status nginx
Steps to Uninstall nginx (in case required) sudo apt purge nginx nginx-common
DNS mapping:
Once nginx server is installed successfully, create DNS mapping for rancher cluster related domains as mentioned in DNS requirement section. (rancher.org.net, keycloak.org.net)
In case used Openssl for wildcard ssl certificate add DNS entries in local hosts file of your system.
For example: /etc/hosts files for Linux machines.
Rancher UI: Rancher provides full CRUD capability of creating and managing kubernetes cluster.
Install rancher using Helm, update hostname
, & add privateCA
to true
in rancher-values.yaml
, and run the following command to install.
Login:
Get Bootstrap password using
Assign a password. IMPORTANT: makes sure this password is securely saved and retrievable by Admin.
keycloak_client.json
: Used to create SAML client on Keycloak for Rancher integration.
Login as admin
user in Keycloak and make sure an email id, and first name field is populated for admin user. This is important for Rancher authentication as given below.
In Keycloak add another Mapper for the rancher client (in Master realm) with following fields:
Protocol: saml
Name: username
Mapper Type: User Property
Property: username
Friendly Name: username
SAML Attribute Name: username
SAML Attribute NameFormat: Basic
Specify the following mappings in Rancher's Authentication Keycloak form:
Display Name Field: givenName
User Name Field: email
UID Field: username
Entity ID Field: https://your-rancher-domain/v1-saml/keycloak/saml/metadata
Rancher API Host: https://your-rancher-domain
Groups Field: member
RBAC :
For users in Keycloak assign roles in Rancher - cluster and project roles. Under default
project add all the namespaces. Then, to a non-admin user you may provide Read-Only role (under projects).
Add a member to cluster/project in Rancher:
Give member name exactly as username
in Keycloak
Assign appropriate role like Cluster Owner, Cluster Viewer etc.
You may create new role with fine grained access control.
Certificates expiry
In case you see certificate expiry message while adding users, on local cluster run these commands:
Pre-requisites:
Install all the required tools mentioned in Pre-requisites for PC.
kubectl
helm
ansible
rke (version 1.3.10)
Setup MOSIP K8 Cluster node VM’s as per the hardware and network requirements as mentioned above.
Run env-check.yaml
to check if cluster nodes are fine and don't have known issues in it.
cd $K8_ROOT/rancher/on-prem
create copy of hosts.ini.sample
as hosts.ini
and update the required details for MOSIP k8 cluster nodes.
cp hosts.ini.sample hosts.ini
ansible-playbook -i hosts.ini env-check.yaml
This ansible checks if localhost mapping is already present in /etc/hosts
file in all cluster nodes, if not it adds the same.
Setup passwordless ssh into the cluster nodes via pem keys. (Ignore if VM’s are accessible via pem’s).
Generate keys on your PC
ssh-keygen -t rsa
Copy the keys to remote rancher node VM’s:
ssh-copy-id <remote-user>@<remote-ip>
SSH into the node to check password-less SSH
ssh -i ~/.ssh/<your private key> <remote-user>@<remote-ip>
Rancher UI : (deployed in Rancher K8 cluster)
Open ports and Install docker on MOSIP K8 Cluster node VM’s.
cd $K8_ROOT/mosip/on-prem
create copy of hosts.ini.sample
as hosts.ini
and update the required details for wireguard VM.
cp hosts.ini.sample hosts.ini
Update vpc_ip
variable in ports.yaml
with vpc CIDR ip
to allow access only from machines inside same vpc.
execute ports.yml
to enable ports on VM level using ufw:
ansible-playbook -i hosts.ini ports.yaml
Disable swap in cluster nodes. (Ignore if swap is already disabled)
ansible-playbook -i hosts.ini swap.yaml
execute docker.yml
to install docker and add user to docker group:
ansible-playbook -i hosts.ini docker.yaml
Creating RKE Cluster Configuration file
rke config
Command will prompt for nodal details related to cluster, provide inputs w.r.t below mentioned points:
SSH Private Key Path
:
Number of Hosts
:
SSH Address of host
:
SSH User of host
:
Make all the nodes Worker host
by default.
To create an HA cluster, specify more than one host with role Control Plane
and etcd host
.
Network Plugin Type
: Continue with canal as default network plugin.
For rest for other configuration opt the required or default value.
As result of rke config command cluster.ymlfile
will be generated inside same directory, update the below mentioned fields:
nano cluster.yml
Remove the default Ingress install
Add the name of the kubernetes cluster
Setup up the cluster:
Once cluster.yml
is ready, you can bring up the kubernetes cluster using simple command.
This command assumes the cluster.yml
file is in the same directory as where you are running the command.
rke up
The last line should read Finished building Kubernetes cluster successfully
to indicate that your cluster is ready to use.
Copy the kubeconfig files
To access the cluster using kubeconfig filr use any one of the below method:
cp $HOME/.kube/<cluster_name>_config $HOME/.kube/config
Alternatively
Test cluster access:
kubect get nodes
Command will result in details of the nodes of the rancher cluster.
Save Your files
Save a copy of the following files in a secure location, they are needed to maintain, troubleshoot and upgrade your cluster.:
cluster.yml
: The RKE cluster configuration file.
In case not having Public DNS system add the custom DNS configuration for the cluster.
Check whether coredns pods are up and running in your cluster via the below command:
Update the IP address and domain name in the below DNS hosts template and add it in the coredns configmap Corefile key in the kube-system namespace.
Update coredns configmap via below command.
Check whether the DNS changes are correctly updated in coredns configmap.
Restart the coredns
pod in the kube-system
namespace.
Check status of coredns restart.
Global configmap: Global configmap contains the list of neccesary details to be used throughout the namespaces of the cluster for common details.
cd $K8_ROOT/mosip
Copy global_configmap.yaml.sample
to global_configmap.yaml
.
Update the domain names in global_configmap.yaml
and run.
kubectl apply -f global_configmap.yaml
cd $K8_ROOT/mosip/on-prem/istio
./install.sh
This will bring up all the Istio components and the Ingress Gateways.
Check Ingress Gateway services:
kubectl get svc -n istio-system
istio-ingressgateway
: external facing istio service.
istio-ingressgateway-internal
: internal facing istio service.
istiod
: Istio daemon for replicating the changes to all envoy filters.
Storage class setup: Longhorn creates a storage class in the cluster for creating pv (persistence volume) and pvc (persistence volume claim).
Pre-requisites:
Install Longhorn via helm
./install.sh
Note: Values of below mentioned parameters are set as by default Longhorn installation script:
PV replica count is set to 1. Set the replicas for the storage class appropriately.
Total available node CPU allocated to each instance-manager
pod in the longhorn-system
namespace.
The value "5" means 5% of the total available node CPU
This value should be fine for sandbox and pilot but you may have to increase the default to "12" for production.
The value can be updated on Longhorn UI after installation.
Login as admin in Rancher console
Select Import
Existing for cluster addition.
Select Generic
as cluster type to add.
Fill the Cluster Name
field with unique cluster name and select Create
.
You will get the kubecl commands to be executed in the kubernetes cluster. Copy the command and execute from your PC (make sure your kube-config
file is correctly set to MOSIP cluster).
Wait for few seconds after executing the command for the cluster to get verified.
Your cluster is now added to the rancher management server.
For Nginx server setup, we need ssl certificate, add the same into Nginx server.
SSL certificates can be generated in multiple ways. Either via lets encrypt if you have public DNS or via openssl certs when you don't have Public DNS.
Letsencrypt: Generate wildcard ssl certificate having 3 months validity when you have public DNS system using below steps.
SSH into the nginx server node.
Install Pre-requisites
Generate wildcard SSL certificates for your domain name.
sudo certbot certonly --agree-tos --manual --preferred-challenges=dns -d *.org.net
replace org.net
with your domain.
The default challenge HTTP is changed to DNS challenge, as we require wildcard certificates.
Create a DNS record in your DNS service of type TXT with host _acme-challenge.org.net
, with the string prompted by the script.
Wait for a few minutes for the above entry to get into effect. Verify: host -t TXT _acme-challenge.org.net
Press enter in the certbot
prompt to proceed.
Certificates are created in /etc/letsencrypt
on your machine.
Certificates created are valid for 3 months only.
Openssl : Generate wildcard ssl certificate using openssl in case you don't have public DNS using below steps. (Ensure to use this only in development env, not suggested for Production env).
Install docker on nginx node.
Generate a self-signed certificate for your domain, such as *.sandbox.xyz.net.
Execute the following command to generate a self-signed SSL certificate. Prior to execution, kindly ensure that the environmental variables passed to the OpenSSL Docker container have been properly updated:
Above command will generate certs in below specified location. Use it when prompted during nginx installation.
fullChain path: /etc/ssl/certs/nginx-selfsigned.crt.
privKey path: /etc/ssl/private/nginx-selfsigned.key.
Install nginx:
Login to nginx server node.
Provide below mentioned inputs as and when prompted
MOSIP nginx server internal ip
MOSIP nginx server public ip
Publically accessible domains (comma separated with no whitespaces)
SSL cert path
SSL key path
Cluster node ip's (comma separated no whitespace)
When utilizing an openssl wildcard SSL certificate, please add the following server block to the nginx server configuration within the http block. Disregard this if using SSL certificates obtained through letsencrypt or for publicly available domains. Please note that this should only be used in a development environment and is not recommended for production environments.
nano /etc/nginx/nginx.conf
Note: HTTP access is enabled for IAM because MOSIP's keymanager expects to have valid SSL certificates. Ensure to use this only for development purposes, and it is not recommended to use it in production environments.
Restart nginx service.
Post installation check:
sudo systemctl status nginx
Steps to Uninstall nginx (in case required) sudo apt purge nginx nginx-common
DNS mapping:
Once nginx server is installed successfully, create DNS mapping for rancher cluster related domains as mentioned in DNS requirement section. (rancher.org.net, keycloak.org.net)
In case used Openssl for wildcard ssl certificate add DNS entries in local hosts file of your system.
For example: /etc/hosts files for Linux machines.
Check Overall if nginx and istio wiring is set correctly
Install httpbin
: This utility docker returns http headers received inside the cluster. You may use it for general debugging - to check ingress, headers etc.
To see what is reaching the httpbin (example, replace with your domain name):
Prometheus and Grafana and Alertmanager tools are used for cluster monitoring.
Select 'Monitoring' App from Rancher console -> Apps & Marketplaces
.
In Helm options, open the YAML file and disable Nginx Ingress.
Click on Install
.
Alerting is part of cluster monitoring, where alert notifications are sent to the configured email or slack channel.
Monitoring should be deployed which includes deployment of prometheus, grafana and alertmanager.
After setting slack incoming webhook update slack_api_url
and slack_channel_name
in alertmanager.yml
.
cd $K8_ROOT/monitoring/alerting/
nano alertmanager.yml
Update:
Update Cluster_name
in patch-cluster-name.yaml
.
cd $K8_ROOT/monitoring/alerting/
nano patch-cluster-name.yaml
Update:
Install Default alerts along some of the defined custom alerts:
Alerting is installed.
Install Rancher FluentD system : for scraping logs outs of all the microservices from MOSIP k8 cluster.
Install Logging from Apps and marketplace within the Rancher UI.
Select Chart Version 100.1.3+up3.17.7
from Rancher console -> Apps & Marketplaces.
Configure Rancher FluentD
Create clusteroutput
kubectl apply -f clusteroutput-elasticsearch.yaml
Start clusterFlow
kubectl apply -f clusterflow-elasticsearch.yaml
Install elasticsearch, kibana and Istio addons\
set min_age
in elasticsearch-ilm-script.sh
and execute the same.
min_age
: is the minimum no. of days for which indices will be stored in elasticsearch.
MOSIP provides set of Kibana Dashboards for checking logs and throughputs.
Brief description of these dashboards are as follows:
Import dashboards:
cd K8_ROOT/logging
./load_kibana_dashboards.sh ./dashboards <cluster-kube-config-file>
View dashboards
Open kibana dashboard from https://kibana.sandbox.xyz.net
.
Kibana --> Menu (on top left) --> Dashboard --> Select the dashboard.
External Dependencies are set of external requirements that are needed for functioning of MOSIP’s core services like DB, Object Store, HSM etc.
Add/ Update the below property in application-default.properties and comment on the below property in the *-default.properties file in the config repo.
Add/ Update the below property in the esignet-default.properties file in the config repo.
Now that all the Kubernetes cluster and external dependencies are already installed, will continue with MOSIP service deployment.
While installing a few modules, installation script prompts to check if you have public domain and valid SSL certificates on the server. Opt option n as we are using self-signed certificates. For example:
Start installing mosip modules:
MOSIP modules are deployed in the form of microservices in kubernetes cluster.
is used as a trust network extension to access the admin, control, and observation pane.
It is also used for the on-the-field registrations.
MOSIP uses server for:
SSL termination
Reverse Proxy
CDN/Cache management
Loadbalancing
Kubernetes cluster is administered using the and tools.
In V3, we have two Kubernetes clusters:
Observation cluster - This cluster is a part of the observation plane and it helps in administrative tasks. By design, this is kept independent of the actual cluster as a good security practice and to ensure clear segregation of roles and responsibilities. As a best practice, this cluster or it's services should be internal and should never be exposed to the external world.
is used for managing the MOSIP cluster.
in this cluster is used for cluster user access management.
It is recommended to configure log monitoring and network monitoring in this cluster.
In case you have an internal container registry, then it should run here.
MOSIP cluster - This cluster runs all the MOSIP components and certain third party components to secure the cluster, API’s and data.
VM’s required can be with any OS as per convenience.
Here, we are referting to Ubuntu OS throughout this installation guide.
All the VM's should be able to communicate with each other.
Need stable Intra network connectivity between these VM's.
All the VM's should have stable internet connectivity for docker image download (in case of local setup ensure to have a locally accesible docker registry).
Server Interface requirement as mentioned in below table:
As only secured https connections are allowed via nginx server will need below mentioned valid ssl certificates:
One valid wildcard ssl certificate related to domain used for accessing Observation cluster, this needs to be stored inside the nginx server VM for Observation cluster. In above e.g.: *.org.net is the similiar example domain.
One valid wildcard ssl certificate related to domain used for accesing Mosip cluster, this needs to be stored inside the nginx server VM for mosip cluster. In above e.g.: *.sandbox.xyz.net is the similiar example domain.
Tools to be installed in Personel Computers for complete deployment
[Ansible](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html: version > 2.12.4
Create a directory as mosip in your PC and:
clone k8’s infra repo with tag : 1.2.0.1-B2 (whichever is the latest version) inside mosip directory.
git clone https://github.com/mosip/k8s-infra -b v1.2.0.1-B2
clone mosip-infra with tag : 1.2.0.1-B2 (whichever is the latest version) inside mosip directory.
git clone https://github.com/mosip/mosip-infra -b v1.2.0.1-B2
Set below mentioned variables in bashrc
source .bashrc
Note: Above mentioned environment variables will be used throughout the installation to move between one directory to other to run install scripts.
A Wireguard bastion host (Wireguard server) provides secure private channel to access MOSIP cluster. The host restricts public access, and enables access to only those clients who have their public key listed in Wireguard server. Wireguard listens on UDP port51820.
Create a Wireguard server VM with above mentioned Hardware and Network requirements.
Open ports and Install docker on Wireguard VM.
cd $K8_ROOT/wireguard/
create copy of hosts.ini.sample
as hosts.ini
and update the required details for wireguard VM\
cp hosts.ini.sample hosts.ini
execute ports.yml to enable ports on VM level using ufw:
ansible-playbook -i hosts.ini ports.yaml
Note:
Permission of the pem files to access nodes should have 400 permission.
sudo chmod 400 ~/.ssh/privkey.pem
These ports are only needed to be opened for sharing packets over UDP.
Take necessary measure on firewall level so that the Wireguard server can be reachable on 51820/udp.
Setup Wireguard server
SSH to wireguard VM
Create directory for storing wireguard config files.
mkdir -p wireguard/config
Install and start wireguard server using docker as given below:
Note:
Increase the no. of peers above in case more than 30 wireguard client confs (-e PEERS=30) are needed.
Change the directory to be mounted to wireguard docker as per need. All your wireguard confs will be generated in the mounted directory (
-v /home/ubuntu/wireguard/config:/config
).
Install Wireguard client in your PC.
Assign wireguard.conf
:
SSH to the wireguard server VM.
cd /home/ubuntu/wireguard/config
assign one of the PR for yourself and use the same from the PC to connect to the server.
create assigned.txt
file to assign the keep track of peer files allocated and update everytime some peer is allocated to someone.
use ls
cmd to see the list of peers.
get inside your selected peer directory, and add mentioned changes in peer.conf
:
cd peer1
nano peer1.conf
Delete the DNS IP.
Update the allowed IP's to subnets CIDR ip . e.g. 10.10.20.0/23
Share the updated peer.conf
with respective peer to connect to wireguard server from Personel PC.
add peer.conf
in your PC’s /etc/wireguard
directory as wg0.conf
.
start the wireguard client and check the status:
Once connected to wireguard, you should be now able to login using private IP’s.
Observation K8s Cluster setup
Install all the required tools mentioned in pre-requisites for PC.
kubectl
helm
ansible
rke (version 1.3.10)
Setup Observation Cluster node VM’s as per the hardware and network requirements as mentioned above.
Setup passwordless SSH into the cluster nodes via pem keys. (Ignore if VM’s are accessible via pem’s).
Generate keys on your PC ssh-keygen -t rsa
Copy the keys to remote observation node VM’s ssh-copy-id <remote-user>@<remote-ip>
SSH into the node to check password-less SSH ssh -i ~/.ssh/<your private key> <remote-user>@<remote-ip>
Note:
Make sure the permission for
privkey.pem
for ssh is set to 400.
Run env-check.yaml
to check if cluster nodes are fine and do not have known issues in it.
cd $K8_ROOT/rancher/on-prem
create copy of hosts.ini.sample
as hosts.ini
and update the required details for Observation k8 cluster nodes.
cp hosts.ini.sample hosts.ini
ansible-playbook -i hosts.ini env-check.yaml
This ansible checks if localhost mapping is already present in /etc/hosts file in all cluster nodes, if not it adds the same.
Open ports and install docker on Observation K8 Cluster node VM’s.
cd $K8_ROOT/rancher/on-prem
Ensure that hosts.ini
is updated with nodal details.
Update vpc_ip variable in ports.yaml
with vpc CIDR ip to allow access only from machines inside same vpc.
Execute ports.yml
to enable ports on VM level using ufw:
ansible-playbook -i hosts.ini ports.yaml
Disable swap in cluster nodes. (Ignore if swap is already disabled)
ansible-playbook -i hosts.ini swap.yaml
execute docker.yml
to install docker and add user to docker group:
ansible-playbook -i hosts.ini docker.yaml
Creating RKE Cluster Configuration file
rke config
Command will prompt for nodal details related to cluster, provide inputs w.r.t below mentioned points:
SSH Private Key Path
:
Number of Hosts
:
SSH Address of host
:
SSH User of host
:
Make all the nodes Worker host
by default.
To create an HA cluster, specify more than one host with role Control Plane
and etcd host
.
Network Plugin Type
: Continue with canal as default network plugin.
For rest of other configurations, opt the required or default value.
As result of rke config
command cluster.yml
file will be generated inside same directory, update the below mentioned fields:
nano cluster.yml
Remove the default Ingress install
Add the name of the kubernetes cluster
cluster_name: sandbox-name
Setup up the cluster:
Once cluster.yml
is ready, you can bring up the kubernetes cluster using simple command.
This command assumes the cluster.yml
file is in the same directory as where you are running the command.
rke up
As part of the Kubernetes creation process, a kubeconfig
file has been created and written at kube_config_cluster.yml
, which can be used to start interacting with your Kubernetes cluster.
Copy the kubeconfig files
To access the cluster using kubeconfig
file use any one of the below method:
cp $HOME/.kube/<cluster_name>_config $HOME/.kube/config
Alternatively
export KUBECONFIG="$HOME/.kube/<cluster_name>_config
Test cluster access:
kubect get nodes
Command will result in details of the nodes of the Observation cluster.
Save your files
Save a copy of the following files in a secure location, they are needed to maintain, troubleshoot and upgrade your cluster.
cluster.yml
: The RKE cluster configuration file.
Once the rancher cluster is ready, we need ingress and storage class to be set for other applications to be installed.
this will install ingress in ingress-nginx namespace of rancher cluster.
The following storage classes can be used:
[ceph-csi](TODO Implementation in progress)
Pre-requisites:
Install Longhorn via helm
./install.sh
Note: Values of below mentioned parametrs are set as by default Longhorn installation script:
PV replica count is set to 1. Set the replicas for the storage class appropriately.
Total available node CPU allocated to each instance-manager
pod in the longhorn-system
namespace.
The value "5" means 5% of the total available node CPU.
This value should be fine for sandbox and pilot but you may have to increase the default to "12" for production.
The value can be updated on Longhorn UI after installation.
Access the Longhorn dashboard from Rancher UI once installed.
For Nginx server setup we need ssl certificate, add the same into Nginx server.
Incase valid ssl certificate is not there generate one using letsencrypt:
SSH into the nginx server
Install Pre-requisites
Generate wildcard SSL certificates for your domain name.
sudo certbot certonly --agree-tos --manual --preferred-challenges=dns -d *.org.net
replace org.net
with your domain.
The default challenge HTTP is changed to DNS challenge, as we require wildcard certificates.
Create a DNS record in your DNS service of type TXT with host _acme-challenge.org.net
, with the string prompted by the script.
Wait for a few minutes for the above entry to get into effect.
Verify:
host -t TXT _acme-challenge.org.net
Press enter in the certbot
prompt to proceed.
Certificates are created in /etc/letsencrypt
on your machine.
Certificates created are valid for 3 months only.
Provide below mentioned inputs as and when promted
Rancher nginx ip : internal ip of the nginx server VM.
SSL cert path : path of the ssl certificate to be used for ssl termination.
SSL key path : path of the ssl key to be used for ssl termination.
Cluster node ip's : ip’s of the rancher cluster node
Post installation check:
sudo systemctl status nginx
Steps to Uninstall nginx (in case required)
sudo apt purge nginx nginx-common
DNS mapping: Once nginx server is installed sucessfully, create DNS mapping for rancher cluster related domains as mentioned in DNS requirement section. (rancher.org.net, keycloak.org.net)
Rancher provides full CRUD capability of creating and managing kubernetes cluster.
Install rancher using Helm, update hostname
in rancher-values.yaml
and run the following command to install.
Login:
Get Bootstrap password using
Assign a password. IMPORTANT: makes sure this password is securely saved and retrievable by Admin.
keycloak_client.json
: Used to create SAML client on Keycloak for Rancher integration.
Login as admin
user in Keycloak and make sure an email id, and first name field is populated for admin user. This is important for Rancher authentication as given below.
In Keycloak add another Mapper for the rancher client (in Master realm) with following fields:
Protocol: saml
Name: username
Mapper Type: User Property
Property: username
Friendly Name: username
SAML Attribute Name: username
SAML Attribute NameFormat: Basic
Specify the following mappings in Rancher's Authentication Keycloak form:
Display Name Field: givenName
User Name Field: email
UID Field: username
Entity ID Field: https://your-rancher-domain/v1-saml/keycloak/saml/metadata
Rancher API Host: https://your-rancher-domain
Groups Field: member
For users in Keycloak assign roles in Rancher - cluster and project roles. Under default
project add all the namespaces. Then, to a non-admin user you may provide Read-Only role (under projects).
Add a member to cluster/project in Rancher:
Navigate to RBAC cluster members
Add member name exactly as username
in Keycloak
Assign appropriate role like Cluster Owner, Cluster Viewer etc.
You may create new role with fine grained acccess control.
Add group to to cluster/project in Rancher:
Navigate to RBAC cluster members
Click on Add
and select a group from the displayed drop-down.
Assign appropriate role like Cluster Owner, Cluster Viewer etc.
To add groups, the user must be a member of the group.
Creating a Keycloak group involves the following steps:
Go to the "Groups" section in Keycloak and create groups with default roles.
Navigate to the "Users" section in Keycloak, select a user, and then go to the "Groups" tab. From the list of groups, add the user to the required group.
Certificates expiry
In case you see certificate expiry message while adding users, on local cluster run these commands:
Pre-requisites:
Install all the required tools mentioned in Pre-requisites for PC.
kubectl
helm
ansible
rke (version 1.3.10)
Setup MOSIP K8 Cluster node VM’s as per the hardware and network requirements as mentioned above.
Run env-check.yaml
to check if cluster nodes are fine and dont have known issues in it.
cd $K8_ROOT/rancher/on-prem
create copy of hosts.ini.sample
as hosts.ini
and update the required details for MOSIP k8 cluster nodes.
cp hosts.ini.sample hosts.ini
ansible-playbook -i hosts.ini env-check.yaml
This ansible checks if localhost mapping ia already present in /etc/hosts
file in all cluster nodes, if not it adds the same.
Setup passwordless ssh into the cluster nodes via pem keys. (Ignore if VM’s are accessible via pem’s).
Generate keys on your PC
ssh-keygen -t rsa
Copy the keys to remote rancher node VM’s:
ssh-copy-id <remote-user>@<remote-ip>
SSH into the node to check password-less SSH
ssh -i ~/.ssh/<your private key> <remote-user>@<remote-ip>
Rancher UI : (deployed in Rancher K8 cluster)
Open ports and Install docker on MOSIP K8 Cluster node VM’s.
cd $K8_ROOT/mosip/on-prem
create copy of hosts.ini.sample
as hosts.ini
and update the required details for wireguard VM.
cp hosts.ini.sample hosts.ini
Update vpc_ip
variable in ports.yaml
with vpc CIDR ip
to allow access only from machines inside same vpc.
execute ports.yml
to enable ports on VM level using ufw:
ansible-playbook -i hosts.ini ports.yaml
Disable swap in cluster nodes. (Ignore if swap is already disabled)
ansible-playbook -i hosts.ini swap.yaml
execute docker.yml
to install docker and add user to docker group:
ansible-playbook -i hosts.ini docker.yaml
Creating RKE Cluster Configuration file
rke config
Command will prompt for nodal details related to cluster, provide inputs w.r.t below mentioned points:
SSH Private Key Path
:
Number of Hosts
:
SSH Address of host
:
SSH User of host
:
Make all the nodes Worker host
by default.
To create an HA cluster, specify more than one host with role Control Plane
and etcd host
.
Network Plugin Type
: Continue with canal as default network plugin.
For rest for other configuration opt the required or default value.
As result of rke config command cluster.ymlfile
will be generated inside same directory, update the below mentioned fields:
nano cluster.yml
Remove the default Ingress install
Add the name of the kubernetes cluster
Setup up the cluster:
Once cluster.yml
is ready, you can bring up the kubernetes cluster using simple command.
This command assumes the cluster.yml
file is in the same directory as where you are running the command.
rke up
The last line should read Finished building Kubernetes cluster successfully
to indicate that your cluster is ready to use.
Copy the kubeconfig files
To access the cluster using kubeconfig filr use any one of the below method:
cp $HOME/.kube/<cluster_name>_config $HOME/.kube/config
Alternatively
Test cluster access:
kubect get nodes
Command will result in details of the nodes of the rancher cluster.
Save Your files
Save a copy of the following files in a secure location, they are needed to maintain, troubleshoot and upgrade your cluster.:
cluster.yml
: The RKE cluster configuration file.
Global configmap: Global configmap contains the list of neccesary details to be used throughout the namespaces of the cluster for common details.
cd $K8_ROOT/mosip
Copy global_configmap.yaml.sample
to global_configmap.yaml
.
Update the domain names in global_configmap.yaml
and run.
kubectl apply -f global_configmap.yaml
cd $K8_ROOT/mosip/on-prem/istio
./install.sh
This will bring up all the Istio components and the Ingress Gateways.
Check Ingress Gateway services:
kubectl get svc -n istio-system
istio-ingressgateway
: external facing istio service.
istio-ingressgateway-internal
: internal facing istio service.
istiod
: Istio daemon for replicating the changes to all envoy filters.
The following storage classes can be used:
[ceph-csi](TODO Implementation in progress)
For time being, we are considering Longhorn as a storage class.
Storage class setup: Longhorn creates a storage class in the cluster for creating pv (persistence volume) and pvc (persistence volume claim).
Pre-requisites:
Install Longhorn via helm
./install.sh
Note: Values of below mentioned parameters are set as by default Longhorn installation script:
PV replica count is set to 1. Set the replicas for the storage class appropriately.
Total available node CPU allocated to each instance-manager
pod in the longhorn-system
namespace.
The value "5" means 5% of the total available node CPU
This value should be fine for sandbox and pilot but you may have to increase the default to "12" for production.
The value can be updated on Longhorn UI after installation.
Login as admin in Rancher console
Select Impor
t Existing for cluster addition.
Select Generic
as cluster type to add.
Fill the Cluster Name
field with unique cluster name and select Create
.
You will get the kubecl commands to be executed in the kubernetes cluster. Copy the command and execute from your PC (make sure your kube-config
file is correctly set to MOSIP cluster).
Wait for few seconds after executing the command for the cluster to get verified.
Your cluster is now added to the rancher management server.
For Nginx server setup, we need ssl certificate, add the same into Nginx server.
Incase valid ssl certificate is not there generate one using letsencrypt:
SSH into the nginx server
Install Pre-requisites:
Generate wildcard SSL certificates for your domain name.
sudo certbot certonly --agree-tos --manual --preferred-challenges=dns -d *.sandbox.mosip.net -d sandbox.mosip.net
replace sanbox.mosip.net
with your domain.
The default challenge HTTP is changed to DNS challenge, as we require wildcard certificates.
Create a DNS record in your DNS service of type TXT with host _acme-challenge.sandbox.xyz.net
, with the string prompted by the script.
Wait for a few minutes for the above entry to get into effect.
** Verify**: host -t TXT _acme-challenge.sandbox.mosip.net
Press enter in the certbot
prompt to proceed.
Certificates are created in /etc/letsencrypt
on your machine.
Certificates created are valid for 3 months only.
Clone k8s-infra
Provide below mentioned inputs as and when prompted
MOSIP nginx server internal ip
MOSIP nginx server public ip
Publically accessible domains (comma seperated with no whitespaces)
SSL cert path
SSL key path
Cluster node ip's (comma seperated no whitespace)
Post installation check\
sudo systemctl status nginx
Steps to uninstall nginx (incase it is required)
sudo apt purge nginx nginx-common
DNS mapping: Once nginx server is installed sucessfully, create DNS mapping for observation cluster related domains as mentioned in DNS requirement section.
Check Overall if nginx and istio wiring is set correctly
Install httpbin
: This utility docker returns http headers received inside the cluster. You may use it for general debugging - to check ingress, headers etc.
To see what is reaching the httpbin (example, replace with your domain name):
Prometheus and Grafana and Alertmanager tools are used for cluster monitoring.
Select 'Monitoring' App from Rancher console -> Apps & Marketplaces
.
In Helm options, open the YAML file and disable Nginx Ingress.
Click on Install
.
Alerting is part of cluster monitoring, where alert notifications are sent to the configured email or slack channel.
Monitoring should be deployed which includes deployment of prometheus, grafana and alertmanager.
After setting slack incoming webhook update slack_api_url
and slack_channel_name
in alertmanager.yml
.
cd $K8_ROOT/monitoring/alerting/
nano alertmanager.yml
Update:
Update Cluster_name
in patch-cluster-name.yaml
.
cd $K8_ROOT/monitoring/alerting/
nano patch-cluster-name.yaml
Update:
Install Default alerts along some of the defined custom alerts:
Alerting is installed.
Install Rancher FluentD system : for screpping logs outs of all the microservices from MOSIP k8 cluster.
Install Logging from Apps and marketplace within the Rancher UI.
Select Chart Version 100.1.3+up3.17.7
from Rancher console -> Apps & Marketplaces.
Configure Rancher FluentD
Create clusteroutput
kubectl apply -f clusteroutput-elasticsearch.yaml
Start clusterFlow
kubectl apply -f clusterflow-elasticsearch.yaml
Install elasticsearch, kibana and Istio addons\
set min_age
in elasticsearch-ilm-script.sh
and execute the same.
min_age
: is the minimum no. of days for which indices will be stored in elasticsearch.
MOSIP provides set of Kibana Dashboards for checking logs and throughputs.
Brief description of these dashboards are as follows:
Import dashboards:
cd K8_ROOT/logging
./load_kibana_dashboards.sh ./dashboards <cluster-kube-config-file>
View dashboards
Open kibana dashboard from https://kibana.sandbox.xyz.net
.
Kibana --> Menu (on top left) --> Dashboard --> Select the dashboard.
External Dependencies are set of external requirements that are needed for functioning of MOSIP’s core services like DB, Object Store, HSM etc.
Now that all the Kubernetes cluster and external dependencies are already installed, will continue with MOSIP service deployment.
: contains scripts to install and configure Kubernetes cluster with required monitoring, logging and alerting tools.
: contains deployment scripts to run charts in defined sequence.
: contains all the configuration files required by the MOSIP modules.
: contains packaged helm charts for all the MOSIP modules.
One valid wildcard ssl certificate related to domain used for accesing Observation cluster which will be created using ACM (Amazon certificate manager). In above e.g. *. is the similiar example domain.
One valid wildcard ssl certificate related to domain used for accessing MOSIP cluster which will be created using ACM (Amazon certificate manager). In above e.g. *. is the similiar example domain.
client version 1.23.6
client version 3.8.2 and add below repos as well :
: version: 1.15.0
: version: 0.121.0
AWS credentials in ~/.aws/
folder as given .
Install docker in the Wireguard machine as given .
Install in your PC.
: used for ingress in rancher cluster.
The above will automatically spawn an .
Obtain AWS TLS certificate as given
we need the EBS driver for our storage class to work, follow the steps to setup EBS driver.
Point the above to internal ip address of the NLB. This assumes that you have a has been installed. On AWS this is done on Route 53 console.
: Keycloak is an OAuth 2.0 compliant Identity Access Management (IAM) system used to manage the access to Rancher for cluster controls.
Enable authentication with Keycloak using the steps given .
Entity ID Field:
Rancher API Host:
If you want to create custom roles, you can follow the steps given .
we need the EBS driver for our storage class to work, follow the steps to setup EBS driver.
also we need EFS CSI driver for the regproc services, because EBS driver only supports RWO but we need RWX, follow these to setup EFS CSI driver.
Ingress is not installed by default on EKS. We use Istio ingress gateway controller to allow traffic in the cluster. Two channels are created - public and internal. See .
Install as given here in your system.
The above istio installation will automatically spawn an .
Obtain AWS TLS certificate as given
After Go live decision enable .
Create .
contains the logstash
Index Pattern required by the rest of the dashboards.
contains a Search dashboard which shows only the error logs of the services, called MOSIP Error Logs
dashboard.
contains a Search dashboard which show all logs of a particular service, called MOSIP Service Logs
dashboard.
contains dashboards which show insights into MOSIP processes, like the number of UINs generated (total and per hr), the number of Biometric deduplications processed, number of packets uploaded etc, called MOSIP Insight
dashboard.
contains dashboards which show how quickly different MOSIP Services are responding to different APIs, over time, called Response Time
dashboard.
Check detailed installation instruction of all the .
Check the detailed MOSIP Modules Deployment steps.
: contains the scripts to install and configure Kubernetes cluster with required monitoring, logging and alerting tools.
: contains the deployment scripts to run charts in defined sequence.
: contains all the configuration files required by the MOSIP modules.
: contains packaged helm charts for all the MOSIP modules.
Sl no. | Purpose | vCPU's | RAM | Storage (HDD) | no. ofVM's | HA |
---|
Sl no. | Purpose | Network Interfaces |
---|
Domain Name | Mapping details | Purpose |
---|
- any client version above 1.19
- any client version above 3.0.0 and add below repos as well:
: version: 1.15.0
: version:
For production deployments edit the cluster.yml
, according to this .
kube_config_cluster.yml
: The for the cluster, this file contains credentials for full access to the cluster.
cluster.rkestate
: The , this file contains credentials for full access to the cluster.
example:
: used for ingress in rancher cluster.
Storage class setup: creates a storage class in the cluster for creating pv (persistence volume) and pvc (persistence volume claim).
Setup Backup : In case you want to backup the pv data from longhorn to s3 periodically follow . (Optional, ignore if not required)
Wildcard SSL certificate . This will increase the validity of the certificate for next 3 months.
Clone
Open page.
: Keycloak is an OAuth 2.0 compliant Identity Access Management (IAM) system used to manage the access to Rancher for cluster controls.
Enable authentication with Keycloak using the steps given .
If you want to create custom roles, you can follow the steps given .
For production deplopyments edit the cluster.yml
, according to this .
kube_config_cluster.yml
: The for the cluster, this file contains credentials for full access to the cluster.
cluster.rkestate
: The , this file contains credentials for full access to the cluster.
example:
Ingress setup: It is a service mesh for the MOSIP K8 cluster which provides transparent layers on top of existing microservices along with powerful features enabling a uniform and more efficient way to secure, connect, and monitor services.
Wildcard SSL certificate . This will increase the validity of the certificate for next 3 months.
Clone
Create .
MOSIP uses and elasticsearch to collect logs from all services and reflect the same in Kibana Dashboard.
contains the logstash Index Pattern required by the rest of the dashboards.
contains a Search dashboard which shows only the error logs of the services, called MOSIP Error Logs
dashboard.
contains a Search dashboard which show all logs of a particular service, called MOSIP Service Logs dashboard.
contains dashboards which show insights into MOSIP processes, like the number of UINs generated (total and per hr), the number of Biometric deduplications processed, number of packets uploaded etc, called MOSIP Insight
dashboard.
contains dashboards which show how quickly different MOSIP Services are responding to different APIs, over time, called Response Time
dashboard.
Click to check the detailed installation instructions of all the external components.
Check detailed installation steps.
: contains the scripts to install and configure Kubernetes cluster with required monitoring, logging and alerting tools.
: contains the deployment scripts to run charts in defined sequence.
: contains all the configuration files required by the MOSIP modules.
: contains packaged helm charts for all the MOSIP modules.
Sl no. | Purpose | vCPU's | RAM | Storage (HDD) | no. ofVM's | HA |
---|
Sl no. | Purpose | Network Interfaces |
---|
Domain Name | Mapping details | Purpose |
---|
- any client version above 1.19
- any client version above 3.0.0 and add below repos as well:
: version: 1.15.0
: version:
For production deplopyments edit the cluster.yml
, according to this .
kube_config_cluster.yml
: The for the cluster, this file contains credentials for full access to the cluster.
cluster.rkestate
: The , this file contains credentials for full access to the cluster.
: used for ingress in rancher cluster.
: If you are already using VMware virtual machines, you can proceed with the vSphere storage class.
.
Storage class setup: creates a storage class in the cluster for creating pv (persistence volume) and pvc (persistence volume claim).
Setup Backup : In case you want to bacup the pv data from longhorn to s3 periodically follow . (Optional, ignore if not required)
Wildcard SSL certificate . This will increase the validity of the certificate for next 3 months.
Clone
Open page.
is an OAuth 2.0 compliant Identity Access Management (IAM) system used to manage the access to Rancher for cluster controls.
Enable authentication with Keycloak using the steps given .
If you want to create custom roles, you can follow the steps given .
For production deplopyments edit the cluster.yml
, according to this .
kube_config_cluster.yml
: The for the cluster, this file contains credentials for full access to the cluster.
cluster.rkestate
: The , this file contains credentials for full access to the cluster.
Ingress setup: It is a service mesh for the MOSIP K8 cluster which provides transparent layers on top of existing microservices along with powerful features enabling a uniform and more efficient way to secure, connect, and monitor services.
: If you are already using VMware virtual machines, you can proceed with the vSphere storage class.
.
Wildcard SSL certificate
. This will increase the validity of the certificate for next 3 months.
Create .
MOSIP uses and elasticsearch to collect logs from all services and reflect the same in Kibana Dashboard.
contains the logstash Index Pattern required by the rest of the dashboards.
contains a Search dashboard which shows only the error logs of the services, called MOSIP Error Logs
dashboard.
contains a Search dashboard which show all logs of a particular service, called MOSIP Service Logs dashboard.
contains dashboards which show insights into MOSIP processes, like the number of UINs generated (total and per hr), the number of Biometric deduplications processed, number of packets uploaded etc, called MOSIP Insight
dashboard.
contains dashboards which show how quickly different MOSIP Services are responding to different APIs, over time, called Response Time
dashboard.
Click to check the detailed installation instructions of all the external components.
Check detailed installation steps.
Purpose | vCPU’s | RAM | Storage (HDD) | no. of VM’s | HA |
1 | Wireguard Bastion Host | 2 | 4 GB | 8 GB | 1 | (ensure to setup active-passive) |
2 | Rancher Cluster nodes (EKS managed) | 2 | 8 GB | 32 GB | 2 | 2 |
3 | Mosip Cluster nodes (EKS managed) | 8 | 32 GB | 64 GB | 6 | 6 |
Loadbalancer | Purpose |
Private loadbalancer Observation cluster | This will be used to access Rancher dashboard and keycloak of observation cluster. Note: access to this will be restricted only with wireguard key holders. |
Public loadbalancer MOSIP cluster | This will be used to access below mentioned services:
|
Private loadbalancer MOSIP cluster | This will be used to access all the services deployed as part of the setup inclusing external as well as all the MOSIP services. Note: access to this will be restricted only with wireguard key holders. |
Purpose VM | Network Interfaces |
1 | Wireguard Bastion Host |
|
1. | Wireguard Bastion Host | 2 | 4 GB | 8 GB | 1 | (ensure to setup active-passive) |
2. | Observation Cluster nodes | 2 | 8 GB | 32 GB | 2 | 2 |
3. | Observation Nginx server (use Loadbalancer if required) | 2 | 4 GB | 16 GB | 2 | Nginx+ |
4. | MOSIP Cluster nodes | 12 | 32 GB | 128 GB | 6 | 6 |
5. | MOSIP Nginx server ( use Loadbalancer if required) | 2 | 4 GB | 16 GB | 1 | Nginx+ |
1. | Wireguard Bastion Host | One Private interface : that is on the same network as all the rest of nodes (e.g.: inside local NAT Network). One public interface : Either has a direct public IP, or a firewall NAT (global address) rule that forwards traffic on 51820/udp port to this interface IP. |
2. | K8 Cluster nodes | One internal interface: with internet access and that is on the same network as all the rest of nodes (e.g.: inside local NAT Network ) |
3. | Observation Nginx server | One internal interface: with internet access and that is on the same network as all the rest of nodes (e.g.: inside local NAT Network). |
4. | Mosip Nginx server | One internal interface : that is on the same network as all the rest of nodes (e.g.: inside local NAT Network). One public interface : Either has a direct public IP, or a firewall NAT (global address) rule that forwards traffic on 443/tcp port to this interface IP. |
1. | rancher.xyz.net | Private IP of Nginx server or load balancer for Observation cluster | Rancher dashboard to monitor and manage the kubernetes cluster. |
2. | keycloak.xyz.net | Private IP of Nginx server for Observation cluster | Administrative IAM tool (keycloak). This is for the kubernetes administration. |
3. | sandbox.xyx.net | Private IP of Nginx server for MOSIP cluster | Index page for links to different dashboards of MOSIP env. (This is just for reference, please do not expose this page in a real production or UAT environment) |
4. | api-internal.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | Internal API’s are exposed through this domain. They are accessible privately over wireguard channel |
5. | api.sandbox.xyx.net | Public IP of Nginx server for MOSIP cluster | All the API’s that are publically usable are exposed using this domain. |
6. | prereg.sandbox.xyz.net | Public IP of Nginx server for MOSIP cluster | Domain name for MOSIP's pre-registration portal. The portal is accessible publicly. |
7. | activemq.sandbox.xyx.net | Private IP of Nginx server for MOSIP cluster | Provides direct access to |
8. | kibana.sandbox.xyx.net | Private IP of Nginx server for MOSIP cluster | Optional installation. Used to access kibana dashboard over wireguard. |
9. | regclient.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | Registration Client can be downloaded from this domain. It should be used over wireguard. |
10. | admin.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | MOSIP's admin portal is exposed using this domain. This is an internal domain and is restricted to access over wireguard |
11. | object-store.sandbox.xyx.net | Private IP of Nginx server for MOSIP cluster | Optional- This domain is used to access the object server. Based on the object server that you choose map this domain accordingly. In our reference implementation, MinIO is used and this domain let's you access MinIO’s Console over wireguard |
12. | kafka.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | Kafka UI is installed as part of the MOSIP’s default installation. We can access kafka UI over wireguard. Mostly used for administrative needs. |
13. | iam.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | MOSIP uses an OpenID Connect server to limit and manage access across all the services. The default installation comes with Keycloak. This domain is used to access the keycloak server over wireguard |
14. | postgres.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | This domain points to the postgres server. You can connect to postgres via port forwarding over wireguard |
15. | pmp.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | MOSIP’s partner management portal is used to manage partners accessing partner management portal over wireguard |
16. | onboarder.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | Accessing reports of MOSIP partner onboarding over wireguard |
17. | resident.sandbox.xyz.net | Public IP of Nginx server for MOSIP cluster | Accessing resident portal publically |
18. | idp.sandbox.xyz.net | Public IP of Nginx server for MOSIP cluster | Accessing IDP over public |
19. | smtp.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | Accessing mock-smtp UI over wireguard |
1. | Wireguard Bastion Host | 2 | 4 GB | 8 GB | 1 | (ensure to setup active-passive) |
2. | Observation Cluster nodes | 2 | 8 GB | 32 GB | 2 | 2 |
3. | Observation Nginx server (use Loadbalancer if required) | 2 | 4 GB | 16 GB | 2 | Nginx+ |
4. | MOSIP Cluster nodes | 12 | 32 GB | 128 GB | 6 | 6 |
5. | MOSIP Nginx server ( use Loadbalancer if required) | 2 | 4 GB | 16 GB | 1 | Nginx+ |
1. | Wireguard Bastion Host | One Private interface : that is on the same network as all the rest of nodes (e.g.: inside local NAT Network). One public interface : Either has a direct public IP, or a firewall NAT (global address) rule that forwards traffic on 51820/udp port to this interface IP. |
2. | K8 Cluster nodes | One internal interface: with internet access and that is on the same network as all the rest of nodes (e.g.: inside local NAT Network ) |
3. | Observation Nginx server | One internal interface: with internet access and that is on the same network as all the rest of nodes (e.g.: inside local NAT Network). |
4. | Mosip Nginx server | One internal interface : that is on the same network as all the rest of nodes (e.g.: inside local NAT Network). One public interface : Either has a direct public IP, or a firewall NAT (global address) rule that forwards traffic on 443/tcp port to this interface IP. |
1. | rancher.xyz.net | Private IP of Nginx server or load balancer for Observation cluster | Rancher dashboard to monitor and manage the kubernetes cluster. |
2. | keycloak.xyz.net | Private IP of Nginx server for Observation cluster | Administrative IAM tool (keycloak). This is for the kubernetes administration. |
3. | sandbox.xyx.net | Private IP of Nginx server for MOSIP cluster | Index page for links to different dashboards of MOSIP env. (This is just for reference, please do not expose this page in a real production or UAT environment) |
4. | api-internal.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | Internal API’s are exposed through this domain. They are accessible privately over wireguard channel |
5. | api.sandbox.xyx.net | Public IP of Nginx server for MOSIP cluster | All the API’s that are publically usable are exposed using this domain. |
6. | prereg.sandbox.xyz.net | Public IP of Nginx server for MOSIP cluster | Domain name for MOSIP's pre-registration portal. The portal is accessible publicly. |
7. | activemq.sandbox.xyx.net | Private IP of Nginx server for MOSIP cluster | Provides direct access to |
8. | kibana.sandbox.xyx.net | Private IP of Nginx server for MOSIP cluster | Optional installation. Used to access kibana dashboard over wireguard. |
9. | regclient.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | Registration Client can be downloaded from this domain. It should be used over wireguard. |
10. | admin.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | MOSIP's admin portal is exposed using this domain. This is an internal domain and is restricted to access over wireguard |
11. | object-store.sandbox.xyx.net | Private IP of Nginx server for MOSIP cluster | Optional- This domain is used to access the object server. Based on the object server that you choose map this domain accordingly. In our reference implementation, MinIO is used and this domain let's you access MinIO’s Console over wireguard |
12. | kafka.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | Kafka UI is installed as part of the MOSIP’s default installation. We can access kafka UI over wireguard. Mostly used for administrative needs. |
13. | iam.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | MOSIP uses an OpenID Connect server to limit and manage access across all the services. The default installation comes with Keycloak. This domain is used to access the keycloak server over wireguard |
14. | postgres.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | This domain points to the postgres server. You can connect to postgres via port forwarding over wireguard |
15. | pmp.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | MOSIP’s partner management portal is used to manage partners accessing partner management portal over wireguard |
16. | onboarder.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | Accessing reports of MOSIP partner onboarding over wireguard |
17. | resident.sandbox.xyz.net | Public IP of Nginx server for MOSIP cluster | Accessing resident portal publically |
18. | idp.sandbox.xyz.net | Public IP of Nginx server for MOSIP cluster | Accessing IDP over public |
19. | smtp.sandbox.xyz.net | Private IP of Nginx server for MOSIP cluster | Accessing mock-smtp UI over wireguard |
Domain name | Mapping details | Purpose |
1 | Load balancer of Observation cluster | Rancher dashboard to monitor and manage the kubernetes cluster. You can share an existing rancher cluser. |
2 | Load balancer of Observation cluster | Administrative IAM tool (keycloak). This is for the kubernetes administration. |
3 | Private Load balancer of MOSIP cluster | Index page for links to different dashboards of Mosip env. (This is just for reference, Please do not expose this page in a real production or uat environment) |
4 | Private Load balancer of MOSIP cluster | Internal API’s are exposed through this domain. They are accessible privately over wireguard channel |
5 | Public Load balancer of MOSIP cluster | All the API’s that are publically usable are exposed using this domain. |
6 | Public Load balancer of MOSIP cluster | Domain name for Mosip’s pre-registration portal. The portal is accessible publicly. |
7 | Private Load balancer of MOSIP cluster | Provides direct access to activemq dashboard. Its limited and can be used only over wireguard |
8 | Private Load balancer of MOSIP cluster | Optional instalation. Used to access kibana dashboard over wireguard |
9 | Private Load balancer of MOSIP cluster | Regclient can be downloaded from this domain. It should be used over wireguard. |
10 | Private Load balancer of MOSIP cluster | Mosip’s admin portal is exposed using this domain. This is an internal domain and is restricted to access over wireguard |
11 | Private Load balancer of MOSIP cluster | Optional- This domain is used to access the object server. Based on the object server that you choose map this domain accordingly. In our reference implementation Minio is used and this domain lets you access Minio’s Console over wireguard |
12 | Private Load balancer of MOSIP cluster | Kafka UI is installed as part of the Mosip’s default installation. We can access kafka ui over wireguard. Mostly used for administrative needs. |
13 | Private Load balancer of MOSIP cluster | Mosip uses an Openid connect server to limit and manage access across all the services. The default installation comes with Keycloak. This domain is used to access the keycloak server over wireguard |
14 | Private Load balancer of MOSIP cluster | This domain points to the postgres server. You can connect to postgres via port forwarding over wireguard |
15 | Public Load balancer of MOSIP cluster | Mosip’s partner management portal is used to manage partners accessing partner management portal over wireguard |
16 | Public Load balancer of MOSIP cluster | accessident resident portal publically |
17 | Public Load balancer of MOSIP cluster | accessing IDP over public |
18 | Private Load balancer of MOSIP cluster | Accessing mock-smtp UI over wireguard |