Masterdata is the necessary base data to run MOSIP services. The data resides in mosip_master
database. This data needs to be customized for a country specific deployment.
Masterdata may be uploaded in the following manner:
One-time bulk upload:
Default masterdata (for sandbox installation): Using Helm chart. The default data uploaded during sandbox installation is available in mosip-data.
Country specific data: Using Python scripts.
Updates: Updates to tables may be done using the Admin Portal.
The tables that need to be modified for country specific data are listed below. Other tables in mosip_master
DB are either system-filled or pre-filled and not to be modified.
Copy Excel files from mosip-data to a folder.
For all tables listed below modify lang_code
and add corresponding rows for your configured languages.
Modify the files for your deployment as per guide below.
Upload first time using scripts given here.
Subsequently, update data ONLY using Admin Portal.
Category | Table | Guide |
---|---|---|
Documents
doc_category
Categories of documents to be captured
doc_type
Specific documents related to a country
applicant_valid_document
Mapping of document category, type and applicant type
Location
loc_hierarchy_list
List of location hierarchy
location
List of locations stored in a hierarchical format
loc_holiday
Holidays specific to different locations. Used in registration centre creation.
Machine
machine_type
Example mobile, stationary. Refers to machine_spec
.
machine_spec
Model, make of the registration machine
ID Schema
identity_schema
Refer to ID Schema customisation. Update the JSON in schema_json
column of identity_schema.xlsx
dynamic_field
Dynamic dropdowns used during data capture
id_type
Only name
and descr
can be changed based on language
ui_spec
UI specification for registration and pre-registration UI screens. See UI spec guides
Registration client
permitted_local_config
List of changeable configurations by the operator
app_authentication_method
Only method_seq
can be changed
app_detail
Only name
and descr
can be changed based on language
app_role_priority
Only priority
can be changed
authentication_method
Only method_seq
can be changed
reason_list
List of reasons for a reason category
reason_category
Only name
and descr
can be changed based on language
Registration center
reg_center_type
Type of center
registration_center
List of registration centers
registration_center_h
Historical data for any modification done on a registration center. One time intialization of this table identical to registration_center
. Thereafter, the data will be system updated
reg_exceptional_holiday
Exception holiday for a registration center
reg_working_nonworking
Working and non-working day for a center
Biometrics
biometric_attribute
Only name
and descr
can be changed based on language
biometric_type
Only name
and descr
can be changed based on language
Templates
template
Only name
, descr
, file_txt
can be changed based on language
template_file_format
Only descr
can be changed based on language
template_type
Only descr
can be changed based on language
Others
blocklisted_words
List of blocked words
daysofweek_list
Only name
can be changed based on language
module_detail
Only name
and descr
can be changed based on language
process_list
Only name
and descr
can be changed based on language
status_list
Only descr
can be changed based on language
status_type
Only name
and descr
can be changed based on language
title
List of titles used in the country
zone
List of administrative zones in a country
The MOSIP platform is configured via the Admin application. This application can be accessed only by a privileged group of administration personnel. The admin module provides the following functions:
Management of resources via CRUD operations:
Zone
Centers (registration centers)
Device
Machine
Users (Admin, registration staff)
Registration administration
Packet status
Retrieve lost RID
Resume RID
Administrative zones are virtual boundaries which a country can define to better manage their resources that are used during registrations. These resources includes Centers, Users, Machines and Devices. These zones can be defined in a hierarchical fashion and a country can allocate resources to such zones based on their requirements.
Resources under each zone is managed by a Zonal Admin. This is done by assigning an Administrative zone to the Zonal Admin during user creation.
These Zonal Admins can exist at any zonal hierarchy. (For e.g, a Zonal Admin can directly be mapped to the whole country as a Zone or can be mapped to a significantly smaller zone such as a city). Thus, these resources when mapped to an Administrative zone can only be managed by the Admin of that zone.
Deactivation refers to a temporary shutdown. This means that the resource will not be available for use unless and until it is activated later through the admin portal as required by the country.
Decommission refers to a permanent shut down of the resource. This also automatically deactivates it.
The primary difference being that a decommissioned resource cannot be bought into commission again as decommission refers to a permanent shutdown.
Also, in cases where a center has some resources mapped to it (e.g. machines, devices or users), the portal will not allow the admin to decommission such a center.
Note- Activation/Deactivation/Decommission of a center in one language will be applied to the same center created in all the languages.
Reference implementation of the Admin portal is available in admin-ui repository.
To know more, refer to the Admin portal user guide.
To know more about the setup, read Admin Services Developer's Guide.
Refer API Documentation.
The admin application is a web-based application used by a privileged group of administrative personnel to manage various master data. The various resources that can be managed by an administrator are:
Center (Registration centers)
Device
Machine
User (Admin, Registration staff)
Along with resource and data management, the admin can generate master keys, check registration status, retrieve lost RIDs, and resume processing of paused packets.
Masterdata Service exposes API to perform CRUD operations on masterdata through Admin service.
Hotlist Service provides functionality to block/unblock any IDs with option of expiry. This hotlisted information will also be published to MOSIP_HOTLIST WebSub topic.
Sync Data Service can be accessed only by the privileged group of admin personnel and enables default configurations and seed data to be setup when the MOSIP platform gets initialized.
The admin module has four services:
Admin service
Kernel Masterdata service
Kernel Syncdata service
Hotlist service
The documentation here will guide you through the pre-requisites required for the developer' setup.
JDK 11
Any IDE (like Eclipse, IntelliJ IDEA)
Apache Maven (zip folder)
pgAdmin
Postman
Git/GitHub Desktop
Notepad++ (optional)
lombok.jar (file)
settings.xml
Follow the steps below to set up Admin Services on your local system:
Download lombok.jar and settings.xml.
Unzip Apache Maven and move the unzipped folder in C:\Program Files
and settings.xml
to conf
folder C:\Program Files\apache-maven-3.8.4\conf
.
Install Eclipse, open the lombok.jar
file and wait for some time until it completes the scan for Eclipse IDE and then click Install/ Update
.
Check the Eclipse installation folder C:\Users\userName\eclipse\jee-2021-12\eclipse
to see if the lombok.jar
is added. By doing this, you don't have to add the dependency of lombok
in your pom.xml
file separately as it is auto-configured by Eclipse.
Configure the JDK (Standard VM) with your Eclipse by traversing through Preferences → Java → Installed JREs
.
For the code setup, clone admin-services repository and follow the guidelines mentioned in the Code Contributions.
Open the project folder where pom.xml
is present.
Open command prompt from the same folder.
Run the command mvn clean install -Dgpg.skip=true -DskipTests=true
to build the project and wait for the build to complete successfully.
After building of a project, open Eclipse and select Import Projects → Maven → Existing Maven Projects → Next → Browse to project directory → Finish
.
After successful importing of project, update the project by right-click on Project → Maven → Update Project
.
For the environment setup, you need an external JAR that is available here with different versions. (E.g.: You can download kernel-auth-adapter.jar
and add to project Libraries → Classpath → Add External JARs → Select Downloaded JAR → Add → Apply and Close
).
Clone admin-services repository.
Any changes in the properties for Masterdata and Admin services should be done in application-local1.properties
file.
By default the Admin-services is connected to dev environment.
To run the specific service from IDE, Open IDE -> Specific service -> src/main/java/io.mosip.specific service -> Right click on the file and select run as Java Application
.
For example, to run the Admin service, open IDE -> admin-service -> src/main/java/io.mosip.admin -> Right click on AdminBootApplication.java and select run as Java Application
.
To run the specific service from Command Prompt, Open Project folder -> open command prompt from same folder -> Execute java -jar target/specific-service-1.2.0.jar
.
For example, to run the admin service, Open Project folder -> open command prompt from same folder -> Execute java -jar target/admin-service-1.2.0-SNAPSHOT.jar
.
The service should now be up and running.
For API documentation, refer here.
The APIs can be tested with the help of Swagger-UI and Postman.
Swagger is an interface description language for describing restful APIs expressed using JSON. You can access Swagger-UI of admin-services for dev-environment from:
Admin service– http://dev.mosip.net/v1/admin/swagger-ui/index.html?configUrl=/v1/admin/v3/api-docs/swagger-config#/
and localhost from http://localhost:8098/v1/admin/swagger-ui/index.html?configUrl=/v1/admin/v3/api-docs/swagger-config#/
.
Masterdata- http://dev.mosip.net/v1/masterdata/swagger-ui/index.html?configUrl=/v1/masterdata/v3/api-docs/swagger-config#/
and localhost from http://localhost:8086/v1/masterdata/swagger-ui/index.html?configUrl=/v1/masterdata/v3/api-docs/swagger-config#/
.
Syncdata- http://dev.mosip.net/v1/syncdata/swagger-ui/index.html?configUrl=/v1/syncdata/v3/api-docs/swagger-config#/
and localhost from http://localhost:8089/v1/syncdata/swagger-ui/index.html?configUrl=/v1/syncdata/v3/api-docs/swagger-config#/
.
Hotlist- http://dev.mosip.net/v1/hotlist/swagger-ui/index.html?configUrl=/v1/hotlist/v3/api-docs/swagger-config#/
and localhost from http://localhost:8095/v1/hotlist/swagger-ui/index.html?configUrl=/v1/hotlist/v3/api-docs/swagger-config#/
Postman is an API platform for building and using APIs. Postman simplifies each step of the API lifecycle and streamlines collaboration so you can create better APIs—faster. It is widely used tool for API testing.
Create an environment as shown in the image below.
This environment is created for dev. Give the variable name as url
and set both the values as https://dev.mosip.net
.
In the similar way, create another environment for localhost as shown below.
This environment is created for localhost. Give the variable name as url
and set both the values as http://localhost:8099
.
Admin application is a web-based application used by a privileged group of administrative personnel to manage various master data. The various resources that can be managed by an Admin are:
Center (Registration centers)
Device
Machine
Users (Admin, Registration staff)
Along with the resource and data management, the admin can generate master keys, check registration status, retrieve lost RID, resume processing of paused packets. To start using the Admin portal, an admin user must be assigned to a zone.
To learn more, refer the videos below!
Session1
Session2
Setup of hierarchial zones
Create Admin roles in KeyCloak
Create first admin user in KeyCloak and assign "GLOBAL_ADMIN" role
Note- On login of first admin user, user zone mapping is handled automatically.
The above are done automatically as part of default sandbox installation.
Select the preferred language.
Login with KeyCloak credentials.
Map the other users(admins/registration operators/supervisors) to respective zones
Create centers and assign the users to a particular center
Highly recommended: Ensure to revoke the first super user's zone mapping and role after first user actions are completed.
GLOBAL_ADMIN
ZONAL_ADMIN
REGISTRATION_ADMIN
MASTERDATA_ADMIN
KEY_MAKER
This portal allows an Admin to view, create, edit, activate, deactivate and decommission registration centers.
An Admin can manage only centers under their administrative zones.
The administrator can filter the list of registration centers based on parameters like Center name, Center type, Status, Location code.
The system does not fetch the details of decommissioned registration centers but only active and inactive centers are displayed.
If the admin does not find a center, they can click the Center not available in logged in language button. Clicking on this button, displays the list of centers that are already created in other languages. On selecting a particular center, the information will be auto-populated in the Create page and be made available to the admin for modifications.
Language specific fields can be modified to create a center with the currently logged in language.
A center is created with multiple attributes and is mapped to the administrative zone that it belongs to.
A center can only be mapped to the configured location hierarchy level.
While defining centers, an admin can also define the working days of the week for a center and any exceptional holidays that might be applicable for a particular center.
An admin can update a center even after it has been created. The updates can include adding the details that were missed during creation of the center or changing the details of a center as required.
To update, click the Edit option from the Actions menu against a center name.
Note- Updates made to language specific fields updates data only for that language in the database while updates made to non-language dependent fields updates data against all the language entries for that center.
Select the Deactivate/Decommission option from the Actions menu against the center.
Activation/Deactivation/Decommission of a center in one language will be applied to the same center created in all the languages.
To know more, refer Activate/deactivate/decommission resources
Using this portal, an admin can manage the devices a country will use for registering residents like devices used for bio-metric capture (Fingerprint, Iris, Web camera, etc.), printers, scanners.
This portal allows an Admin to view, create, edit, activate, deactivate and decommission registration centers.
The admin portal allows an admin to view the list of all the devices available in the jurisdiction of their administrative zone.
The system does not fetch the details of decommissioned devices but only the active and inactive devices.
Note
Device entity is language agnostic (independent of languages).
The data collected about Devices is used only for book keeping, i.e., MOSIP system does not use this data for any validation.
The Admin can filter the list of Registration centers based on parameters like Device Name, Mac Address, Serial Number, Status, Map Status, Device Type, Device Spec ID.
A Device can be created with the multiple attributes and be mapped to the Administrative Zone it belongs to.
An admin can update missing information or change device details even after it is created.
To update, click the Edit option from the Actions menu against a device.
Select the Deactivate/Decommission option from the Actions menu against the device.
Admin portal allows an Admin to map/un-map each device to a center.
This mapping specifies as to which center the device will be used in.
A device can only be mapped to a center which belongs under the device’s Administrative Zone.
To do so, select the device and choose a Center Name from the dropdowm.
Admin portal allows an administrator to manage the machines a country will use for registering residents.
This portal allows an Admin to view, create, edit, activate, deactivate and decommission machines.
The admin portal allows an admin to view the list of all the machines available in the jurisdiction of their administrative zone.
The system does not fetch the details of decommissioned machines but only shows the active and inactive machines.
Note:
Machine entity is also language agnostic.
The administrator can filter the list of machines based on parameters like Machine name, Mac address, Serial number, Status, Machine type.
A machine can be created with the attributes like Machine ID, machine name, mac address, serial number, machine spec ID and administrative zone the machine belongs to.
A machine needs to be mapped to an administrative zone.
An admin can update missing details or make changes to machine details even after it is created.
To update, click the Edit option from the Actions menu against a machine.
Note- Updates made to language specific fields updates data only for that language in the database while updates made to non-language dependent fileds updates data against all the language entries for that center.
An admin can deactivate or decommission a machine through the admin portal.
Admin portal allows an Admin to map/un-map each machine to a center.
This mapping specifies as to which center the machine will be used in.
A machine can only be mapped to a center which belongs under the machine’s Administrative Zone.
To do so, select the machine and choose a Center Name from the dropdowm.
MOSIP uses Keycloak as an IAM (Identity access management tool) for managing Users. These users are internal users of MOSIP including Registration Officers, Registration Supervisors, Zonal Admins, Global Admins etc.
using this portal, an Admin can map the users to a zone and a center.
Once the user is created in KeyCloak, they need to be mapped to a zone to get access to specific information available in that zone.
Admin portal allows an admin to map users to a zone. This mapping specifies as to which zone the user will belong to.
A user can only be mapped to a zone which belongs under the user’s Administrative Zone.
A user can later be un-mapped from the zone in case a user needs to be moved to another zone. In such cases, the user will later need to be mapped to the new zone. Below image displays the list of users that are mapped to a zone.
Click Resources-> User Zone mapping
Click +Map Zone
Select the User Name, Administrative Zone from the dropdown.
Click Save.
To re-map a user to a zone,
Click Resources-> User Zone mapping
Select Remap from the Actions menu against the mapped user.
Update the User Name/ Administrative Zone from the dropdown.
Click Save.
Note- If the center is already mapped, the admin needs to unmap the center to remap the zone.
Once the user is mapped to a zone, they will be listed in the screen below. Now, the user will be mapped to a center to be able to manage their assigned center.
Admin portal allows an admin to map users to a center. This mapping specifies as to which center the user will be used in.
A user can only be mapped to a center which belongs under the user’s Administrative Zone.
A user can later be un-mapped from the Center in cases where a User is needed to be moved to another Center. In such cases, the user will later need to be mapped to the new center. In case the user is required to be mapped to a Registration center outside the Administrative Zonal restriction, the Administrative Zone of the user must be changed.
Map/un-map/re-map user to a registration center
To map a user to a center,
Click Resources-> User Center Mapping
Select Map from the Actions menu against the mapped user.
Select the Center Name from the dropdown against the User Name, Administrative Zone.
Click Save.
To get the results starting with specific character/ set of characters, prepend that specific character/set of characters with asterisk
symbol.
Similarly to get the results ending with specific character/ set of characters, append that specific character/ set of characters with asterisk
.
For the results containing a specific character/ set of characters, prepend and append that specific character/ set of characters with asterisk
.
Below is the image illustrating the same.
A Registration packet generated in Registration client is sent to Registration Processor for further processing and UIN generation.
Using this Portal, A Registration Admin can view the status of a packet by entering the RID of the packet.
The packet status will contain all the stages the packet has passed through along with the last stage the packet is in.
In case the packet has not been processed or is marked for Re-Send/Re-Register, the admin will be able to view specific comments indicating the reason for that particular status.
The Registration Admin has the privilege to view the registration packets that are in a paused state.
Admin can use this portal to resume or reject paused packets. They would have 3 options:
Resume processing (from where it was paused)
Resume from the beginning
Reject
Once processing of a packet is resumed, it will be removed from this list
The Registration Admin can use this feature to retrieve lost RID.
For instance, if the resident did not provide any valid email and/or phone number and has lost the RID slip received during the registration, in order to find their RID details, the resident contact MOSIP helpline and share details such as name, centre name, registration date and postal code to the admin, who will use the lost RID feature and try to retrieve the RID number.
A few filters may be applied to retrieve the RID.
Note: This feature is currently under development.
Admin portal allows an Admin to manage the Masterdata applicable for a country.
These data includes list of Genders, list of Holidays, Templates, Center Types, Machine Types etc.
To know more, refer to Masterdata guide.
If a country decides to uplaod the data through the .csv files, they could use this feature to upload the existing data into the MOSIP platform.
The listing screen displays the uploaded data transaction information.
As the information inside .csv files may be huge, it would go through the batch job to process the information and store it in the tables. Also, it may take time to get unique transaction ID against the particular action.
To upload Master data using Admin portal,
Go to Bulk Upload > Master Data
On the master data dashboard, click Upload Data.
Select the operation (insert/update/delete)
Select the table name into which the data needs to be uploaded into.
Click Choose file to select the data and click Upload
To view the format for inserting data in a particular table, click on the Download icon.
A CSV file gets downloaded in which the first row represents the column names and the rest of the rows are the data which will be inserted into the table(sample).
From 1.2.0.1-B2 version, apart from comma, other special characters (i.e., '|','$'etc.) can also be used as a separator in the csv file used for masterdata bulk upload. This can be done by updating the property mosip.admin.batch.line.delimiter
with the same special character.
Note: While editing CSV files, it is recommended to keep track of the Date format and Time format to be the same as the acceptable formats. The acceptable Date format is YYYY-MM-DD and the acceptable Time format is HH:MM:SS. Any other Date and Time formats in CSV files will result in a DataType Mismatch Error
.
To upload packets using Admin portal,
Go to Bulk Upload > Packets
On the packet upload dashboard, click Upload Packet.
Select the following from the dropdown:
Center name
Source (currently displays Registration Client)
Process (New, Update UIN, Lost, Biometric correction)
Supervisor status (Approved/Rejected)
These details are important if the packet needs to be synced before upload.
Click Choose file to select the packets and click Upload.
How is the packet upload performed with or without DATA_READ role?
For uploading the packets through the Admin portal, ensure that the packets are available in the machine or the external hard disk connected from where the Admin Portal is being used.
With the help of this feature, the Admin user can generate and manage the keys required in MOSIP.
The logged in user with KEY_MAKER
role will have access to view and generate the master key in the Admin portal.
Using this option, the logged in user will be able to generate only the Root key and Module master key. To generate the key, the user has to select the Application ID from the options available in the drop down, leave the Reference ID as blank for Root and Module master key and provide other certificate attributes to be used at the time of generation of certificate for the key.
This certificate attributes in the portal are optional, if not provided, default values configured in Key Manager service will be used.
For Kernel signature key (which is considered as the master key and stored in HSM), Reference ID needs to be provided and the value has to be SIGN
.
Force flag option is available in key generation. The logged in user can select option value True to force invalidating existing key and generate new key in Key Manager service.
The logged in user has to select the return object after the generation of key.
The user can select either Certificate or CSR (Certificate Signing Request). The key will be generated only when the key is not available in Key Manager service otherwise already generated key certificate will be returned for the generation request.
CSR (certificate signing request) is required when there is a need to procure a valid certificate from a valid CA.
GenerateCSR option can be used to request for a CSR and this option will be visible to all the users who log in to the Admin portal.
The logged in user can request for generation of CSR for any key generated in Key Manager service.
The user has to provide the Application ID and Reference ID to get a CSR.
New key will be auto-generated in case the key does not exist and the already existing key has expired for the Module Encryption keys.
The user can get certificate for all the keys generated in Keymanager and any partner certificates uploaded in Keymanager service for partner data share purpose.
GetCertificate option is visible to all the users who log in to the Admin portal.
The user has to provide the Application ID and Reference ID to get a certificate.
New key will be auto generated in case the key does not exist and the already existing key has expired for Module encryption keys.
Only current valid certificates will be returned when the user requests for a certificate.
The logged in user can use this option to update the certificate for all the keys generated in Key Manager service.
This option is used in scenarios where a valid CA certificate has been procured for a key available in Key Manager service.
The logged in use can use this option to upload partner certificate in Key Manager service.
Partner certificates will be used in Key Manager service to encrypt any sharable data using the partner certificate required in datashare from MOSIP to any partner.
Partner certificates can also be used in Key Manager service for signature verification purpose.
GLOBAL_ADMIN | ZONAL_ADMIN | REGISTRATION_ADMIN | MASTERDATA_ADMIN | KEY_MAKER |
---|---|---|---|---|
To map a user to a zone,
LoggedIn User Role | Packet Sync | Packet Upload |
---|---|---|
Centers
Devices
Packet Status
Devices
GenerateMasterKey
User Zone Mapping
Machines
Pause/ Resume RID
Machines
GenerateCSR
All Master Data
User Zone Mapping
Retrieve Lost RID
All Master Data
GetCertificate
Masterdata Bulk Upload
User Center Mapping
Packet Bulk Upload
Masterdata Bulk Upload
UploadCertificate
All Master Data
UploadOtherDomainCertificate
Masterdata Bulk Upload
With DATA_READ
Yes
only after successful sync
Without DATA_READ
No
Yes