Comment on page
Performance Test Report
This document provides details on the performance measurement approach of prioritized scenarios of MOSIP modules and provides detailed reports on the results.
APIs in all modules are prioritized based on the high load expectation and usage frequency by other modules.
There are 3 types of performance scenarios covered in this report
- Prioritized single API performance (For most modules)
- End to end UI API sequence performance (For pre-registration module)
- End to end message delivery performance (For web-sub module)
All individual test scenarios from the modules are listed below.
Module Name | API Scenario | API Endpoint |
Kernel | Add Audits | v1/auditmanager/audits |
Kernel | Client Id - Secret Key | v1/authmanager/authenticate/clientidsecretkey |
Kernel | User Id Pwd | v1/authmanager/authenticate/internal/useridPwd |
Kernel | Validate Token | v1/authmanager/authorize/admin/validateToken |
Kernel | Generate OTP | v1/otpmanager/otp/generate |
Kernel | Validate OTP | v1/otpmanager/otp/validate?key={keyValidate}&otp={otpValidate} |
Kernel | Send SMS | v1/notifier/sms/send |
Kernel | Send EMAIL | v1/notifier/email/send |
Kernel | Generate PRID | v1/pridgenerator/prid |
Kernel | Generate UIN | v1/idgenerator/uin |
Kernel | Generate VID | v1/idgenerator/vid |
Kernel | Generate RID | v1/ridgenerator/generate/rid/{centerID}/{machineID} |
IDA | Auth with OTP | idauthentication/v1/auth/{mispLicenseKey}/{rpPartnerId}/{rpApiKey} |
IDA | KYC with OTP | idauthentication/v1/kyc/{mispLicenseKey}/{ekycPartnerId}/{ekycApiKey} |
IDA | Auth with bio | idauthentication/v1/auth/{mispLicenseKey}/{rpPartnerId}/{rpApiKey} |
IDA | KYC with bio | idauthentication/v1/kyc/{mispLicenseKey}/{ekycPartnerId}/{ekycApiKey} |
IDA | Send OTP | idauthentication/v1/otp/{mispLicenseKey}/{rpPartnerId}/{rpApiKey} |
Pre-Registration | Prereg UI end to end flow | Prereg UI end to end flow |
Syncdata | Public Key Verify | v1/syncdata/tpm/publickey/verify |
Syncdata | Get Certificate | v1/syncdata/getCertificate?applicationId={appID}&referenceId={refID} |
Syncdata | Get User Details | v1/syncdata/userdetails?keyindex={keyIndex} |
Syncdata | Get Client Settings | v1/syncdata/v2/clientsettings?keyindex={keyIndex} |
Syncdata | Get Configs | v1/syncdata/configs/{name} |
Syncdata | Get LatestID Schema | v1/syncdata/latestidschema?schemaVersion={schemaVersion} |
Syncdata | Get CAcertificates | v1/syncdata/getcacertificates |
Regproc | Sync Registration Packet Details | registrationprocessor/v1/registrationstatus/sync |
Regproc | Get Packet status | registrationprocessor/v1/registrationstatus/search |
Regproc | Upload Registration Packet | registrationprocessor/v1/packetreceiver/registrationpackets |
Resident | Request/Send OTP | resident/v1/req/otp |
Resident | RID Check Status | resident/v1/rid/check-status |
Resident | Auth Lock | resident/v1/req/auth-lock |
Resident | Auth Unlock | resident/v1/req/auth-unlock |
Resident | Credential Request | resident/v1/req/credential |
Resident | Auth History | resident/v1/req/auth-history |
Resident | Generate VID | resident/v1/vid |
Datashare | Create Data Share URL | v1/datashare/create/{policyId}/{subscriberId} |
Datashare | Get Data Share File | v1/datashare/get/{policyId}/{subscriberId}/{randomShareKey} |
IDRepo | Retrieve Identity using UIN | idrepository/v1/identity/idvid/{uin} |
IDRepo | Retrieve Identity using VID | idrepository/v1/identity/idvid/{vid} |
IDRepo | Add Identity | idrepository/v1/identity/ |
IDRepo | Update Identity | idrepository/v1/identity/ |
IDRepo | Auth Type Status | idrepository/v1/identity/authtypes/status |
IDRepo | Create VID | idrepository/v1/vid |
IDRepo | Update VID | idrepository/v1/vid/{vidGenerated} |
Masterdata | Get Registration Centers | v1/masterdata/registrationcenters |
Masterdata | Get Templates | v1/masterdata/templates |
Masterdata | Get Templates By Lang & Temp Code | v1/masterdata/templates/{lang&TempTypeCode} |
Masterdata | Get Latest ID Schema | v1/masterdata/idschema/latest?schemaVersion={schemaVersion} |
Masterdata | Get Dynamic Fields | v1/masterdata/dynamicfields?pageNumber=0&pageSize=10&sortBy=name&orderBy=desc |
Masterdata | Get Users | v1/masterdata/users/{userID}/{time(yyyy-MM-dd'T'HH:mm:ss.sss'Z',)} |
Keymanager | Encrypt Data | v1/keymanager/encrypt |
Keymanager | Encrypt Data With Pin | v1/keymanager/encryptWithPin |
Keymanager | Generate JWT Signature | v1/keymanager/jwtSign |
Keymanager | Decrypt Data | v1/keymanager/decrypt |
Keymanager | Decrypt Data With Pin API | v1/keymanager/decryptWithPin |
Keymanager | Verify JWT Signature | v1/keymanager/jwtVerify |
Tools | Purpose |
Jmeter | To verify the Load and Performance of all the applications. |
The approach to performance testing differs based on the testing scenarios and are detailed as per the categorization as below.
- Run a single pod of the application hosting the required API
- Run four pods each of the dependent applications to cater the load implied from the test application
- Simulate varied load profilers like 10, 30, 50 etc.. concurrent users from Jmeter for that single API for a period of one hour
- Record the average and 90th percentile response time metrics specific to each load profiles
- Verify that JVMs show stable memory and CPU usage over the increasing load profiles.
- Run a single pod for each application hosting the required APIs called directly from Jmeter
- Run four pods each of the dependent applications to cater the load implied from the test application
- Simulate varied load profilers like 50, 80, 100 etc.. concurrent users from Jmeter for end to end sequence of APIs called from UI for a period of one hour
- Record the average and 90th percentile response time metrics for entire end to end API sequence for each load profiles
- Verify that JVMs show stable memory and CPU usage over the increasing load profiles.
- Run a single pod for each application hosting the required APIs called directly from Jmeter
- Run four pods each of the dependent applications to cater the load implied from the test application
- Simulate varied load profilers like 10, 30, 50 etc.. concurrent users to publish continues messages to the test application for a period of one hour
- Record the average and 90th percentile response time metrics for publish API for each load profiles
- Record the average and 90th percentile delivery time metrics for entire end to end message delivery for each load profiles
Title | Description |
Build Tag Version | 1.2.0-rc2 |
Performance script source location | https://github.com/mosip/mosip-performance-tests-mt/tree/1.2.0 |
CPU (cores) | Memory (GB) | HDD (GB) | OS | Count | Purpose |
8 | 30 | 768 | Cent OS | 1 | Kubernetes console machine |
8 | | | Kubernetes node machines | | |
4 | 16 | 500 | Ubuntu | 1 | Database machine |
4 | 16 | 835 | Windows | 3 | Jmeter test machines |
Pod/Application Names | Resource Allocation (Limits) | | |
| Memory (M) | CPU (m) | Heap (M) |
kernel-auditmanager-service | 2500 | 1000 | 1750 |
kernel-auth-service | 2250 | 500 | 1500 |
kernel-otpmanager-service | 1750 | 500 | 1000 |
kernel-notification-service | 1500 | 500 | 1000 |
kernel-pridgenerator-service | 1750 | 300 | 1000 |
kernel-idgenerator-service | 3500 | 500 | 2000 |
kernel-ridgenerator-service | 1750 | 500 | 1000 |
ida-auth-service | 4000 | 4500 | 2000 |
ida-otp-service | 3000 | 3000 | 2000 |
prereg-application-service | 2500 | 1000 | 2000 |
kernel-syncdata-service | 5000 | 500 | 4000 |
regproc-registration-status-service | 4000 | 500 | 2000 |
regproc-stage-group-1 | 5000 | 1000 | 4000 |
resident-service | 3000 | 500 | 2000 |
datashare-service | 3000 | 500 | 2000 |
idrepo-identity-service | 4000 | 1000 | 2000 |
idrepo-vid-service | 3000 | 500 | 1000 |
kernel-masterdata-service | 2250 | 500 | 1500 |
kernel-keymanager-service | 5000 | 1000 | 4000 |
90th Percentile Response Times
Detailed Metrics for all 14 modules is available below:
Run 1: 50 concurrency
Jmeter Aggregate Report
Label | # Samples | Average | Median | 90% Line | 95% Line | 99% Line | Min | Max | Error % | Throughput | Received KB/sec | Sent KB/sec |
---|---|---|---|---|---|---|---|---|---|---|---|---|
Add Audit API Execution | 231236 | 777 | 395 | 2206 | 2700 | 3402 | 3 | 8420 | 0.00% | 64.22209 | 38.88 | 186.99 |
Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana


Application Response Time Graph from Kibana

Dependent Application Response Time Graph from Kibana
Dependency app graph for auth manager service was empty so not attached.
Run 2: 30 concurrency
Jmeter Aggregate Report
Label | # Samples | Average | Median | 90% Line | 95% Line | 99% Line | Min | Max | Error % | Throughput | Received KB/sec | Sent KB/sec |
---|---|---|---|---|---|---|---|---|---|---|---|---|
Add Audit API Execution | 266879 | 403 | 105 | 1199 | 1893 | 2409 | 2 | 3870 | 0.00% | 74.12657 | 44.88 | 215.82 |
Response Time Percentile Graph using Jmeter

CPU and Memory Utilization Graphs using Grafana


Application Response Time Graph from Kibana

Dependent Application Response Time Graph from Kibana
Application dependency graph was empty so not attached.
Run 1: 30 concurrency
Aggregate Report from Jmeter
Label | # Samples | Average | Median | 90% Line | 95% Line | 99% Line | Min | Max | Error % | Throughput | Received KB/sec | Sent KB/sec |
---|---|---|---|---|---|---|---|---|---|---|---|---|
Client Id - Secret Key API Execution | 241902 | 449 | 202 | 483 | 597 | 3352 | 9 | 60051 | 0.16% | 66.1073 | 290.08 | 28.49 |
User Id Pwd API Execution | 220878 | 488 | 488 | 680 | 711 | 872 | 70 | 6616 | 0.00% | 61.34721 | 215.9 | 30 |
Validate Token API Execution | 451378 | 239 | 201 | 400 | 496 | 680 | 6 | 3506 | 0.00% | 125.3778 | 582.61 | 235.51 |
Response Time Percentile Graph using Jmeter

We only have the graph for Validate token as the previous ones were cleared
CPU and Memory Utilization Graphs using Grafana


Application Response Time Graph from Kibana

Dependent Application Response Time from Kibana
Dependency app graph was empty so so not attached.
Notes:
Keycloak was having issue by end of client id secret key API run, so the keycloak was restarted before the start of user id password API
Run 2: 50 concurrency
Aggregate Report from Jmeter
Label | # Samples | Average | Median | 90% Line | 95% Line | 99% Line | Min | Max | Error % | Throughput | Received KB/sec | Sent KB/sec |
---|---|---|---|---|---|---|---|---|---|---|---|---|
Client Id - Secret Key API Execution | 243069 | 740 | 499 | 1694 | 2100 | 2833 | 9 | 7511 | 0.00% | 67.51412 | 296.36 | 29.09 |
User Id Pwd API Execution | 27121 | 5519 | 3290 | 11676 | 22351 | 43894 | 40 | 60047 | 0.39% | 9.05834 | 31.82 | 4.42 |
Validate Token API Execution | 181953 | 989 | 810 | 2107 | 2496 | 3206 | 6 | 8111 | 0.00% | 50.53694 | 234.74 | 94.93 |
Response Time Percentile Graph using Jmeter


CPU and Memory Utilization Graphs using Grafana


Application Response Time Graph from Kibana



Dependent Application Response Time Graph from Kibana
Dependency app graph was empty so so not attached.
Notes:
Keycloak was having issue by end of user id password API run, so the keycloak was restarted before the start of validate token API
Run 1: 30 concurrency
Aggregate Report from Jmeter
Label | # Samples | Average | Median | 90% Line | 95% Line | 99% Line | Min | Max | Error % | Throughput | Received KB/sec | Sent KB/sec |
---|---|---|---|---|---|---|---|---|---|---|---|---|
Generate OTP API Execution | 87777 | 1229 | 1104 | 2160 | 2496 | 3207 | 5 | 7418 | 0.00% | 24.37841 | 15.57 | 48.04 |
Validate OTP API Execution | 150000 | 1275 | 1128 | 2192 | 2404 | 3196 | 0 | 6268 | 0.02% | 23.4176 | 14.89 | 42.63 |
Notification Manager SMS API Request | 433004 | 246 | 99 | 692 | 808 | 1119 | 2 | 2625 | 0.00% | 120.2695 | 80.92 | 259.67 |
Notification Manager EMAIL API Request | 69727 | 1550 | 1396 | 2992 | 3602 | 5101 | 4 | 13607 | 0.04% | 14.87585 | 9.48 | 41.37 |
PRID Generator API Execution | 102606 | 1052 | 1022 | 1893 | 2073 | 2432 | 5 | 5104 | 0.00% | 28.49246 | 11.74 | 51 |
Generate UIN API Execution | 39971 | 2698 | 2672 | 2813 | 2833 | 3536 | 127 | 8776 | 0.00% | 11.09746 | 26.27 | 23.71 |
Generate VID API Execution | 205377 | 154 | 139 | 266 | 321 | 423 | 5 | 12380 | 6.39% | 193.1971 | 458.89 | 412.75 |
RID Generator API Execution | 368802 | 291 | 193 | 743 | 930 | 1324 | 5 | 5761 | 0.00% | 102.4346 | 63.42 | 221.07 |
Note: idgenerator- generate vid api threw error as “VID not available for allocation error”, so it ran only for around 20 mins rest all other api’s ran for the complete 1 hour run.
Response Time Percentile Graph using Jmeter


CPU and Memory Utilization Graphs using Grafana




Application Response Time Graph from Kibana




Note: idgenerator & pridgenerator services are vertx apps so there are no kibana graphs for the same
Dependent Application Response Time Graph from Kibana
Dependency app graph was empty so above are the graphs for the application services from kibana
Notes: Some of the kernel service api’s failed with oomkilled & authentication failed issues, so restarted the same and then continued the load test from the next set of api’s that’s why we have multiple graphs from jmeter & grafana.
Run 2: 10 & 50 concurrency
Aggregate Report from Jmeter
Label | # Samples | Average | Median | 90% Line | 95% Line | 99% Line | Min | Max | Error % | Throughput | Received KB/sec | Sent KB/sec |
---|---|---|---|---|---|---|---|---|---|---|---|---|
Generate OTP API Execution | 768397 | 45 | 9 | 92 | 95 |