Pre-requisites

CTK should be deployed along with the required dockers as mentioned below.

• compliance-toolkit-service: 1.4.0

• compliance-toolkit-ui: 1.4.0

• compliance-toolkit-batch-job: 1.4.0

To successfully deploy Compliance Toolkit, below mentioned services are mandatorily required.

Dependent Service (dockers)

• Artifactory: mosipid/artifactory-ref-impl: 1.2.0.1-B2

• Audit manager: mosipid/kernel-auditmanager-service: 1.2.0.1-B1

• Auth Manager: mosipid/kernel-authmanager: 1.2.0.1-B1

• Key Manager: modipid/kernel-keymanager-service: 1.2.0.1-B1

• Partner Management: mosipid/partner-management-service: 1.2.0.1-B1

• KeyCloak: mosipid/keycloak-init: 1.2.0.1-B1

• Postgres: mosipid/postgres-init: 1.2.0.1-B1

• Config Server: config-server: mosipid/config-server: 1.1.2

• Notification Service: mosipid/kernel-notification-service: 1.2.0.1-B1

• ClamAV: clamav/clamav: latest

• MinIO

The Setup guide serves as a checklist for the following three categories.

1. Configuration checks

2. Steps to load testdata, schemas, testcases and terms and conditions templates

3. Steps to generate Android APK

Configuration checks

1. Make sure that the kernel-default.properties file includes the mosip-toolkit-client and mosip-toolkit-android-client values in the auth.server.admin.allowed.audience setting. If these values are not set by default, configure them and then restart the kernel-auth-service and compliance-toolkit-service.

2. Ensure that in compliance-toolkit-default.properties, CORS is enabled to allow access to mosip-toolkit-android-client:

   Copy

    mosip.security.cors-enable=true
    mosip.security.origins=http://localhost
    auth.server.admin.allowed.audience=mosip-toolkit-client,mosip-toolkit-android-client

   If this was not set by default, then set it and restart compliance-toolkit-service.

3. Check if the roles given to mosip-pms-client match with any of the roles for the following config property: mosip.role.keymanager.postverifycertificatetrust=XXX

   This config property is available here.

   For Example:

   mosip.role.keymanager.postverifycertificatetrust=ZONAL_ADMIN, GLOBAL_ADMIN, PMS_ADMIN, PMS_USER

   Ensure that the mosip-pms-client possesses any of the roles mentioned above.

4. Check that mosip-pms-client has the role REGISTRATION_PROCESSOR, PARTNER_ADMIN, PMS_ADMIN in Key Cloak. If this was not set by default, then set it and restart keymanager and compliance-toolkit-service.

5. It is also needed to generate an encryption key for CTK.

   • Insert the following row to create a new app ID.

     INSERT INTO keymgr.key_policy_def(app_id, key_validity_duration, is_active,pre_expire_days, access_allowed, cr_by, cr_dtimes, upd_by, upd_dtimes, is_deleted, del_dtimes) VALUES ('COMPLIANCE_TOOLKIT', 1095, true, 60, 'NA', 'mosipadmin', '2022-11-28 09:00:40.822625', null, null, false, null);

   • Using the auth manager swagger URL, obtain the client token.

     Swagger URL:

     Copy

     `https://api-internal.${env}.mosip.net/v1/authmanager/swagger-ui/index.html?configUrl=/v1/authmanager/v3/api-docs/swagger-config`

     Endpoint:

     Copy

     `/authenticate/clientidsecretkey`

     Request:

     Copy

     {
        "id": "string",
        "version": "string",
        "requesttime": "2022-12-22T07:13:35.010Z",
        "metadata": {},
        "request": {
           "clientId": " mosip-pms-client ",
           "secretKey": " XXXXXX ",
           "appId": " partner "
        }
     }

   • Now using the key manager swagger URL, generate module level certificate.

     Swagger URL:

     Copy

     `https://api-internal.${env}.mosip.net/v1/keymanager/swagger-ui/index.html?configUrl=/v1/keymanager/v3/api-docs/swagger-config`

     Endpoint:

     Copy

     `/generateMasterKey/Certificate`

     Request:
   • Directly download the certificate through key manager swagger URL and getCertificate endpoint, with App Id as COMPLIANCE_TOOLKIT and Ref Id as COMP-FIR.
   • This certificate serves as the encryption key specifically for SBI devices.

     For Mock MDS, when running in Auth mode: update the below values in the application.properties file.

     Copy

     mosip.auth.appid=regproc
     mosip.auth.clientid=mosip-pms-client
     mosip.auth.secretkey=XXXXXXXXXXXXXXXX
     mosip.auth.server.url=https://api-internal.${env}.mosip.net/v1/authmanager/authenticate/clientidsecretkey 
     mosip.ida.server.url=https://api-internal.${env}.mosip.net/v1/keymanager/getCertificate?applicationId=COMPLIANCE_TOOLKIT&referenceId=COMP-FIR

     For real MDS/SBI, the vendors can download the new encryption key from the UI and test with the updated SBI which uses this encryption key. It can be downloaded for Auth SBI projects from UI.
6. Ensure that reporting module is deployed from the develop branch. This is required for the Kibana Dashboard.

7. Ensure that the kernel-auth-adapter-1.2.0.1 has been successfully deployed in the compliance-toolkit-service, and verify that the identical authentication adapter was utilized to configure the mock ABIS.

8. Check datashare configurations for ABIS3030 and ABIS3031 testcases.

   • Ensure that in data-share-default.properties and compliance-toolkit-default.properties, the value of mosip-abis-client is set in auth.server.admin.allowed.audience. If this was not set by default, then set it and restart data-share-service and compliance-toolkit-service.

   • For handle CTK datashare token flow, add the below values in data-share-default.properties.

     Copy

     auth.handle.ctk.flow=true
     mosip.api.internal.toolkit.url=https://${mosip.api.internal.host}/v1/toolkit
     mosip.compliance.toolkit.saveDataShareToken.url=${mosip.api.internal.toolkit.url}/saveDataShareToken
     mosip.compliance.toolkit.invalidateDataShareToken.url=${mosip.api.internal.toolkit.url}/invalidateDataShareToken
     mosip.compliance.toolkit.invalidateDataShareToken.testCaseId=ABIS3031

Steps to load testdata, schemas in MINIO

1. Browse mosip-compliance-toolkit

2. Project structure will be displayed as mentioned below.

3. The resources folder has schemas and test data that need to be added to MinIO.

Steps to load testdata and schemas directly in MinIO

1. Log in to MinIO from the browser.

2. Create a compliance-toolkit bucket.

3. Create a new folder named testdata in the above bucket.Upload MOSIP_DEFAULT_XXX.zip files from resources to it.

4. Create a new folder named schemas in the above bucket. Upload all SBI, SDK and ABIS schemas along with subfolders in it.

5. Upload testcase_schema.json from resources folder to schemas folder.

6. Please restart the compliance pods after adding new files in minio to refresh the cache.

Alternately, swagger endpoint can also be used to upload data in Minio. In this case there is no need to restart CTK services.

Steps to upload resources to Minio using Swagger

1. The swagger url is:

https://{api-internal-env-url}/v1/toolkit/swagger-ui/index.html?configUrl=/v1/toolkit/v3/api-docs/swagger-config

2. Using keycloak/ register option in CTK UI, create a new user for compliance toolkit.

3. Make sure to add the email ID. Also, give the role CTK_ADMIN.

4. Login to compliance toolkit in your environment from browser with the above Keycloak user.

5. Go to ResourceManagementController in swagger and upload the schema alongwith testdata files.

6. Then, select any one of type mentioned above and also mention the version (SBI/SDK/ABIS Version).

7. Select the type of required file which you want to upload from resources folder in project.

8. After execution you can see the response.

9. Check inside the Minio, in dev environment, whether the files have been uploaded into it.

Steps to upload test cases to the Database

1. Open swagger and go to saveTestCases endpoint in test-cases-controller.

2. Currently, CTK has a separate repository called compliance-toolkit-testcases, which includes the testcases for the SBI, SDK and ABIS.

3. Open compliance-toolkit-testcases repo.

4. compliance_test_definitions_sbi.json file have all the test cases in it.

5. Copy test cases array from this file and prepare a request as shown below.

6. Request body for saveTestCases request.

{
   "id": "mosip.toolkit.projects.retrieve",
   "version": "1.0",
   "requesttime": "2022-10-29T06:06:51.174Z",
   "metadata": null,
   "request": {
      "testCases": [
         <Paste the testcases here>
      ]
   }
}

7. Then, execute it.

8. The same should be done for compliance_test_definitions_sdk.json and compliance_test_definitions_abis.json.

Steps to upload terms and conditions velocity templates to the Database

1. Open swagger and go to uploadTemplate endpoint in resource-management-controller.

2. Provide the required parameters:

   • Specify the language for the template using ISO 639 standard language codes.

   • Enter the version of the template. (the template version should be in format as v1,v2,v3 and so on)

   • Enter the template name as terms_and_conditions_template.

3. The template file extension should be.vm(Velocity Template Language).

4. Then , execute it

Steps to generate Android APK

• Once all the steps mentioned above are completed, you can trigger the Android APK build for your environment. https://github.com/mosip/mosip-compliance-toolkit-ui/actions/workflows/android.yml

• You may need GitHub repository write access.

• Add values for the URL’s according to your deployment env.

Pre-requisites

CTK should be deployed with the required dockers.

• compliance-toolkit-service: 1.0.0

• compliance-toolkit-ui: 1.0.0

Dependent Service (dockers)

• Artifactory: mosipid/artifactory-ref-impl: 1.2.0.1-B2

• Audit manager: mosipid/kernel-auditmanager-service: 1.2.0.1-B1

• Auth Manager: mosipid/kernel-authmanager: 1.2.0.1-B1

• Key Manager: modipid/kernel-keymanager-service: 1.2.0.1-B1

• Partner Management: mosipid/partner-management-service: 1.2.0.1-B1

• KeyCloak: mosipid/keycloak-init: 1.2.0.1-B1

• Postgres: mosipid/postgres-init: 1.2.0.1-B1

• Config Server: config-server: mosipid/config-server: 1.1.2

• Notification Service: mosipid/kernel-notification-service: 1.2.0.1-B1

• ClamAV: clamav/clamav: latest

• MinIO

Note:

• Ensure that in the kernel-default.properties, the value of mosip-toolkit-client is set in auth.server.admin.allowed.audience.If this was not set by default, then set it and restart kernel-auth-service and compliance-toolkit-service.

• Check if the roles given to mosip-pms-client should match with any of the roles for following config property.

  This config property is available in: https://github.com/mosip/mosip-config/blob/${ENV_NAME}/kernel-default.properties

  For Example:

  mosip.role.keymanager.postverifycertificatetrust=ZONAL_ADMIN, GLOBAL_ADMIN, PMS_ADMIN, PMS_USER

  Then mosip-pms-client should have any of the above roles.

• Check that mosip-pms-client has the role REGISTRATION_PROCESSOR, PARTNER_ADMIN, PMS_ADMIN in Key Cloak.If this was not set by default, then set it and restart key manager and compliance-toolkit-service.

• From the 1.0.0 version onwards, we need to generate an encryption key for CTK.

  • Create a new app id by directly inserting the below row.
  • Get the client token using auth manager swagger by calling endpoint.

    https://api-internal.dev.mosip.net/v1/authmanager/authenticate/clientidsecretkey
  • Use generateMasterKey endpoint to generate module-level certificate.

  • Directly download the certificate via key manager swagger getCertificate with App Id as COMPLIANCE_TOOLKIT and Ref Id as COMP-FIR.

  • This certificate is to be used by SBI devices as the encryption key.

  • For Mock MDS, when running in Auth mode, update the below values in the application.properties file.
• For REAL MDS/SBI.

  • You must communicate to the vendors to download the new encryption key from UI and give us an updated SBI which uses this encryption key.

  • It can be downloaded for Auth SBI projects from UI.

Steps to load testdata and schemas

2. The resources folder would contain schemas, test data and test cases that need to be added to MinIO and DB.

Steps to upload resources to MinIO

1. Log in to MinIO from the browser.

2. Create a compliance-toolkit bucket.

3. Create a new folder named testdata in the above bucket and upload all test data zip files from the resources folder to this folder.

4. Create a new folder named schemas in the above bucket and upload all sbi and sdk schemas, test case schema from the resources folder to this folder.

Note: There is no need to upload compliance_test_definitions_sbi.json and compliance_test_definitions_sdk.json.

5. Restart the pods after adding new files in MinIO.

Steps to upload resources to MinIO using Swagger

1. Using Keycloak, create a new user for the compliance toolkit.

2. Make sure to add the email ID. Also, give the user GLOBAL_ADMIN.

3. Log in to the compliance toolkit in your environment with above the Keycloak user.

5. Go to uploadResourceFile the endpoint in ResourceManagementController.

6. Select any one of the types which are mentioned in swagger and version (SBI or SDK).

7. Upload the schema and test data files from the resources folder in the project.

8. You can see the uploaded schema and test data files in the MinIO dev environment.

Steps to upload test cases to the Database

2. Open the resources folder in the project.

3. compliance_test_definitions_sbi.json file has all the test cases in it.

4. Copy the test cases array from this file and prepare a request as shown below.

5. Request body for saveTestCases request.

9. The same should be done for compliance_test_definitions_sdk.json.

Pre-requisites

CTK should be deployed with the required dockers.

• compliance-toolkit-service: 1.3.0

• compliance-toolkit-ui: 1.3.0

To deploy Compliance Toolkit, we require the below mandatory services:

Dependent Service (dockers)

• Artifactory: mosipid/artifactory-ref-impl: 1.2.0.1-B2

• Audit manager: mosipid/kernel-auditmanager-service: 1.2.0.1-B1

• Auth Manager: mosipid/kernel-authmanager: 1.2.0.1-B1

• Key Manager: modipid/kernel-keymanager-service: 1.2.0.1-B1

• Partner Management: mosipid/partner-management-service: 1.2.0.1-B1

• KeyCloak: mosipid/keycloak-init: 1.2.0.1-B1

• Postgres: mosipid/postgres-init: 1.2.0.1-B1

• Config Server: config-server: mosipid/config-server: 1.1.2

• Notification Service: mosipid/kernel-notification-service: 1.2.0.1-B1

• ClamAV: clamav/clamav: latest

• MinIO

The Setup guide is a checklist for the three categories below:

1. Configuration checks

2. Steps to load testdata, schemas and testcases

3. Steps to generate Android APK

Configuration checks

1. Ensure that in the kernel-default.properties, the value of mosip-toolkit-client and mosip-toolkit-android-client is set in auth.server.admin.allowed.audience. If this was not set by default, then set it and restart kernel-auth-service and compliance-toolkit-service.

2. Ensure that in compliance-toolkit-default.properties, CORS is enabled to allow access to mosip-toolkit-android-client:

   Copy

    mosip.security.cors-enable=true
    mosip.security.origins=http://localhost
    auth.server.admin.allowed.audience=mosip-toolkit-client,mosip-toolkit-android-client

   If this was not set by default, then set it and restart compliance-toolkit-service.

3. Check if the roles given to mosip-pms-client match with any of the roles for the following config property: mosip.role.keymanager.postverifycertificatetrust=XXX

   This config property is available here.

   For Example:

   mosip.role.keymanager.postverifycertificatetrust=ZONAL_ADMIN, GLOBAL_ADMIN, PMS_ADMIN, PMS_USER

   Then mosip-pms-client should have any of the above roles.

4. Check that mosip-pms-client has the role REGISTRATION_PROCESSOR, PARTNER_ADMIN, PMS_ADMIN in Key Cloak. If this was not set by default, then set it and restart keymanager and compliance-toolkit-service.

5. It is also needed to generate an encryption key for CTK.

   • Create a new app id by directly inserting the below row.

     INSERT INTO keymgr.key_policy_def(app_id, key_validity_duration, is_active,pre_expire_days, access_allowed, cr_by, cr_dtimes, upd_by, upd_dtimes, is_deleted, del_dtimes) VALUES ('COMPLIANCE_TOOLKIT', 1095, true, 60, 'NA', 'mosipadmin', '2022-11-28 09:00:40.822625', null, null, false, null);

   • Using the auth manager swagger URL, obtain the client token.

     Swagger URL:

     Copy

     `https://api-internal.${env}.mosip.net/v1/authmanager/swagger-ui/index.html?configUrl=/v1/authmanager/v3/api-docs/swagger-config`

     Endpoint:

     Copy

     `/authenticate/clientidsecretkey`

     Request:

     Copy

     {
        "id": "string",
        "version": "string",
        "requesttime": "2022-12-22T07:13:35.010Z",
        "metadata": {},
        "request": {
           "clientId": " mosip-pms-client ",
           "secretKey": " XXXXXX ",
           "appId": " partner "
        }
     }

   • Now using the key manager swagger URL, generate module level certificate.

     Swagger URL:

     Copy

     `https://api-internal.${env}.mosip.net/v1/keymanager/swagger-ui/index.html?configUrl=/v1/keymanager/v3/api-docs/swagger-config`

     Endpoint:

     Copy

     `/generateMasterKey/Certificate`

     Request:
   • Directly download the certificate via key manager swagger URL and getCertificate endpoint, with App Id as COMPLIANCE_TOOLKIT and Ref Id as COMP-FIR.
   • This certificate is to be used by SBI devices as the encryption key.

     For Mock MDS, when running in Auth mode: update the below values in the application.properties file.

     Copy

     mosip.auth.appid=regproc
     mosip.auth.clientid=mosip-pms-client
     mosip.auth.secretkey=XXXXXXXXXXXXXXXX
     mosip.auth.server.url=https://api-internal.${env}.mosip.net/v1/authmanager/authenticate/clientidsecretkey 
     mosip.ida.server.url=https://api-internal.${env}.mosip.net/v1/keymanager/getCertificate?applicationId=COMPLIANCE_TOOLKIT&referenceId=COMP-FIR

     For real MDS/SBI, the vendors can download the new encryption key from the UI and test with the updated SBI which uses this encryption key.It can be downloaded for Auth SBI projects from UI.
6. Ensure that reporting module is deployed from the develop branch. This is required for the Kibana Dashboard.

Steps to load testdata, schemas in MINIO

1. Browse mosip-compliance-toolkit

2. Project structure will be as shown below.

3. The resources folder has schemas, test data and testcases that need to be added to MinIO and DB.

Steps to load testdata and schemas directly in MinIO

1. Log in to MinIO from the browser.

2. Create a compliance-toolkit bucket.

3. Create a new folder named testdata in the above bucket.Upload MOSIP_DEFAULT_XXX.zip files from resources to it.

4. Create a new folder named schemas in the above bucket. Upload all SBI, SDK and ABIS schemas along with subfolders in it.

5. Upload testcase_schema.json from resources folder to schemas folder.

6. There is no need to upload compliance_test_definitions_sbi.json ,compliance_test_definitions_sdk.json and compliance_test_definitions_abis.json

7. Please restart the compliance pods after adding new files in minio to refresh the cache.

Alternately, swagger endpoint can also be used to upload data in Minio. In this case there is no need to restart CTK services.

Steps to upload resources to Minio using Swagger

1. The swagger url is:

https://{api-internal-env-url}/v1/toolkit/swagger-ui/index.html?configUrl=/v1/toolkit/v3/api-docs/swagger-config

2. Using keycloak/ register option in CTK UI, create a new user for compliance toolkit.

3. Make sure to add the email ID. Also, give the role CTK_ADMIN.

4. Login to compliance toolkit in your environment from browser with the above Keycloak user.

5. Go to ResourceManagementController in swagger and upload the schema alongwith testdata files.

6. Then, select any one of type mentioned above and also mention the version (SBI/SDK/ABIS Version).

7. Select the type of required file which you want to upload from resources folder in project.

8. After execution you can see the response.

9. Check inside the Minio, in dev environment, whether the files have been uploaded into it.

Steps to upload test cases to the Database

1. Open swagger and go to saveTestCases in test-cases-controller.

2. Open resources folder in project.

3. compliance_test_definitions_sbi.json file have all the test cases in it.

4. Copy test cases array from this file and prepare a request as shown below.

5. Request body for saveTestCases request.

{
   "id": "mosip.toolkit.projects.retrieve",
   "version": "1.0",
   "requesttime": "2022-10-29T06:06:51.174Z",
   "metadata": null,
   "request": {
      "testCases": [
         <Paste the testcases here>
      ]
   }
}

6. Then, execute it.

7. The same should be done for compliance_test_definitions_sdk.json and compliance_test_definitions_abis.json.

Steps to generate Android APK

• Once all the steps mentioned above are completed, you can trigger the Android APK build for your environment. https://github.com/mosip/mosip-compliance-toolkit-ui/actions/workflows/android.yml

• You may need GitHub repository write access.

• Add values for the URL’s according to your deployment env.

Pre-requisites

CTK should be deployed with the required dockers.

• compliance-toolkit-service: 1.2.0

• compliance-toolkit-ui: 1.2.0

To deploy Compliance Toolkit, we require the below mandatory services:

Dependent Service (dockers)

• Artifactory: mosipid/artifactory-ref-impl: 1.2.0.1-B2

• Audit manager: mosipid/kernel-auditmanager-service: 1.2.0.1-B1

• Auth Manager: mosipid/kernel-authmanager: 1.2.0.1-B1

• Key Manager: modipid/kernel-keymanager-service: 1.2.0.1-B1

• Partner Management: mosipid/partner-management-service: 1.2.0.1-B1

• KeyCloak: mosipid/keycloak-init: 1.2.0.1-B1

• Postgres: mosipid/postgres-init: 1.2.0.1-B1

• Config Server: config-server: mosipid/config-server: 1.1.2

• Notification Service: mosipid/kernel-notification-service: 1.2.0.1-B1

• ClamAV: clamav/clamav: latest

• MinIO

The Setup guide is a checklist for the three categories below:

1. Configuration Checks

2. Steps to load testdata, schemas and testcases

3. Steps to generate Android APK

Configuration checks

1. Ensure that in the kernel-default.properties, the value of mosip-toolkit-client and mosip-toolkit-android-client is set in auth.server.admin.allowed.audience. If this was not set by default, then set it and restart kernel-auth-service and compliance-toolkit-service.

2. Ensure that in compliance-toolkit-default.properties, CORS is enabled to allow access to mosip-toolkit-android-client:

   Copy

    mosip.security.cors-enable=true
    mosip.security.origins=http://localhost
    auth.server.admin.allowed.audience=mosip-toolkit-client,mosip-toolkit-android-client

   If this was not set by default, then set it and restart compliance-toolkit-service.

3. Check if the roles given to mosip-pms-client match with any of the roles for the following config property: mosip.role.keymanager.postverifycertificatetrust=XXX

   This config property is available here.

   For Example:

   mosip.role.keymanager.postverifycertificatetrust=ZONAL_ADMIN, GLOBAL_ADMIN, PMS_ADMIN, PMS_USER

   Then mosip-pms-client should have any of the above roles.

4. Check that mosip-pms-client has the role REGISTRATION_PROCESSOR, PARTNER_ADMIN, PMS_ADMIN in Key Cloak. If this was not set by default, then set it and restart keymanager and compliance-toolkit-service.

5. From the CTK v 1.0.0 version onwards, we need to generate an encryption key for CTK.

   • Create a new app id by directly inserting the below row.

     INSERT INTO keymgr.key_policy_def(app_id, key_validity_duration, is_active,pre_expire_days, access_allowed, cr_by, cr_dtimes, upd_by, upd_dtimes, is_deleted, del_dtimes) VALUES ('COMPLIANCE_TOOLKIT', 1095, true, 60, 'NA', 'mosipadmin', '2022-11-28 09:00:40.822625', null, null, false, null);

   • Using the auth manager swagger URL, get the client token.

     Swagger URL:

     Copy

     `https://api-internal.${env}.mosip.net/v1/authmanager/swagger-ui/index.html?configUrl=/v1/authmanager/v3/api-docs/swagger-config`

     Endpoint:

     Copy

     `/authenticate/clientidsecretkey`

     Request:

     Copy

     {
        "id": "string",
        "version": "string",
        "requesttime": "2022-12-22T07:13:35.010Z",
        "metadata": {},
        "request": {
           "clientId": " mosip-pms-client ",
           "secretKey": " XXXXXX ",
           "appId": " partner "
        }
     }

   • Now using the key manager swagger URL, generate module level certificate.

     Swagger URL:

     Copy

     `https://api-internal.${env}.mosip.net/v1/keymanager/swagger-ui/index.html?configUrl=/v1/keymanager/v3/api-docs/swagger-config`

     Endpoint:

     Copy

     `/generateMasterKey/Certificate`

     Request:
   • Directly download the certificate via key manager swagger URL and getCertificate endpoint, with App Id as COMPLIANCE_TOOLKIT and Ref Id as COMP-FIR.
   • This certificate is to be used by SBI devices as the encryption key.

     For Mock MDS, when running in Auth mode: update the below values in the application.properties file.

     Copy

     mosip.auth.appid=regproc
     mosip.auth.clientid=mosip-pms-client
     mosip.auth.secretkey=XXXXXXXXXXXXXXXX
     mosip.auth.server.url=https://api-internal.${env}.mosip.net/v1/authmanager/authenticate/clientidsecretkey 
     mosip.ida.server.url=https://api-internal.${env}.mosip.net/v1/keymanager/getCertificate?applicationId=COMPLIANCE_TOOLKIT&referenceId=COMP-FIR

     For real MDS/SBI, the vendors can download the new encryption key from the UI and test with the updated SBI which uses this encryption key.It can be downloaded for Auth SBI projects from UI.

Steps to load testdata, schemas in MINIO

1. Browse mosip-compliance-toolkit

2. Project structure will be as shown below.

3. The resources folder has schemas, test data and testcases that need to be added to MinIO and DB.

Steps to load testdata and schemas directly in MinIO

1. Log in to MinIO from the browser.

2. Create a compliance-toolkit bucket.

3. Create a new folder named testdata in the above bucket.Upload MOSIP_DEFAULT_XXX.zip files from resources to it.

4. Create a new folder named schemas in the above bucket. Upload all SBI, SDK and ABIS schemas along with subfolders in it.

5. Upload testcase_schema.json from resources folder to `schemas folder.

6. There is no need to upload compliance_test_definitions_sbi.json ,compliance_test_definitions_sdk.json and compliance_test_definitions_abis.json

7. Please restart the compliance pods after adding new files in minio to refresh the cache.

Alernately swagger endpoint can also be used to upload data in Minio. In this case there is no need to restart CTK services.

Steps to upload resources to MinIO using Swagger

1. The swagger url is:

https://{api-internal-env-url}/v1/toolkit/swagger-ui/index.html?configUrl=/v1/toolkit/v3/api-docs/swagger-config

2. Using keycloak/ register option in CTK UI, create a new user for compliance toolkit.

3. Make sure to add the email ID. Also, give the role GLOBAL_ADMIN.

4. Login to compliance toolkit in your environment from browser with the above Keycloak user.

5. Go to ResourceManagementController in swagger and upload the schema alongwith testdata files.

6. Then, select any one of type mentioned above and also mention the version (SBI/SDK/ABIS Version).

7. Select the type of required file which you want to upload from resources folder in project.

8. After execution you can see the response.

9. Check inside the MinIo in dev environment whether the files have been uploaded into it.

Steps to upload test cases to the Database

1. Open swagger and go to saveTestCases in test-cases-controller.

2. Open resources folder in project.

3. compliance_test_definitions_sbi.json file have all the test cases in it.

4. Copy test cases array from this file and prepare a request as shown below.

5. Request body for saveTestCases request.

{
   "id": "mosip.toolkit.projects.retrieve",
   "version": "1.0",
   "requesttime": "2022-10-29T06:06:51.174Z",
   "metadata": null,
   "request": {
      "testCases": [
         <Paste the testcases here>
      ]
   }
}

6. Then, execute it.

7. The same should be done for compliance_test_definitions_sdk.json and compliance_test_definitions_abis.json.

Steps to generate Android APK

• Once all the steps mentioned above are completed, you can trigger the Android APK build for your environment. https://github.com/mosip/mosip-compliance-toolkit-ui/actions/workflows/android.yml

• You may need GitHub repository write access.

• Add values for the URL’s according to your deployment env.

Pre-requisites

CTK should be deployed with the required dockers.

• compliance-toolkit-service: 0.0.9-B1

• compliance-toolkit-ui: 0.0.9-B1

Dependent Service (dockers)

• Artifactory: mosipid/artifactory-ref-impl: 1.2.0.1-B2

• Audit manager: mosipid/kernel-auditmanager-service: 1.2.0.1-B1

• Auth Manager: mosipid/kernel-authmanager: 1.2.0.1-B1

• Key Manager: modipid/kernel-keymanager-service: 1.2.0.1-B1

• Partner Management: mosipid/partner-management-service: 1.2.0.1-B1

• KeyCloak: mosipid/keycloak-init: 1.2.0.1-B1

• Postgres: mosipid/postgres-init: 1.2.0.1-B1

• Config Server: config-server: mosipid/config-server: 1.1.2

• Notification Service: mosipid/kernel-notification-service: 1.2.0.1-B1

• ClamAV: clamav/clamav: latest

• MinIO

Note:

• Ensure that in the kernel-default.properties, the value of mosip-toolkit-client is set in auth.server.admin.allowed.audience. If this was not set by default, then set it and restart kernel-auth-service and compliance-toolkit-service.

• Check if the roles given to mosip-pms-client should match with any of the roles for following config property.

  Copy

      mosip.role.keymanager.postverifycertificatetrust=XXX

  This config property is available in: https://github.com/mosip/mosip-config/blob/${ENV_NAME}/kernel-default.properties

  For Example:

  mosip.role.keymanager.postverifycertificatetrust=ZONAL_ADMIN, GLOBAL_ADMIN, PMS_ADMIN, PMS_USER

  Then mosip-pms-client should have any of the above roles.

Steps to setup mosip-compliance-toolkit

1. Browse to mosip-compliance-toolkit.

2. The resources folder would contain schemas, test data and test cases that need to be added to MinIO and DB.

Steps to upload resources to MinIO

1. Log in to MinIO from the browser.

2. Create a compliance-toolkit bucket.

3. Create a new folder named testdata in the above bucket and upload all test data zip files from the resources folder to this folder.

4. Create a new folder named schemas in the above bucket and upload all sbi and sdk schemas, testcase schema from the resources folder to this folder.

Note: There is no need to upload compliance_test_definitions_sbi.json and compliance_test_definitions_sdk.json.

5. Restart the pods after adding new files in MinIO.

Using Postman

1. Using Keycloak, create a new user for the compliance toolkit.

2. Make sure to add the email ID. Also, give the user GLOBAL_ADMIN.

3. Log in to the compliance toolkit in your environment with above the Keycloak user.

4. Open the developer tools and copy the Authorization token from the headers section under the Networks tab.

5. Add the Authorization token in postman, copy the token and place it in the headers section of the request (Cookie=Authentication:eyAjksa...) and send the request.

Steps to upload test cases to the Database

1. Open postman and create a POST request.

2. URL endpoint https://{base_URL}/v1/toolkit/saveTestCases

3. Copy the Authorization token in the request header as mentioned in the Using Postman section.

4. Open the resources folder in the project.

5. compliance_test_definitions_sbi.json file has all the test cases in it.

6. Copy the test cases array from this file and prepare a request as shown below.

7. Request body for saveTestCases request

{
    "version": "1.0",
    "requesttime": "2022-10-29T06:06:51.174Z",
    "metadata": null,
    "request": {
        "testCases": [
<Paste the testcases here>
                     ]
               }
}

8. Change the requesttime to the current day and send the request.

9. The same should be done for compliance_test_definitions_sdk.json.

Prerequisite

Ensure that all the deployment steps are followed as mentioned in the README.md files of the below repositories:

1. mosip-compliance-toolkit

2. mosip/mosip-compliance-toolki-ui

Additional steps post CTK deployment

Below are details of some additional steps that you may need to follow so as to make CTK publicly available post the regular deployment.

DNS Mapping changes

Update the DNS records for the below mentioned domains to point to the public IP of nginx server associated with the corresponding cluster.

onboarder.sandbox.mosip.net ----> public IP of nginx server for Mosip cluster

sandbox.mosip.net ----> public IP of nginx server for Mosip cluster

pmp.sandbox.mosip.net ----> public IP of nginx server for Mosip cluster

iam.sandbox.mosip.net ----> public IP of nginx server for Observation cluster

Nginx Server changes

Add the below mentioned domains in server_name section of pubic nginx server.

1. sandbox.mosip.net

2. api.sandbox.mosip.net

3. compliance.sandbox.mosip.net

4. pmp.sandbox.mosip.net

Note: Replace “sandbox” appropriately.

Istio changes

1. Update the below mentioned istio ingress gateway to point to public IstioOperator:

   Change spec.selector.istio: ingressgateway-internal to spec.selector.istio: ingressgateway as shown in the image below.

   • pmp-gateaway

   • compliance-toolkit-ui-gateway

   • keycloak

   • landing-page

2. Update below mentioned Istio virtualservice to add public gateway in spec.gateways:

* partnermanager
* policymanager
* toolkit-service

3. Update Istio gateway in compliance toolkit EnvoyFilter compliance-toolkit-set-cookie-header to public gateway.

  workloadSelector:
    labels:
      istio: ingressgateway

Config map changes for UI

• Update compliance-toolkit-ui.json to point to api.sandbox.mosip.net instead of api-internal.sandbox.mosip.net.

• Update pmp config.json to point to api.sandbox.mosip.net instead of api-internal.sandbox.mosip.net.

Configuration changes

• Add mosip.api.external.url=https://${mosip.api.public.host} property in compliance-toolkit-default.properties file.

• Update mosip.iam.module.redirecturi=${mosip.api.external.url}/v1/toolkit/login-redirect/ property in compliance-toolkit-default.properties file.

• Update mosip.iam.module.redirecturi=${mosip.api.external.url}/v1/partnermanager/login-redirect/ property in partner-management-default.properties file.

Pre-requisites

CTK should be deployed with the required dockers.

• compliance-toolkit-service: 1.1.0

• compliance-toolkit-ui: 1.1.0

To deploy Compliance Toolkit, we require only below mandatory services:

Dependent Service (dockers)

• Artifactory: mosipid/artifactory-ref-impl: 1.2.0.1-B2

• Audit manager: mosipid/kernel-auditmanager-service: 1.2.0.1-B1

• Auth Manager: mosipid/kernel-authmanager: 1.2.0.1-B1

• Key Manager: modipid/kernel-keymanager-service: 1.2.0.1-B1

• Partner Management: mosipid/partner-management-service: 1.2.0.1-B1

• KeyCloak: mosipid/keycloak-init: 1.2.0.1-B1

• Postgres: mosipid/postgres-init: 1.2.0.1-B1

• Config Server: config-server: mosipid/config-server: 1.1.2

• Notification Service: mosipid/kernel-notification-service: 1.2.0.1-B1

• ClamAV: clamav/clamav: latest

• MinIO

The Setup guide is a checklist for the three categories below:

1. Configuration Checks

2. Steps to load testdata, schemas and testcases

3. Steps to generate Android APK

Configuration checks

1. Ensure that in the kernel-default.properties, the value of mosip-toolkit-client and mosip-toolkit-android-client is set in auth.server.admin.allowed.audience. If this was not set by default, then set it and restart kernel-auth-service and compliance-toolkit-service.

2. Ensure that in compliance-toolkit-default.properties, CORS is enabled to allow access to mosip-toolkit-android-client:

   Copy

    mosip.security.cors-enable=true
    mosip.security.origins=http://localhost
    auth.server.admin.allowed.audience=mosip-toolkit-client,mosip-toolkit-android-client

   If this was not set by default, then set it and restart compliance-toolkit-service.

3. Check if the roles given to mosip-pms-client match with any of the roles for the following config property: mosip.role.keymanager.postverifycertificatetrust=XXX

   This config property is available here.

   For Example:

   mosip.role.keymanager.postverifycertificatetrust=ZONAL_ADMIN, GLOBAL_ADMIN, PMS_ADMIN, PMS_USER

   Then mosip-pms-client should have any of the above roles.

4. Check that mosip-pms-client has the role REGISTRATION_PROCESSOR, PARTNER_ADMIN, PMS_ADMIN in Key Cloak. If this was not set by default, then set it and restart keymanager and compliance-toolkit-service.

5. From the CTK v 1.0.0 version onwards, we need to generate an encryption key for CTK.

   • Create a new app id by directly inserting the below row.

     INSERT INTO keymgr.key_policy_def(app_id, key_validity_duration, is_active,pre_expire_days, access_allowed, cr_by, cr_dtimes, upd_by, upd_dtimes, is_deleted, del_dtimes) VALUES ('COMPLIANCE_TOOLKIT', 1095, true, 60, 'NA', 'mosipadmin', '2022-11-28 09:00:40.822625', null, null, false, null);

   • Using the auth manager swagger URL, get the client token.

     Swagger URL:

     Copy

     `https://api-internal.${env}.mosip.net/v1/authmanager/swagger-ui/index.html?configUrl=/v1/authmanager/v3/api-docs/swagger-config`

     Endpoint:

     Copy

     `/authenticate/clientidsecretkey`

     Request:

     Copy

     {
        "id": "string",
        "version": "string",
        "requesttime": "2022-12-22T07:13:35.010Z",
        "metadata": {},
        "request": {
           "clientId": " mosip-pms-client ",
           "secretKey": " XXXXXX ",
           "appId": " partner "
        }
     }

   • Now using the key manager swagger URL, generate module level certificate.

     Swagger URL:

     Copy

     `https://api-internal.${env}.mosip.net/v1/keymanager/swagger-ui/index.html?configUrl=/v1/keymanager/v3/api-docs/swagger-config`

     Endpoint:

     Copy

     `/generateMasterKey/Certificate`

     Request:
   • Directly download the certificate via key manager swagger URL and getCertificate endpoint, with App Id as COMPLIANCE_TOOLKIT and Ref Id as COMP-FIR.
   • This certificate is to be used by SBI devices as the encryption key.

     For Mock MDS, when running in Auth mode: update the below values in the application.properties file.

     Copy

     mosip.auth.appid=regproc
     mosip.auth.clientid=mosip-pms-client
     mosip.auth.secretkey=XXXXXXXXXXXXXXXX
     mosip.auth.server.url=https://api-internal.${env}.mosip.net/v1/authmanager/authenticate/clientidsecretkey 
     mosip.ida.server.url=https://api-internal.${env}.mosip.net/v1/keymanager/getCertificate?applicationId=COMPLIANCE_TOOLKIT&referenceId=COMP-FIR

     For real MDS/SBI, the vendors can download the new encryption key from the UI and test with the updated SBI which uses this encryption key.It can be downloaded for Auth SBI projects from UI.

Steps to load testdata, schemas in MINIO

1. Browse mosip-compliance-toolkit

2. Project structure will be as shown below.

3. The resources folder has schemas, test data and testcases that need to be added to MinIO and DB.

Steps to load testdata and schemas directly in MinIO

1. Log in to MinIO from the browser.

2. Create a compliance-toolkit bucket.

3. Create a new folder named testdata in the above bucket.Upload MOSIP_DEFAULT_XXX.zip files from resources to it.

4. Create a new folder named schemas in the above bucket. Upload all SBI and SDK schemas along with subfolders in it.

5. Upload testcase_schema.json from resources folder to `schemas folder.

6. There is no need to upload compliance_test_definitions_sbi.json & compliance_test_definitions_sdk.json

7. Please restart the compliance pods after adding new files in minio to refresh the cache.

Alernately swagger endpoint can also be used to upload data in Minio. In this case there is no need to restart CTK services.

Steps to upload resources to MinIO using Swagger

1. The swagger url is:

https://{api-internal-env-url}/v1/toolkit/swagger-ui/index.html?configUrl=/v1/toolkit/v3/api-docs/swagger-config

2. Using keycloak/ register option in CTK UI create a new user for compliance toolkit.

3. Make sure to add the email ID. Also, give the role GLOBAL_ADMIN.

4. Login to compliance toolkit in your environment from browser with above the Keycloak user.

5. Go to ResourceManagementController in swagger, there you need to upload schema and testdata files.

6. Then you have to choose any one of the types which is mentioned above and also mention the version (SBI/SDK Version).

7. Select the type of required file which you want to upload from resources folder in project.

8. After execution you can see the response.

9. Check inside the MinIo in dev environment whether the files have been uploaded into it.

Steps to upload test cases to the Database

1. Open swagger and go to saveTestCases in test-cases-controller.

2. Open resources folder in project.

3. compliance_test_definitions_sbi.json file have all the test cases in it.

4. Copy test cases array from this file and prepare a request as shown below.

5. Request body for saveTestCases request.

{
   "id": "mosip.toolkit.projects.retrieve",
   "version": "1.0",
   "requesttime": "2022-10-29T06:06:51.174Z",
   "metadata": null,
   "request": {
      "testCases": [
         <Paste the testcases here>
      ]
   }
}

6. Then execute it.

7. Same should be done for compliance_test_definitions_sdk.json.

Steps to generate Android APK

• Once all the steps mentioned above are completed, you can trigger the Android APK build for your env https://github.com/mosip/mosip-compliance-toolkit-ui/actions/workflows/android.yml

• You may need GitHub repository write access.

• Add values for the URL’s according to your deployment env.

A partner can test both their JAR based or docker service based biometric SDKs with MOSIP's Compliance Tool kit. In this document, we have provided the steps that can be followed by the partner to enable the testing of both these solutions.

Test using a SDK JAR

Partners having SDK JARs can test their biometric SDK JARs using Compliance Tool kit, by wrapping their SDKs in MOSIP’s BioSDK Services which provides REST endpoints to interact with the SDK jar.

• Checkout MOSIP’s Bio SDK Services from https://github.com/mosip/biosdk-services.git. Make sure to checkout the code from develop branch.

• Build the code with command mvn clean install -Dgpg.skip.

• After the build is successful, place your SDK jar in the biosdk-services\biosdk-services\lib folder.

• Create a bat file to run biosdk-services.

  Copy

  cd <LOCAL_PATH>\biosdk-services\target
  java -Dloader.path=../lib/<SDK_JAR_NAME> -Dbiosdk_bioapi_impl=io.mosip.mock.sdk.impl.SampleSDK -jar biosdk-services-1.2.1-SNAPSHOT.jar

  Here, the LOCAL_PATH is the installation directory path for the Bio SDK Service and SDK_JAR_NAME is the SDK JAR name.

• Once the Bio SDK Service is running, check if the JAR is working using the Swagger available at URL: http://localhost:9099/biosdk-service/swagger-ui.html

• In Compliance Tool kit, for the SDK project, configure the BASE_URL as: http://localhost:9099/biosdk-service.

Test using docker service

Partners having SDK docker service can test their SDKs using Compliance Tool kit by running their docker service to provide REST API’s to access SDK methods.

Once the docker is deployed and accessible, the partner needs to add the correct URL in the SDK project to access the docker service directly.

Note:The partner may face the CORS issue. To get around this, they can allow the Compliance Tool kit URL in their controllers using @CrossOrigin("<URL>") annotation.

Otherwise, a proxy service can be used, which would redirect all the calls to the docker.

A partner can test their biometric ABIS’s with MOSIP’s Compliance Toolkit. In this document, we have provided the steps that can be followed by the partner to enable the ABIS testing.

The diagram below illustrates the CTK deployment architecture.

Test using a Mock ABIS

Below is a quick demonstration followed by steps to set up ABIS for testing.

• Checkout MOSIP’s Mock Services from https://github.com/mosip/mosip-mock-services. Make sure to checkout the code from develop branch.

• Go to REPO_ROOT/mock-abis.

• For setting ABIS queue configuration, follow the steps below:

  • Step 1: Create registration-processor-abis.json in the resources folder with the below details.

  Copy

  {
      "abis": [{
      "name": "ABIS",
      "host": "",
      "port": "",
      "brokerUrl": "tcp://activemq.${env}.mosip.net:61616",
      "inboundQueueName": "ctk-to-abis",
      "outboundQueueName": "abis-to-ctk",
      "pingInboundQueueName": "ctk-to-abis",
      "pingOutboundQueueName": "abis-to-ctk",
      "userName": "artemis",
      "password": "XXXXXXX",
      "typeOfQueue": "ACTIVEMQ",
      "inboundMessageTTL": 2700
      }]
  }

  • Step 2: Update the following details in application-local.properties.

  Copy

  secret_url=https://api-internal.${env}.mosip.net/v1/authmanager/authenticate/clientidsecretkey
  local.development=true
  abis.bio.encryption=false

  • Step 3: Update the following details in config.properties.

  Copy

  secret_url=https://api-internal.${env}.mosip.net/v1/authmanager/authenticate/clientidsecretkey
  secret_url.clientnId=mosip-regproc-client
  secret_url.id=string
  secret_url.secretKey=XXXXXXXXXXX
  secret_url.appId=regproc
  secret_url.requesttime=2023-03-27T06:12:52.994Z
  secret_url.version=string
  
  analytics.confidence=90
  analytics.internalscore=112
  analytics.key1=value1
  analytics.key2=value2

  • Step 4: Build the code with the command mvn clean install -Dmaven.test.skip=true -Dgpg.skip=true.

• If you are testing with newer queues, then you need to first create them manually in active mq console.

Create Address > ctk-to-abis >  anycast > save
Create Address > abis-to-ctk >  anycast > save

• After build is successful, place your ABIS jar in the \mock-abis folder

• create a bat file to run mock ABIS.

    cd <LOCAL_PATH>\mock-abis
    java -jar -Dlocal.development=true -Dabis.bio.encryption=true -Dspring.profiles.active=local -Dmosip_host=https://$   {env}.mosip.net target/mockabis.jar

Here LOCAL_PATH is the installation folder path for the mosip-mock-service.

• Once the mock ABIS service is running, check if the JAR is working using the Swagger available at URL: http://localhost:8081/v1/mock-abis-service/swagger-ui.html

• In CTK ,for the ABIS project configure the below details:

"Url": "wss://activemq.${env}.mosip.net/ws",
"RequestQueueName": "ctk-to-abis",
"ResponseQueueName": "abis-to-ctk",
"userName": "artemis",
"password": "XXXXXX"