How to add more test cases
Compliance Tool Kit (CTK) is an online portal that can be used by MOSIP partners to test the compliance of their product developed as per the specifications (specs) published by MOSIP. CTK currently supports compliance verification of SBI, SDK and ABIS specifications.
What are the CTK test cases?
To verify the compliance of the partner software with MOSIP specifications, MOSIP has created predefined test cases for the type of specification.
Each test case in CTK runs on a given method of the specs. For example, there will be a test case for the
device
method of SBI specs.Each test case in CTK defines the attributes required to create the request to be sent to the partner application.
Each test case also defines the response expected from the partner application.
Each test case also defines the validators to be run in the response received.
Each validator also performs a predefined check on the response.
If all validations are successful, the test case is considered as passed, otherwise, the test case fails.
Partners can use CTK to run these test cases to verify if their implementation adheres to the MOSIP’s specs or not.
These test cases are defined in JSON format and saved in the CTK database.
Sample SBI test case
This test case is for verifying the check device discovery endpoint of an SBI. This test case is available for both Registration
and Auth
SBI projects and all device type
and sub type
combinations.
Sample SDK test case
This test case is for verifying the quality of face biometrics (if it is above the acceptable threshold). This test case will be available for Face modality
only and for the Quality Check project
.
Sample ABIS test case
This test case is for verifying the insert endpoint of an ABIS. This test case is available for all ABIS projects.
Attributes in a test case
As can be seen from the above samples, few attributes are common across the test cases for SBI, SDK and ABIS while few are optional. Below is the list of each attribute and its use.
Name | Type | Description | Allowed Values |
---|---|---|---|
| Required | Type of test case | SBI / SDK / ABIS |
| Required | Name of test case | |
| Required | Unique ID for test case | SBI10XX, SDK20XX |
| Required | Spec version being tested | 0.9.5/1.0.0 for SBI, 0.9.0 for SDK |
| Required | Test description. It can also include the steps to execute the test case. | |
| Required | Indicates if the validator is for a positive or a negative scenario. | E.g.: for a bad face quality test, we expect a low score which is a negative scenario, so the validator should mark the test case as passed on receiving low score |
| Optional | Indicates that the test case is not active. | E.g.: The testcase SDK2026 is marked as inactive. So the user is unable to obtain this testcase. |
| Optional | Indicates that the test case is not active in CTK Android App. | E.g.: The testcase SBI1070 is marked as inactiveForAndroid. So the user is unable to obtain this testcase on android app. |
| Required | The name of the method to be invoked for the test case. It accepts an array. For SBI, this array will have only one value. For SDK, in case of a combination test case you can give 2 method names. E.g.: ["extract-template", "check-quality"] | device(SBI), info(SBI), capture(SBI), rcapture(SBI), stream(SBI), insert(ABIS), identify(ABIS), delete(ABIS), init(SDK), check-quality(SDK), match(SDK), extract-template(SDK), convert-format(SDK), segment(SDK) |
| Required | Name of the JSON schema file which will be used to validate the HTTP request. This HTTP request will be used to invoke the HTTP method defined in the spec. It accepts an array. For SBI, this array will have only one value. For SDK, in case of a combination test case you can give 2 request schema names. E.g: ["ExtractTemplateRequestSchema", "MatchRequestSchema"] | The request schema JSON files are saved in the MINIO in the following folder structure. E.g.: compliance-toolkit/schemas/sbi/0.9.5/ DiscoverRequestSchema.json |
| Required | Name of the JSON schema file which will be used to validate the HTTP response. This is the HTTP response that we will receive after invoking the HTTP method. It accepts an array. For SBI, this array will have only one value For SDK, in case of a combination test case you can give 2 request schema names. E.g.: ["ExtractTemplateResponseSchema", "MatchResponseSchema"] | The response schema JSON files are saved in the MINIO in the following folder structure. E.g.: compliance-toolkit/schemas/sbi/0.9.5/ DiscoverResponseSchema.json |
| Required | Names of Validators that are to be invoked to run various validations on the response. It accepts an array of arrays. Each array is a list of validators to be applied to the response of the corresponding method in the test case. In the case of SBI, it is an array with a single array. In the case of SDK, for a combination test case it can be an array with a list of arrays. | Validators are available in the folder: |
| Required | Name of the Java class which implements the | Must implement method
|
| Required | Description of the validations performed. These are shown in UI along with the test case description. | |
| Optional | Extra attributes about a test case | |
| SBI | Valid only for SBI test case. It accepts an array. The project purpose is used to filter the test cases with matching purposes. | Registration, Auth |
| SBI | Valid only for SBI test case.
The project device type is used to filter the test cases with matching | Finger, Iris, Face |
| SBI | Valid only for SBI test case.
The project device type is used to filter the test cases with matching | Slap, Single, Touchless, Single, Double, Full face |
| SBI | Valid only for SBI test case. It accepts an array. This array is used to populate “bioSubType” attribute in the request sent to the SBI method. Before the values are sent they are mapped to corresponding values. E.g.: RightIndex will be mapped to Right IndexFinger. | LeftIndex, LeftMiddle, LeftRing, LeftLittle, LeftThumb, RightIndex, RightMiddle, RightRing, RightLittle, RightThumb, UNKNOWN, Left, Right |
| SBI | Valid only for SBI test case.
It accepts an array.
This array is used to populate the | LeftIndex, LeftMiddle, LeftRing, LeftLittle, LeftThumb, RightIndex, RightMiddle, RightRing, RightLittle, RightThumb, UNKNOWN, Left, Right |
| SBI | Valid only for SBI test case.
It accepts a string or null.
This value is used to populate | |
| SBI | Valid only for SBI test case.
It accepts a string or null.
This value is used to populate the | |
| SBI | Valid only for SBI test case.
It accepts a string or null. This value is used to populate the | |
| SBI | Valid only for SBI test case. It accepts a string or null. This value is used to populate the “timeout” attribute in the request sent to the SBI method. | |
| SBI | Valid only for SBI test case. It accepts a string or null. This value is used to display a Resume button in UI before the SBI method is invoked. This pauses the test run and makes it possible for the user to make some changes in the SBI before the test case is executed. Helps in Device Status test cases. | |
| SBI | Valid only for SBI test case. It accepts a string or null. This value is used to display a Resume Again button in UI after the SBI method is invoked. This pauses the test run and makes it possible for the user to make some changes in the SBI after the test case is executed. Helps in Device Status test cases. | |
| SBI | Valid only for SBI test case. This value is used to determine if the test case is hash-validation testcase or not. The application will perform hash validations for different captures if the testcase is a hash validation testcase. | |
| SBI | Valid only for SBI test case. It accepts a string. This value is used to populate the | |
| SBI | It accepts a string. This value is used to populate the invalid attribute in the request sent to the SBI method. | |
| SBI | Valid only for SBI test case. This value is used to determine if the test case is quality assessment testcase or not. It's used for creating quality assessment collection. | |
| SBI | Valid only for SBI test case. It accepts a string. This is used to define the different age groups, and it will also be available only for quality assessment test cases. | |
| SBI | Valid only for SBI test case. It accepts a string. This is used to define the different gender, and it will also be available only for quality assessment test cases. | |
| SBI | Valid only for SBI test case. It accepts a string. This is used to define the different occupations, and it will also be available only for quality assessment test cases. | |
| SBI | Valid only for SBI test case. It accepts a string. This is used to define the different races, and it will also be available only for quality assessment test cases. | |
| SBI | Valid only for SBI test case. It accepts a string. This value is used to repeat the testcase based on the count. | |
| SDK | This is applicable only for SDK testcases. It accepts an array. | finger, face, iris |
| SDK | Valid only for SDK test case. It accepts an array. The project purpose is used to filter the test cases with matching sdkPurpose. | Check Quality, Matcher, Extract Template, Convert Format, Segment |
| SDK | Valid only for SDK test case.
It accepts a string.
For | ISO19794_4_2011, ISO19794_5_2011, ISO19794_6_2011 |
| SDK | Valid only for SDK test case.
It accepts a string.
For | IMAGE/JPEG, IMAGE/PNG, ISO19794_4_2011/JPEG, ISO19794_5_2011/JPEG, ISO19794_4_2011/PNG, ISO19794_5_2011/PNG, ISO19794_6_2011/PNG |
| ABIS | Valid only for ABIS test case. This value is used to check the bulk insert condition. If we add more than one person's record, then it will be true. | |
| ABIS | Valid only for ABIS test case. It accepts a string. This value is used to determine how many times the insert should happen. | |
| ABIS | Valid only for ABIS test case. It accepts a string. This value is used to determine if the given reference ID already exists or not. | |
| ABIS | Valid only for ABIS test case. It accepts a string. It is used to identify the duplicant count of a given reference ID. | |
| ABIS | Valid only for ABIS test case. It accepts a string. It is used to find the duplicant for the reference ID against the given gallery ID. | |
| ABIS | Valid only for ABIS test case. It accepts a string. It is used to define the expected duplicant count. | |
| ABIS | Valid only for ABIS test case. It accepts a string. It is used to define the expected failure reason. |
How to add a test case to the Database
Any new test case is to be uploaded to the database. Editing the testcases is not possible, the only actions you can take are to activate or deactivate them using the same steps.
1. Open postman and create a POST request.
2. URL endpoint https://{base_URL}/v1/toolkit/saveTestCases
3. Copy the Authorization
token in the request header by logging into the Compliance Toolkit in your env with a user having a CTK_ADMIN
role. Open the developer tools and copy the Authorization token from the headers section under the Networks
tab. Add the Authorization token in postman, copy the token and place it in the headers section of the request (Cookie=Authorization:eyAjksa...
)
4. Copy the test cases array JSON and prepare a request as shown below.
5. Request body for saveTestCases request
6. Change the requesttime
to the current day and send the request.
Where can you find the existing CTK test cases?
In the location below, you can find all the existing test cases: https://github.com/mosip/compliance-toolkit-testcases/tree/release-1.4.0
compliance_test_definitions_sbi.json
: has all existing SBI test casescompliance_test_definitions_sdk.json
: has all existing SDK test casescompliance_test_definitions_abis.json
: has all existing ABIS test cases
Last updated