1. Postgres DB

2. Object Store

Overview

Object Store is a storage module for MOSIP named as Khazana. The module is an abstraction of storage layer used across Registration Client, Packet Manager, Datashare or Durian for packets and biometric data.

Khazana provides following adapters to store objects

1. POSIX - Supports storage of packets on a filesystem. Its typically used by registration client to store packets locally on the machine. This adapter is not receommended for usage in low latency environments like packet manager.

2. S3 - S3 is one of the well known API for object stores. AWS Java S3 Client is used in Khazana to support any S3 compliant object storage solutions.

3. Swift - An object storage system from Openstack. Swift has published API's and Khazana supports the same on experimental basis using JOSS SDK. This works with any Swift or Swift compliant object storage solutions.

Object Store is used for following purpose wihin mosip

1. Registration Client - Encrypted packets

2. Pre-registration - Uploaded Documents

3. Idrepo - Individual's biometrics and documents

4. Datashare - On demand individual's biometrics, documents and other information.

As part of our sandbox deployment we have provided an example use case with minio for on-prem deployment and AWS S3 with AWS deployment. Object Store is installed as part of default sandbox deployment.

Note: Please note its important to choose the right partner for object storage and work with them to scale acordingly. Please follow the hardware estimate for Object Store based on respective Object store products.

The below is the list of S3 Java API's used by MOSIP. This can be used to understand the vendor compatibility. Khazana does not use any internal business logic and is purely an storage abstraction layer.

Overview

MOSIP uses Postgres DB for all relational data storage. The DB creation SQL scripts are located under /db_scripts the folder of the module repository. In sandbox deployment, Postgres is installed as a docker inside the cluster. However, in production deployment, typically, Postgres will be installed external to the cluster.

ER diagrams

Entity relationships diagrams for all databases used in MOSIP are given below.

• mosip_audit

• mosip_authdevice

• mosip_credential

• mosip_hotlist

• mosip_ida

• mosip_idmap

• mosip_idrepo

• mosip_kernel

• mosip_keymgr

• mosip_master

• mosip_pms

• mosip_prereg

• mosip_regdevice

• mosip_regprc

Configuration parameters

• Connection details

  • {module_name}_database_url

  • {module_name}_database_username

  • {module_name}_database_password

• Hibernate configurations

  • javax.persistence.jdbc.driver

  • hibernate.dialect

  • hibernate.jdbc.lob.non_contextual_creation

  • hibernate.hbm2ddl.auto

  • hibernate.show_sql

  • hibernate.format_sql

  • hibernate.connection.charSet

  • hibernate.cache.use_second_level_cache

  • hibernate.cache.use_query_cache

  • hibernate.cache.use_structured_entries

  • hibernate.generate_statistics

  • logging.level.org.hibernate.SQL

  • logging.level.org.hibernate.type

Production DB configuration

These are some of the reference settings of a production database. It is expected that these are reviewed and finalized for a given deployment.

resources: 
  limits: {}
  #   cpu: 250m
  #   memory: 1Gi
  requests: 
    cpu: 8000m
    memory: 32000Mi

postgresqlExtendedConf:                 
   wal_level: logical
   max_wal_senders: 20
   max_replication_slots: 10
   shared_buffers: 16GB                         
   max_prepared_transactions: 1000
   huge_pages: try                              
   work_mem: 16MB                       # min 64kB
   maintenance_work_mem: 3GB            # min 1MB
   effective_cache_size: 32GB                   
   log_min_duration_statement: 1000