All pages
Powered by GitBook
1 of 3

Loading...

Loading...

Loading...

Postgres DB

Overview

MOSIP uses Postgres DB for all relational data storage. The DB creation SQL scripts are located under /db_scripts the folder of the module repository. In sandbox deployment, Postgres is installed as a docker inside the cluster. However, in production deployment, typically, Postgres will be installed external to the cluster.

ER diagrams

Entity relationships diagrams for all databases used in MOSIP are given below.

Configuration parameters

  • Connection details

    • {module_name}_database_url

    • {module_name}_database_username

Production DB configuration

These are some of the reference settings of a production database. It is expected that these are reviewed and finalized for a given deployment.

mosip_ida

  • mosip_idmap

  • mosip_idrepo

  • mosip_kernel

  • mosip_keymgr

  • mosip_master

  • mosip_pms

  • mosip_prereg

  • mosip_regdevice

  • mosip_regprc

  • {module_name}_database_password

  • Hibernate configurations

    • javax.persistence.jdbc.driver

    • hibernate.dialect

    • hibernate.jdbc.lob.non_contextual_creation

    • hibernate.hbm2ddl.auto

    • hibernate.show_sql

    • hibernate.format_sql

    • hibernate.connection.charSet

    • hibernate.cache.use_second_level_cache

    • hibernate.cache.use_query_cache

    • hibernate.cache.use_structured_entries

    • hibernate.generate_statistics

    • logging.level.org.hibernate.SQL

    • logging.level.org.hibernate.type

  • mosip_audit
    mosip_authdevice
    mosip_credential
    mosip_hotlist
    resources: 
      limits: {}
      #   cpu: 250m
      #   memory: 1Gi
      requests: 
        cpu: 8000m
        memory: 32000Mi
    
    postgresqlExtendedConf:                 
       wal_level: logical
       max_wal_senders: 20
       max_replication_slots: 10
       shared_buffers: 16GB                         
       max_prepared_transactions: 1000
       huge_pages: try                              
       work_mem: 16MB                       # min 64kB
       maintenance_work_mem: 3GB            # min 1MB
       effective_cache_size: 32GB                   
       log_min_duration_statement: 1000

    Persistence

    1. Postgres DB

    2. Object Store

    Object Store

    Overview

    Object Store is a storage module for MOSIP named as Khazana. The module is an abstraction of storage layer used across Registration Client, Packet Manager, Datashare or Durian for packets and biometric data.

    Khazana provides following adapters to store objects

    1. POSIX - Supports storage of packets on a filesystem. Its typically used by registration client to store packets locally on the machine. This adapter is not receommended for usage in low latency environments like packet manager.

    2. S3 - is one of the well known API for object stores. is used in Khazana to support any S3 compliant object storage solutions.

    3. Swift - An object storage system from . Swift has published API's and Khazana supports the same on experimental basis using . This works with any Swift or Swift compliant object storage solutions.

    Object Store is used for following purpose wihin mosip

    1. Registration Client - Encrypted packets

    2. Pre-registration - Uploaded Documents

    3. Idrepo - Individual's biometrics and documents

    4. Datashare - On demand individual's biometrics, documents and other information.

    As part of our sandbox deployment we have provided an example use case with for on-prem deployment and AWS S3 with AWS deployment. Object Store is installed as part of .

    Note: Please note its important to choose the right partner for object storage and work with them to scale acordingly. Please follow the hardware estimate for Object Store based on respective Object store products.

    The below is the list of S3 Java API's used by MOSIP. This can be used to understand the vendor compatibility. Khazana does not use any internal business logic and is purely an storage abstraction layer.

    Java API Used by MOSIP
    S3 Documentation URL

    getObjectMetadata().getUserMetadata()

    addUserMetadata(m.getKey(), m.getValue())

    PutObjectRequest(bucketName, finalObjectName, s3Object.getObjectContent(), objectMetadata)

    getRequestClientOptions()

    setReadLimit(readlimit)

    putObject(putObjectRequest)

    deleteObject(bucketName, objectName)

    listObjects(account, searchPattern)

    getObjectSummaries()

    listObjects(searchPattern)

    getObjectSummaries()

    doesObjectExist(bucketName, finalObjectName)

    GetObjectTaggingRequest(bucketName,finalObjectName)

    getObjectTagging(getObjectTaggingRequest)

    SetObjectTaggingRequest(bucketName,finalObjectName,objectTagging)

    setObjectTagging(setObjectTaggingRequest)

    getConnection(bucketName).getObject(bucketName, finalObjectName)

    public S3Object getObject(String bucketName, String key)

    getConnection(bucketName).getObjectMetadata(bucketName, finalObjectName)

    public ObjectMetadata getObjectMetadata(String bucketName, String key)

    doesBucketExistV2(bucketName)

    public boolean doesBucketExistV2(String bucketName)

    createBucket(bucketName)

    public Bucket createBucket(String bucketName)

    getObjectMetadata()

    public ObjectMetadata getObjectMetadata()

    S3
    AWS Java S3 Client
    Openstack
    JOSS SDK
    minio
    default sandbox deployment
    public Map<String,String> getUserMetadata()
    public void addUserMetadata(String key, String value)
    public PutObjectRequest(String bucketName, String key, InputStream input, ObjectMetadata metadata)
    public RequestClientOptions getRequestClientOptions()
    public final void setReadLimit(int readLimit)
    public PutObjectResult putObject(PutObjectRequest putObjectRequest)
    public void deleteObject(String bucketName, String key)
    public ObjectListing listObjects(String bucketName, String prefix)
    public List getObjectSummaries()
    public ObjectListing listObjects(String bucketName)
    public List getObjectSummaries()
    boolean doesObjectExist(String bucketName, String objectName)
    public GetObjectTaggingRequest(String bucketName, String key)
    GetObjectTaggingResult getObjectTagging(GetObjectTaggingRequest getObjectTaggingRequest)
    public SetObjectTaggingRequest(String bucketName, String key, ObjectTagging tagging)
    SetObjectTaggingResult setObjectTagging(SetObjectTaggingRequest setObjectTaggingRequest)