Improvements v2

[tushar5526]

Fix:

• Vault needed for UCI (to be added in docker compose) Fix local setup of UCI #34
• Run prisma migrations in UCI for fresh installs Add Dockerfile uci-apis#60
• No Dockerfile in uci-api-v2 Add Dockerfile uci-apis#60
• Fix networks in docker compose for services. Fix local setup of UCI #34
• Fix UCI admin Dockerfile Use env vars in Dockerfile uci-admin#18
• Document manual steps Fix local setup of UCI #34
• Add missing env vars Fix local setup of UCI #34
• Add more details in manual steps or automate them in install script

Enhancements

• Identify and un-expose ports of service that are not required over the internet Removed unnecessarily exposed ports #49
• Fusionauth forms-db-gql admin key was written directly in compose Refactor Fusionauth Forms-DB-GQL Admin Key Configuration #38
• Some data in docker volumes and some in folder (everything should be in folders to prevent accidental removal of prod data volumes) Using bind mounts instead of docker volumes #43
• FusionAuth Application Setup using Kickstart
• It seems only one redis is being used, check this and remove extra caches.
• https://github.com/samagra-comms/transformer/blob/master/src/main/resources/application.properties - make doubtnut optional
• Some secrets are hardcoded in docker-compose. Have to be moved to env.
• Rename services for better understanding. Cache should not be named as db. Service owners can clarify here.
• Cassandra password is not being picked from .env
• Ask for github token that has read access to packages in install script after cloning uci-apis Ask for github token that has read access to packages in install script after cloning uci-apis #47
• Minio needed for UCI (to be added in docker compose) https://github.com/Samagra-Development/minio-oauth2-docs
• Automatically change ODK password
• env_file is mapped to .env directly and all secrets available in each service. .env refactoring and clean up  #33
• Prefill the adapters, conversation-logic, bot tables so there is a bot out of the box.

Good to have

• Fix the VAULT_ variable name - Fix variable name utils#34
• Using sample.env and generating a .env using some script. installv2.sh should generate populate .env with random secrets #22
• AKHQ username password is written directly to compose. Also test it so that its not available to internet. AKHQ username password is written directly to compose. Also test it so that its not available to internet. #37
• Optimise dockerfiles
• Add docker-compose-swarm.yml and db-compose from Dockerswarm servers for users who want to deploy it as a swarm. stateless and stateful services are already segregated.

[ChakshuGautam]

Actual env files that are required by every service

1. Orchestrator
2. Transformer

[karntrehan]

Anvansh to pick up enhancements.

[RyanWalker277]

Marking the enhancements for which I have created a PR here. Still unable to link issues and PRs to the original checklist.

• Identify and un-expose ports of service that are not required over the internet Removed unnecessarily exposed ports #49
• Fusionauth forms-db-gql admin key was written directly in compose Refactor Fusionauth Forms-DB-GQL Admin Key Configuration #38
• Some data in docker volumes and some in folder (everything should be in folders to prevent accidental removal of prod data volumes) Using bind mounts instead of docker volumes #43
• It seems only one redis is being used, check this and remove extra caches. Removed unused redis service #45 ( on hold )
• Minio needed for UCI (to be added in docker compose) Fusionauth minio setup #53
• Cassandra password is not being picked from .env Changed Cassandra image to bitnami #54
• Ask for github token that has read access to packages in install script after cloning uci-apis Added Github token login in manual script #55
• env_file is mapped to .env directly and all secrets available in each service. Refactored .env for some services #46
• AKHQ username password is written directly to compose. Also test it so that its not available to internet. Update AKHQ username and password to be taken from .env #39

[susheelvar]

@tushar5526 to review these PRs and merging.

[susheelvar]

Previous PRs merged by @tushar5526

@RyanWalker277 to raise 1 PR today.

[RyanWalker277]

Previous PRs merged by @tushar5526

@RyanWalker277 to raise 1 PR today.

Raised the PR yesterday, synced dev with main and also completed work on some existing PRs and got them merged. Currently working on #58, since I am not that familiar with ODK, it might take a while to figure out the exact cause of the issue, will talk to @pankajjangid05 to figure out exact issue. Had a talk with yesterday with @tushar5526 and @chinmoy12c regarding the same.

[RyanWalker277]

There are 3 containers related to ODK, namely
aggregate-server : a tomcat container running odk
aggregate-db : postgres image
wait-for-db : waits for db to be up

The the aggregate-db container works fine. I can log in with the same creds as given in .env

Unable to replicate the issue at my end till now. Will look a bit deeper to replicate and solve this . Also, reached out to @pankajjangid05 on discord yesterday. Awaiting a response as of now

cc- @karntrehan @susheelvar

[RyanWalker277]

Had a talk with @tushar5526, the issue is with the aggregate-server, admin creds are not picked up from .env. Looking into it. Will send a patch soon.

[RyanWalker277]

PR raised on odk (samagra-comms/odk#6) for #58.

[susheelvar]

@tushar5526 to verify PR.

[karntrehan]

Improvements are almost closed. @RyanWalker277 to present macro wins with these changes.