How to connect to Yandex Cloud DBaaS (CH, Postgres)?

[m1ker1n]

Hi! Can you help me please with connection to yandex cloud dbaas (clickhouse & postgres)? I dont really understand, how I should connect to it. Well, I know credentials, but I don't understand how to provide certificates.

Where do I need to get certificates for clickhouse and postgres from? I tried to download RootCA.pem and convert it with

openssl x509 -outform der -in RootCA.pem -out ca.crt

and pass it in my-values.yaml, but it didn't work

[MikailBag]

Hello!

convert it with

This step should not be necessary, PEM format is exactly what you need.

but it didn't work

Could you please describe, what exactly went wrong?

As far as I know, currently there is only one place where configuring custom CA is tricky - when running ClickHouse migrations.
As for me, I do it this way (I'm running migrations using chart, so I'm using experimental databases.*.migrations feature):

      extraMounts:
      - name: yc-ca
        mountPath: /etc/pki/tls/certs/yandex-cloud.pem
        subPath: ca.crt
        configMap: yc-ca
        readOnly: true

Effectively I'm copying CA certificate to a special location so that it's trusted systemwide.
We plan on improving this, but it seems that some upstream work is necessary :(

[m1ker1n]

My config:

databases:
  postgresql:
    endpoints:
      - host: "xxxx"
        port: 6432
    db: "perforator"
    user: "perforator"
    password: "xxxx"
  clickhouse:
    replicas:
      - "xxx:8443"
    db: "perforator"
    user: "perforator"
    password: "xxxx"
    ca_cert_path: /home/bidder/perforator/RootCA.pem
  s3:
    buckets:
      # If buckets were created with recommended names
      profiles: "perforator-profile"
      binaries: "perforator-binary"
      taskResults: "perforator-task-results"
      binariesGSYM: "perforator-binary-gsym"
    endpoint: "http://xxx:9000"
    accessKey: "xxx"
    secretKey: "xxx"

I got this error in storage service:

Error: failed to create storage bundle: failed to init dbs: failed to init clickhouse conn: open /home/bidder/perforator/RootCA.pem: no such file or directory

[m1ker1n]

Ok. I solved it this way:

1. Created configmap with RootCA.pem

kubectl create configmap rootca-pem --from-file=RootCA.pem -n perforator

2. Added to values.yml

storage:
 mounts:
   extra:
     - name: rootca-pem
       configMap: rootca-pem
       mountPath: /etc/RootCA.pem
       subPath: RootCA.pem
       readOnly: true
proxy:
  mounts:
   extra:
     - name: rootca-pem
       configMap: rootca-pem
       mountPath: /etc/RootCA.pem
       subPath: RootCA.pem
       readOnly: true

Now pods are running and don't crush, so I need to apply migrations and it should work then

[m1ker1n]

And how do I migrate postgres and clickhouse?
Do I need to enter storage pod and execute scripts from the docs https://perforator.tech/docs/en/guides/migrate-schema
?

[MikailBag]

That instruction assumes that you have Perforator sources checked out somewhere. Checking out perforator on the Storage pod is possible, but problematic (e.g. ./ya run will trigger full build which takes a while). On the other hand, migration tool is already packaged as a container image (https://github.com/yandex/perforator/pkgs/container/perforator%2Fmigrate), so running a one-off pod should work.

There is also experimental migration support directly in chart (you can see details in base values.yaml, e.g .

perforator/perforator/deploy/kubernetes/helm/perforator/values.yaml

Line 34 in f93a61f

enabled: false

)