Support podAnnotations

[DanLindeman]

Environment

• OS: N/A
• Erlang/OTP: N/A
• EMQ: N/A

Description

The helm chart could allow users to annotate the individual pods by exposing a podAnnotations stanza in values.yaml which would be consumed by the StatefulSet definition. This is done in other popular charts (for example, timescaledb), and would allow users to leverage annotations. For context, my personal use-case is for adding datadog autodiscovery annotations -- which this feature would enable.

Fix could be implemented by adding the following here:

{{- if .Values.podAnnotations }}
      annotations:
{{ toYaml .Values.podAnnotations | indent 8 }}
{{- end }}

Update: the solution will likely have to work in tandem with the support for recreatePods -- so it may end up being more like the following:

      {{- if or .Values.recreatePods .Values.podAnnotations }}
      annotations:
      {{- end }}
      {{- if .Values.recreatePods }}
        checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum | quote }}
      {{- end }}
{{- if .Values.podAnnotations }}
{{ toYaml .Values.podAnnotations | indent 8 }}
{{- end }}