Настройка мониторинга

Docker

Для включения метрик prometheus TeamStorm необходимо установить параметр PROM_METRICS_ENABLED: true в файле .env. В prometheus необходимо создать ScrapeConfig по каждому сервису. Например для kube-prometheus-stack

apiVersion: monitoring.coreos.com/v1alpha1
kind: ScrapeConfig
metadata:
  name: workspace
  namespace: monitoring
  labels:
    prometheus: system-monitoring-prometheus
spec:
  scrapeInterval: 30s
  staticConfigs:
    - targets:
        - mysrv.example.com
      labels:
        job: workspace
        instance: mysrv.example.com
  metricsPath: /admin/metrics
  tlsConfig:
    insecureSkipVerify: false
  scheme: HTTPS

---
apiVersion: monitoring.coreos.com/v1alpha1
kind: ScrapeConfig
metadata:
  name: template
  namespace: monitoring
  labels:
    prometheus: system-monitoring-prometheus
spec:
  scrapeInterval: 30s
  staticConfigs:
    - targets:
        - mysrv.example.com
      labels:
        job: template
        instance: mysrv.example.com
  metricsPath: /template/metrics
  tlsConfig:
    insecureSkipVerify: false
  scheme: HTTPS

---
apiVersion: monitoring.coreos.com/v1alpha1
kind: ScrapeConfig
metadata:
  name: automation
  namespace: monitoring
  labels:
    prometheus: system-monitoring-prometheus
    app: kube-prometheus-stack
spec:
  scrapeInterval: 30s
  staticConfigs:
    - targets:
        - mysrv.example.com
      labels:
        job: automation
        instance: mysrv.example.com
  metricsPath: /automation/metrics
  tlsConfig:
    insecureSkipVerify: false 
  scheme: HTTPS

---
apiVersion: monitoring.coreos.com/v1alpha1
kind: ScrapeConfig
metadata:
  name: gitintegration
  namespace: monitoring
  labels:
    prometheus: system-monitoring-prometheus
    release: kube-prometheus-stack
    app: kube-prometheus-stack
spec:
  scrapeInterval: 30s
  staticConfigs:
    - targets:
        - mysrv.example.com
      labels:
        job: gitintegration
        instance: mysrv.example.com
  metricsPath: /vcs/metrics
  tlsConfig:
    insecureSkipVerify: false 
  scheme: HTTPS

---
apiVersion: monitoring.coreos.com/v1alpha1
kind: ScrapeConfig
metadata:
  name: history
  namespace: monitoring
  labels:
    prometheus: system-monitoring-prometheus
spec:
  scrapeInterval: 30s
  staticConfigs:
    - targets:
        - mysrv.example.com
      labels:
        job: history
        instance: mysrv.example.com
  metricsPath: /history/metrics
  tlsConfig:
    insecureSkipVerify: true 
  scheme: HTTPS


---
apiVersion: monitoring.coreos.com/v1alpha1
kind: ScrapeConfig
metadata:
  name: webhook
  namespace: monitoring
  labels:
    prometheus: system-monitoring-prometheus
spec:
  scrapeInterval: 30s
  staticConfigs:
    - targets:
        - mysrv.example.com
      labels:
        job: webhook
        instance: mysrv.example.com
  metricsPath: /webhook/metrics
  tlsConfig:
    insecureSkipVerify: false 
  scheme: HTTPS

---
apiVersion: monitoring.coreos.com/v1alpha1
kind: ScrapeConfig
metadata:
  name: comment
  namespace: monitoring
  labels:
    prometheus: system-monitoring-prometheus
spec:
  scrapeInterval: 30s
  staticConfigs:
    - targets:
        - mysrv.example.com
      labels:
        job: comment
        instance: mysrv.example.com
  metricsPath: /comments/metrics
  tlsConfig:
    insecureSkipVerify: false 
  scheme: HTTPS

---
apiVersion: monitoring.coreos.com/v1alpha1
kind: ScrapeConfig
metadata:
  name: notification
  namespace: monitoring
  labels:
    prometheus: system-monitoring-prometheus
spec:
  scrapeInterval: 30s
  staticConfigs:
    - targets:
        - mysrv.example.com
      labels:
        job: notification
        instance: mysrv.example.com
  metricsPath: /notifications/metrics
  tlsConfig:
    insecureSkipVerify: false 
  scheme: HTTPS

---
apiVersion: monitoring.coreos.com/v1alpha1
kind: ScrapeConfig
metadata:
  name: attachment
  namespace: monitoring
  labels:
    prometheus: system-monitoring-prometheus
spec:
  scrapeInterval: 30s
  staticConfigs:
    - targets:
        - mysrv.example.com
      labels:
        job: attachment
        instance: mysrv.example.com
  metricsPath: /attachments/metrics
  tlsConfig:
    insecureSkipVerify: false 
  scheme: HTTPS

---
apiVersion: monitoring.coreos.com/v1alpha1
kind: ScrapeConfig
metadata:
  name: workitem
  namespace: monitoring
  labels:
    prometheus: system-monitoring-prometheus
spec:
  scrapeInterval: 30s
  staticConfigs:
    - targets:
        - mysrv.example.com
      labels:
        job: workitem
        instance: mysrv.example.com
  metricsPath: /tasks/metrics
  tlsConfig:
    insecureSkipVerify: false 
  scheme: HTTPS

---
apiVersion: monitoring.coreos.com/v1alpha1
kind: ScrapeConfig
metadata:
  name: wiki
  namespace: monitoring
  labels:
    prometheus: system-monitoring-prometheus
spec:
  scrapeInterval: 30s
  staticConfigs:
    - targets:
        - mysrv.example.com
      labels:
        job: wiki
        instance: mysrv.example.com
  metricsPath: /wiki/metrics
  tlsConfig:
    insecureSkipVerify: false 
  scheme: HTTPS

Kubernetes

Для включения метрик prometheus TeamStorm необходимо установить параметр .Values.metrics.enabled: true в values.yaml. Автоматически будут созданы ServiceMonitor, с помощью которых prometheus будет забирать метрики. Если в kubernetes не используется prometheus operator или нет crd, то необходимо отключить создание ServiceMonitor .Values.metrics.serviceMonitor.enabled: false в values.yaml и настроить с помощью ScrapeConfig.