安装kube-state-metrics

  1. 下载

    $ cd /opt/k8s/work/
    $ wget -O kube-state-metrics.zip  https://github.com/kubernetes/kube-state-metrics/archive/master.zip
    $ unzip kube-state-metrics.tar.gz
    $ cd kube-state-metrics-master/examples/standard
    $ ls
    cluster-role-binding.yaml  cluster-role.yaml  deployment.yaml  service-account.yaml  service.yaml
    
  2. 修改namespace

    SRE实战 互联网时代守护先锋,助力企业售后服务体系运筹帷幄!一键直达领取阿里云限量特价优惠。

    将默认的namespace从kube-system修改成monitoring

    $ cd /opt/k8s/work/kube-state-metrics-master/examples/standard
    $ ls | xargs -I {} sed -i "s/namespace: kube-system/namespace: monitoring/g" {}
    
  3. 在service.yaml追加如下annotation

    annotations:
      prometheus.io/scraped: "true"
      
    

    修改后的server.yml文件

    cat service.yaml
     apiVersion: v1
     kind: Service
     metadata:
       annotations:
         prometheus.io/scraped: "true"
       labels:
         app.kubernetes.io/name: kube-state-metrics
         app.kubernetes.io/version: 1.9.5
       name: kube-state-metrics
       namespace: monitoring
     spec:
       clusterIP: None
       ports:
       - name: http-metrics
         port: 8080
         targetPort: http-metrics
       - name: telemetry
         port: 8081
         targetPort: telemetry
       selector:
         app.kubernetes.io/name: kube-state-metrics
     
    
  4. 启动服务

    $ cd /opt/k8s/work/kube-state-metrics-master/examples/standard
    $ kubectl create -f .
    
    

prometheus 收集指标

  1. prometheus 追加配置,仅抓取带有prometheus.io/scraped: "true" 注解的endpoint

    - job_name: "kubernetes-service-endpoints"
      kubernetes_sd_configs:
      - role: endpoints
      relabel_configs:
      - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scraped]
        action: keep
        regex: true
      - action: labelmap
        regex: __meta_kubernetes_service_label_(.+)
      - source_labels: [__meta_kubernetes_namespace]
        action: replace
        target_label: kubernetes_namespace
      - source_labels: [__meta_kubernetes_service_name]
        action: replace
        target_label: service_name
     
    
  2. prometheus重新加载配置后,查看对应的target

    prometheus(九) 收集kube-state-metrics指标信息 Cloud 第1张

    kube-state-metrics对应的pod有两个端口,在prometheus中显示了两个target

配置grafana

  1. 从grafana.com 下载state-metrics监控模版1. Kubernetes Deployment Statefulset Daemonset metrics,导入模版

    grafana效果图
    prometheus(九) 收集kube-state-metrics指标信息 Cloud 第2张

扫码关注我们
微信号:SRE实战
拒绝背锅 运筹帷幄