zeebe 集成elasticsearch exporter

佚名 6年前 (2019-04-27) 随笔 988人围观抢沙发百度已收录

zeebe 目前还在一直的开发中，同时一些变动还是挺大的，比如simple monitor 的以前是不需要配置HazelcastExporter的
估计是为了进行集群功能处理，新添加的，以前写的配置基本都没法使用了，所以写了一个新的运行配置

说明运行的时候注意版本，我测试的是Zeebe 0.17.0，可能其他版本有变动
SRE实战互联网时代守护先锋，助力企业售后服务体系运筹帷幄！一键直达领取阿里云限量特价优惠。

环境准备

docker-compose 文件

es 使用aws 开源的，同时禁用了ssl，使用http 请求

version: "3"
services:
  db:
    image: oscarfonts/h2
    container_name: zeebe_db
    ports:
      - "1521:1521"
      - "81:81"
  monitor:
    image: camunda/zeebe-simple-monitor
    environment:
      - spring.datasource.url=jdbc:h2:tcp://db:1521/zeebe-monitor
      - io.zeebe.monitor.connectionString=app:26500
      - io.zeebe.monitor.hazelcast.connection=app:5701
    ports:
    - "8080:8080"
  app:
    image: camunda/zeebe
    volumes:
      - ./zeebe-simple-monitor-exporter-0.13.0.jar:/usr/local/zeebe/lib/zeebe-simple-monitor-exporter-0.13.0.jar
      - ./zeebe-hazelcast-exporter-0.2.0.jar:/usr/local/zeebe/lib/zeebe-hazelcast-exporter-0.2.0.jar
      - ./zeebe.cfg.toml:/usr/local/zeebe/conf/zeebe.cfg.toml
    ports:
    - "26500:26500"
    - "26501:26501"
    - "26502:26502"
    - "26503:26503"
    - "26504:26504"
    - "5701:5701"
  odfe-node1:
    image: amazon/opendistro-for-elasticsearch:0.8.0
    container_name: odfe-node1
    environment:
      - opendistro_security.ssl.http.enabled=false
      - cluster.name=odfe-cluster
      - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./odfe-data1:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
  odfe-node2:
    image: amazon/opendistro-for-elasticsearch:0.8.0
    container_name: odfe-node2
    environment:
      - opendistro_security.ssl.http.enabled=false
      - cluster.name=odfe-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - discovery.zen.ping.unicast.hosts=odfe-node1
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./odfe-data2:/usr/share/elasticsearch/data
  kibana:
    image: amazon/opendistro-for-elasticsearch-kibana:0.8.0
    container_name: odfe-kibana
    ports:
      - 5601:5601
    expose:
      - "5601"
    environment:
      ELASTICSEARCH_URL: http://odfe-node1:9200

几个exporter 下载

写好了一个简单的shell，执行就可以了，或者clone github 代码 camunda-zeebe-es-exporter-demo

#!/bin/bash
wget https://github.com/zeebe-io/zeebe-simple-monitor/releases/download/0.13.0/zeebe-simple-monitor-exporter-0.13.0.jar
wget https://github.com/zeebe-io/zeebe-hazelcast-exporter/releases/download/0.2.0/zeebe-hazelcast-exporter-0.2.0.jar

zeebe 配置

主要是几个exporter 的配置,es 以及simple monitor 的,配置有点多，只写添加的，完整参考githubcamunda-zeebe-es-exporter-demo

[[exporters]]
id = "simple-monitor"
className = "io.zeebe.monitor.SimpleMonitorExporter"
[exporters.args]
  jdbcUrl = "jdbc:h2:tcp://db:1521/zeebe-monitor"
  driverName = "org.h2.Driver"
  userName = "sa"
  password = ""
[[exporters]]
 id = "hazelcast"
 className = "io.zeebe.hazelcast.exporter.HazelcastExporter"
[[exporters]]
id = "elasticsearch"
className = "io.zeebe.exporter.ElasticsearchExporter"
[exporters.args]
  url = "http://odfe-node1:9200"
 [exporters.args.bulk]
 delay = 5
 size = 1_000

 [exporters.args.authentication] 
   username = "admin"
   password = "admin"
 [exporters.args.index]
  prefix = "zeebe-record"

  ## 这个参数挺重要，设置为true，有异常，false 是可以的，应该和es版本有关系
  createTemplate = false
  command = false
  event = true
  rejection = false
  deployment = true
  incident = true
  job = true
  message = false
  messageSubscription = false
  raft = false
  workflowInstance = true
  workflowInstanceSubscription = false

启动&&测试

启动

docker-compose up -d

几个问题
我们需要配置宿主机几个参数
vm.max_map_count：
/etc/sysctl.conf 添加
vm.max_map_count=262144 生效 sysctl -p
容器启动失败，可能需要设置挂载目录权限，简单的方法是 chmod 777 odfe-data1 odfe-data2
效果
es 集群

monitor