Youluo's Blog

概述

Chproxy 是一个Golang实现的，用于 ClickHouse 数据库的 HTTP 代理、负载均衡器。

最初的想法有两个：

来自于clickhouse写操作，不希望通过分布式表来写入，而是写本地表，但是本地表又需要负载均衡。
clickhouse客户端查询，加了一个定时检查clickhouse server是否健康的语句，不太优雅。

目标

安装chproxy 1.26.5，代理clickhouse http请求（实现负载均衡）。

环境说明

shell

# cat /etc/centos-release
CentOS Linux release 7.4.1708 (Core)

安装步骤

下载安装包并解压

shell

wget https://github.com/ContentSquare/chproxy/releases/download/v1.26.5/chproxy_1.26.5_linux_amd64.tar.gz

mkdir -p chproxy
tar -zxvf chproxy_1.26.5_linux_amd64.tar.gz -C chproxy
cd chproxy
chmod +x chproxy
mv chproxy /usr/bin/
ls /usr/bin/ | grep chproxy

编写配置文件

shell

mkdir -p /etc/chproxy/
vim /etc/chproxy/chproxy.yml

追加：

yaml

server:
  http:
      listen_addr: ":18123"
      allowed_networks: ["11.123.247.0/24"]

users:
  - name: "distributed-write"
    password: "CertS0c"
    to_cluster: "distributed-write"
    to_user: "default"

  - name: "replica-write"
    password: "CertS0c"  
    to_cluster: "replica-write"
    to_user: "default"

  - name: "distributed-read"
    password: "CertS0c"  
    to_cluster: "distributed-read"
    to_user: "default"
    max_concurrent_queries: 6
    max_execution_time: 1m

clusters:
  - name: "replica-write"
    replicas:
      - name: "replica1"
        nodes: ["ck01:8123", "ck02:8123"]
      - name: "replica2"
        nodes: ["ck03:8123", "ck04:8123"]
    users:
      - name: "default"
        password: "p@$$W0rd"

  - name: "distributed-write"
    nodes: [
      "ck01:8123",
      "ck02:8123",
      "ck03:8123",
      "ck04:8123"
    ]
    users:
      - name: "default"
        password: "p@$$W0rd"

  - name: "distributed-read"
    nodes: [
      "ck01:8123",
      "ck02:8123",
      "ck03:8123",
      "ck04:8123"
    ]
    users:
    - name: "default"
      password: "p@$$W0rd"

#caches:
#  - name: "shortterm"
#    dir: "/etc/chproxy/cache/shortterm"
#    max_size: 150Mb
#    expire: 130s

配置成service

vim /etc/systemd/system/chproxy.service，追加内容如下：

shell

[Unit]
Description=Chproxy
Documentation=https://www.chproxy.org/
Requires=network.target
After=network.target

[Service]
Type=notify
ExecStart=/usr/bin/chproxy -config /etc/chproxy/chproxy.yml
TimeoutSec=30
Restart=on-failure

# quick response on bad config
TimeoutStartSec=5

# avoid resource intensive cycles
RestartSec=30

[Install]
WantedBy=default.target

启动

shell

systemctl start chproxy
systemctl status chproxy

检查状态

shell

#方式1：查看systemctl 状态
systemctl status chproxy

#方式2：查看进程关键字
ps -ef | grep -v grep | grep chproxy

# 方式3：查看端口
netstat  -lunpt | grep 18123

验证

shell

echo 'select * from system.clusters' | curl 'http://11.123.247.217:18123/?user=distributed-read&password=CertS0c' --data-binary @-
echo 'select * from system.clusters' | curl 'http://11.123.247.217:18123/?user=distributed-write&password=CertS0c' --data-binary @-

echo "INSERT INTO cluster_db.user_logs_local (user_id, event_type, event_time, event_data, ip_address) VALUES (4, 'click--', '2024-11-12 10:00:00', 'Clicked button', '192.168.1.1'), (5, 'view--', '2024-11-12 10:05:00', 'Viewed page', '192.168.1.2'), (6, 'purchase--', '2024-11-12 10:10:00', 'Purchased item', '192.168.1.3'); " | curl 'http://11.123.247.217:18123/?user=replica-write&password=CertS0c' --data-binary @-

监控

检查metrics

请求metrics，预期有Prometheus标准的返回值

shell

curl http://11.123.247.217:18123/metrics

Prometheus配置

Prometheus主机上操作

将 chproxy 实例指标导出添加到 /etc/prometheus/prometheus.yml

yaml

global:
  scrape_interval: 15s
scrape_configs:
  - job_name: 'chproxy'
    static_configs:
      - targets: ['11.123.247.217:18123']

重启Prometheus :

shell

cd /opt/yeyouluo/docker-apps/prometheus/
docker compose restart prometheus

在Prometheus页面中搜索 status_codes_proxy_total关键字

配置Grafana

模板地址：https://github.com/ContentSquare/chproxy/blob/master/chproxy_overview.json

Grafana暂时没有显示出来，原因待排查

细节总结

chproxy 用户可以没有密码，但使用Apache SeaTunnel的clickhouse sink要求必须有password
写操作使用用户 replica-write

概述 ​

目标 ​

环境说明 ​

安装步骤 ​

监控 ​

检查metrics ​

Prometheus配置 ​

配置Grafana ​

细节总结 ​

参考链接 ​

概述

目标

环境说明

安装步骤

监控

检查metrics

Prometheus配置

配置Grafana

细节总结

参考链接