ClickHouse 各版本在不同 CPU 架构上的性能差异对比

引子

需求嘛，总是一步一步，慢慢迭代到这么样子的，最初，我只是想在 ARM64 的平台上安安静静地运行 ClickHouse，然后发现官方没有提供 ARM64 的 Docker 镜像，已有的 ARM64 镜像（比如 altinity/clickhouse-server:21.12.3.32.altinitydev.arm 和 lunalabsltd/clickhouse-server:21.7.2.7-arm 找不到对应的 Dockerfile 而且版本很少），本来想自己构建一下 ClickHouse 的 ARM64 镜像却发现官方只提供 Master 分支的 Binary https://builds.clickhouse.com/master/aarch64/clickhouse）

构建 Multi-Arch 的 ClickHouse

ARM 无人权这句话，是真的

为了构建 Multi-Arch 的 ClickHouse，我们需要了解官方的 ClickHouse 是怎么构建的，在阅读了官方 Dockerfile 后，我们了解到，要成功构建 ClickHouse，需要以下：

• Clang 12
• 高版本的 CMake
• ninjia 等构建工具

要构建 Multi-Arch 的镜像就需要 Multi-Arch 的构建工具，由于安装 CMake 等构建工具需要手动编译，耗时很久，且是一个通用组件（应该由 GitHub Actions 这类 CI 来完成），于是我将构建工具放到了 https://github.com/knatnetwork/clickhouse-builder ，在有了构建工具之后我们就可以按照 「在 GitHub Actions 上使用多 Job 并行构建，提升 Multi-Arch 镜像制作速度」的方式进行分类构建，相关的 Workflow 类似如下：

build-arm64-image:
  name: Build v${{ github.event.inputs.clickhouse-version }} ARM64 Image
  runs-on: [self-hosted,arm64]
  steps:
    - uses: actions/checkout@v3

    - name: Clone Clickhouse and submodules
      run: |
        git clone https://github.com/ClickHouse/ClickHouse.git
        cd ClickHouse && git checkout v${{ github.event.inputs.clickhouse-version }} && git submodule update --init --recursive        

    - name: Login to Docker Hub
      uses: docker/login-action@v1
      with:
        username: knatnetwork
        password: ${{ secrets.DOCKERHUB_PASSWD }}

    - name: Set up QEMU
      uses: docker/setup-qemu-action@v1

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v1

    - name: Build and push latest images
      uses: docker/build-push-action@v2
      with:
        context: .
        platforms: linux/arm64
        push: true
        tags: |
          knatnetwork/clickhouse-arm64:${{ github.event.inputs.clickhouse-version }}          

关于构建的 Dockerfile 可以直接参考：https://github.com/knatnetwork/clickhouse-server/blob/master/Dockerfile

这里由于 GitHub Actions 机器的配置过于拉跨，6 小时内都很难构建完成 AMD64 版本的 Binary，所以只能使用 Self-hosted 的机器。

Self-Hosted Runner

为了快速构建 ClickHouse，这里我使用了两类机器来构建：

• ARM64: AWS a1.metal
• AMD64: Hetzner CPX51

由于有了之前开源的 https://github.com/knatnetwork/github-runner ，要快速水平扩展 Runner 只需要大量开机器，然后用 Ansible 把 docker-compose.yml 和 config.json 文件推上去启动即可，很快，我们就有了一些 Runner：

很快，我的钱包也瘪了，AWS 一个晚上花了 100+ USD

不过问题不大，钱已经花了，经过一个晚上的持续运行，我们成功地构建了 ClickHouse 上从最新 Tag 到 21.7 之间的所有版本（21.7 版本构建会报错，后文会涉及），准确来说，构建了如下镜像：

knatnetwork/clickhouse-server:22.2.3.5-stable,knatnetwork/clickhouse-server:22.2.2.1-stable,knatnetwork/clickhouse-server:22.1.4.30-stable,knatnetwork/clickhouse-server:22.1.3.7-stable,knatnetwork/clickhouse-server:22.1.2.2-stable,knatnetwork/clickhouse-server:21.12.4.1-stable,knatnetwork/clickhouse-server:21.12.3.32-stable,knatnetwork/clickhouse-server:21.12.2.17-stable,knatnetwork/clickhouse-server:21.11.11.1-stable,knatnetwork/clickhouse-server:21.11.10.1-stable,knatnetwork/clickhouse-server:21.11.9.1-stable,knatnetwork/clickhouse-server:21.11.8.4-stable,knatnetwork/clickhouse-server:21.11.7.9-stable,knatnetwork/clickhouse-server:21.11.6.7-stable,knatnetwork/clickhouse-server:21.11.5.33-stable,knatnetwork/clickhouse-server:21.11.4.14-stable,knatnetwork/clickhouse-server:21.11.3.6-stable,knatnetwork/clickhouse-server:21.11.2.2-stable,knatnetwork/clickhouse-server:21.10.6.2-stable,knatnetwork/clickhouse-server:21.10.5.3-stable,knatnetwork/clickhouse-server:21.10.4.26-stable,knatnetwork/clickhouse-server:21.10.3.9-stable,knatnetwork/clickhouse-server:21.10.2.15-stable,knatnetwork/clickhouse-server:21.9.6.24-stable,knatnetwork/clickhouse-server:21.9.5.16-stable,knatnetwork/clickhouse-server:21.9.4.35-stable,knatnetwork/clickhouse-server:21.9.3.30-stable,knatnetwork/clickhouse-server:21.9.2.17-stable,knatnetwork/clickhouse-server:21.8.15.7-lts,knatnetwork/clickhouse-server:21.8.14.5-lts,knatnetwork/clickhouse-server:21.8.13.6-lts,knatnetwork/clickhouse-server:21.8.12.29-lts,knatnetwork/clickhouse-server:21.8.11.4-lts,knatnetwork/clickhouse-server:21.8.10.19-lts,knatnetwork/clickhouse-server:21.8.9.13-lts,knatnetwork/clickhouse-server:21.8.8.29-lts,knatnetwork/clickhouse-server:21.8.7.22-lts,knatnetwork/clickhouse-server:21.8.6.15-lts,knatnetwork/clickhouse-server:21.8.5.7-lts,knatnetwork/clickhouse-server:21.8.4.51-lts,knatnetwork/clickhouse-server:21.8.3.44-lts

而且都是 Multi-Arch 的，地址在： https://hub.docker.com/r/knatnetwork/clickhouse-server

Benchmark

花了这么多钱贡献了这么多镜像，总希望可以做点测试，由于我代码写的不行，又没有搞测试的经验，暂时只想到了以下两个测试点：

• 这么多不同的版本之间是否会有性能差异（横向比较）
• 对于不同的类型的机器而言，他们的性能差异是如何的（纵向比较）

Benchmark Platform

由于我不懂 SQL 也对 ClickHouse 一窍不通，所以这里参考了 ClickHouse 官方的测试方式：（官方的展示：https://clickhouse.com/benchmark/hardware/），

数据集采用了一个公开的数据集，地址在：https://datasets.clickhouse.com/hits/partitions/hits_100m_obfuscated_v1.tar.xz， 总共 100000000 条数据，解压后 16G，测试的方式为用 ClickHouse 官方提供的 43 条 SQL 语句进行测试（可能这 43 条语句覆盖了大部分使用场景）

SELECT count() FROM hits_100m_obfuscated;
SELECT count() FROM hits_100m_obfuscated WHERE AdvEngineID != 0;
SELECT sum(AdvEngineID), count(), avg(ResolutionWidth) FROM hits_100m_obfuscated ;
SELECT sum(UserID) FROM hits_100m_obfuscated ;
SELECT uniq(UserID) FROM hits_100m_obfuscated ;
SELECT uniq(SearchPhrase) FROM hits_100m_obfuscated ;
SELECT min(EventDate), max(EventDate) FROM hits_100m_obfuscated ;
SELECT AdvEngineID, count() FROM hits_100m_obfuscated WHERE AdvEngineID != 0 GROUP BY AdvEngineID ORDER BY count() DESC;
SELECT RegionID, uniq(UserID) AS u FROM hits_100m_obfuscated GROUP BY RegionID ORDER BY u DESC LIMIT 10;
SELECT RegionID, sum(AdvEngineID), count() AS c, avg(ResolutionWidth), uniq(UserID) FROM hits_100m_obfuscated GROUP BY RegionID ORDER BY c DESC LIMIT 10;
SELECT MobilePhoneModel, uniq(UserID) AS u FROM hits_100m_obfuscated WHERE MobilePhoneModel != '' GROUP BY MobilePhoneModel ORDER BY u DESC LIMIT 10;
SELECT MobilePhone, MobilePhoneModel, uniq(UserID) AS u FROM hits_100m_obfuscated WHERE MobilePhoneModel != '' GROUP BY MobilePhone, MobilePhoneModel ORDER BY u DESC LIMIT 10;
SELECT SearchPhrase, count() AS c FROM hits_100m_obfuscated WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
SELECT SearchPhrase, uniq(UserID) AS u FROM hits_100m_obfuscated WHERE SearchPhrase != '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10;
SELECT SearchEngineID, SearchPhrase, count() AS c FROM hits_100m_obfuscated WHERE SearchPhrase != '' GROUP BY SearchEngineID, SearchPhrase ORDER BY c DESC LIMIT 10;
SELECT UserID, count() FROM hits_100m_obfuscated GROUP BY UserID ORDER BY count() DESC LIMIT 10;
SELECT UserID, SearchPhrase, count() FROM hits_100m_obfuscated GROUP BY UserID, SearchPhrase ORDER BY count() DESC LIMIT 10;
SELECT UserID, SearchPhrase, count() FROM hits_100m_obfuscated GROUP BY UserID, SearchPhrase LIMIT 10;
SELECT UserID, toMinute(EventTime) AS m, SearchPhrase, count() FROM hits_100m_obfuscated GROUP BY UserID, m, SearchPhrase ORDER BY count() DESC LIMIT 10;
SELECT UserID FROM hits_100m_obfuscated WHERE UserID = 12345678901234567890;
SELECT count() FROM hits_100m_obfuscated WHERE URL LIKE '%metrika%';
SELECT SearchPhrase, any(URL), count() AS c FROM hits_100m_obfuscated WHERE URL LIKE '%metrika%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
SELECT SearchPhrase, any(URL), any(Title), count() AS c, uniq(UserID) FROM hits_100m_obfuscated WHERE Title LIKE '%Яндекс%' AND URL NOT LIKE '%.yandex.%' AND SearchPhrase != '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10;
SELECT * FROM hits_100m_obfuscated WHERE URL LIKE '%metrika%' ORDER BY EventTime LIMIT 10;
SELECT SearchPhrase FROM hits_100m_obfuscated WHERE SearchPhrase != '' ORDER BY EventTime LIMIT 10;
SELECT SearchPhrase FROM hits_100m_obfuscated WHERE SearchPhrase != '' ORDER BY SearchPhrase LIMIT 10;
SELECT SearchPhrase FROM hits_100m_obfuscated WHERE SearchPhrase != '' ORDER BY EventTime, SearchPhrase LIMIT 10;
SELECT CounterID, avg(length(URL)) AS l, count() AS c FROM hits_100m_obfuscated WHERE URL != '' GROUP BY CounterID HAVING c > 100000 ORDER BY l DESC LIMIT 25;
SELECT domainWithoutWWW(Referer) AS key, avg(length(Referer)) AS l, count() AS c, any(Referer) FROM hits_100m_obfuscated WHERE Referer != '' GROUP BY key HAVING c > 100000 ORDER BY l DESC LIMIT 25;
SELECT sum(ResolutionWidth), sum(ResolutionWidth + 1), sum(ResolutionWidth + 2), sum(ResolutionWidth + 3), sum(ResolutionWidth + 4), sum(ResolutionWidth + 5), sum(ResolutionWidth + 6), sum(ResolutionWidth + 7), sum(ResolutionWidth + 8), sum(ResolutionWidth + 9), sum(ResolutionWidth + 10), sum(ResolutionWidth + 11), sum(ResolutionWidth + 12), sum(ResolutionWidth + 13), sum(ResolutionWidth + 14), sum(ResolutionWidth + 15), sum(ResolutionWidth + 16), sum(ResolutionWidth + 17), sum(ResolutionWidth + 18), sum(ResolutionWidth + 19), sum(ResolutionWidth + 20), sum(ResolutionWidth + 21), sum(ResolutionWidth + 22), sum(ResolutionWidth + 23), sum(ResolutionWidth + 24), sum(ResolutionWidth + 25), sum(ResolutionWidth + 26), sum(ResolutionWidth + 27), sum(ResolutionWidth + 28), sum(ResolutionWidth + 29), sum(ResolutionWidth + 30), sum(ResolutionWidth + 31), sum(ResolutionWidth + 32), sum(ResolutionWidth + 33), sum(ResolutionWidth + 34), sum(ResolutionWidth + 35), sum(ResolutionWidth + 36), sum(ResolutionWidth + 37), sum(ResolutionWidth + 38), sum(ResolutionWidth + 39), sum(ResolutionWidth + 40), sum(ResolutionWidth + 41), sum(ResolutionWidth + 42), sum(ResolutionWidth + 43), sum(ResolutionWidth + 44), sum(ResolutionWidth + 45), sum(ResolutionWidth + 46), sum(ResolutionWidth + 47), sum(ResolutionWidth + 48), sum(ResolutionWidth + 49), sum(ResolutionWidth + 50), sum(ResolutionWidth + 51), sum(ResolutionWidth + 52), sum(ResolutionWidth + 53), sum(ResolutionWidth + 54), sum(ResolutionWidth + 55), sum(ResolutionWidth + 56), sum(ResolutionWidth + 57), sum(ResolutionWidth + 58), sum(ResolutionWidth + 59), sum(ResolutionWidth + 60), sum(ResolutionWidth + 61), sum(ResolutionWidth + 62), sum(ResolutionWidth + 63), sum(ResolutionWidth + 64), sum(ResolutionWidth + 65), sum(ResolutionWidth + 66), sum(ResolutionWidth + 67), sum(ResolutionWidth + 68), sum(ResolutionWidth + 69), sum(ResolutionWidth + 70), sum(ResolutionWidth + 71), sum(ResolutionWidth + 72), sum(ResolutionWidth + 73), sum(ResolutionWidth + 74), sum(ResolutionWidth + 75), sum(ResolutionWidth + 76), sum(ResolutionWidth + 77), sum(ResolutionWidth + 78), sum(ResolutionWidth + 79), sum(ResolutionWidth + 80), sum(ResolutionWidth + 81), sum(ResolutionWidth + 82), sum(ResolutionWidth + 83), sum(ResolutionWidth + 84), sum(ResolutionWidth + 85), sum(ResolutionWidth + 86), sum(ResolutionWidth + 87), sum(ResolutionWidth + 88), sum(ResolutionWidth + 89) FROM hits_100m_obfuscated;
SELECT SearchEngineID, ClientIP, count() AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m_obfuscated WHERE SearchPhrase != '' GROUP BY SearchEngineID, ClientIP ORDER BY c DESC LIMIT 10;
SELECT WatchID, ClientIP, count() AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m_obfuscated WHERE SearchPhrase != '' GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;
SELECT WatchID, ClientIP, count() AS c, sum(Refresh), avg(ResolutionWidth) FROM hits_100m_obfuscated GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10;
SELECT URL, count() AS c FROM hits_100m_obfuscated GROUP BY URL ORDER BY c DESC LIMIT 10;
SELECT 1, URL, count() AS c FROM hits_100m_obfuscated GROUP BY 1, URL ORDER BY c DESC LIMIT 10;
SELECT ClientIP AS x, x - 1, x - 2, x - 3, count() AS c FROM hits_100m_obfuscated GROUP BY x, x - 1, x - 2, x - 3 ORDER BY c DESC LIMIT 10;
SELECT URL, count() AS PageViews FROM hits_100m_obfuscated WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT DontCountHits AND NOT Refresh AND notEmpty(URL) GROUP BY URL ORDER BY PageViews DESC LIMIT 10;
SELECT Title, count() AS PageViews FROM hits_100m_obfuscated WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT DontCountHits AND NOT Refresh AND notEmpty(Title) GROUP BY Title ORDER BY PageViews DESC LIMIT 10;
SELECT URL, count() AS PageViews FROM hits_100m_obfuscated WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT Refresh AND IsLink AND NOT IsDownload GROUP BY URL ORDER BY PageViews DESC LIMIT 1000;
SELECT TraficSourceID, SearchEngineID, AdvEngineID, ((SearchEngineID = 0 AND AdvEngineID = 0) ? Referer : '') AS Src, URL AS Dst, count() AS PageViews FROM hits_100m_obfuscated WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT Refresh GROUP BY TraficSourceID, SearchEngineID, AdvEngineID, Src, Dst ORDER BY PageViews DESC LIMIT 1000;
SELECT URLHash, EventDate, count() AS PageViews FROM hits_100m_obfuscated WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT Refresh AND TraficSourceID IN (-1, 6) AND RefererHash = halfMD5('http://example.ru/') GROUP BY URLHash, EventDate ORDER BY PageViews DESC LIMIT 100;
SELECT WindowClientWidth, WindowClientHeight, count() AS PageViews FROM hits_100m_obfuscated WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND NOT Refresh AND NOT DontCountHits AND URLHash = halfMD5('http://example.ru/') GROUP BY WindowClientWidth, WindowClientHeight ORDER BY PageViews DESC LIMIT 10000;
SELECT toStartOfMinute(EventTime) AS Minute, count() AS PageViews FROM hits_100m_obfuscated WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-02' AND NOT Refresh AND NOT DontCountHits GROUP BY Minute ORDER BY Minute;

为了快速在某一特定机型上完成所有版本的测试，我还自己做了一个 driver.py，大概的流程就是，对于每一个版本的镜像：

• 输出 docker-compose.yml 并拉镜像+使用特定版本启动
• 针对每个 SQL 语句跑三次取平均时间
• 上报测试数据到 MongoDB Atlas（Free Tier）
• 停止 ClickHouse 并删除当前镜像

对于每个机器的测试会输出一个类似如下的 JSON 格式的报告方便后续查询和展示：

{
  "clickhouse_commit_date": "2022-02-17T10:21:02Z",
  "clickhouse_commit_id": "97317e8bb51d2723b539b7863dd6471a68b11d01",
  "clickhouse_version": "22.2.3.5-stable",
  "date": "2022-04-18",
  "image_name": "knatnetwork/clickhouse-server:22.2.3.5-stable",
  "machine_arch": "ARM64",
  "machine_type": "c6g.2xlarge",
  "machine_vendor": "AWS",
  "results": [
      {
          "query": "SELECT count() FROM hits_100m_obfuscated;",
          "time": 0.0016631285349527996
      },
  ]
}

Machines

关于机器信息，之前看到过一个文章关于 AWS 上 AMD64 和 ARM64 性价比的对比，文章地址在： Evaluating Graviton 2 for data-intensive applications: an Arm vs Intel comparison 所以打算借这个机会对比一下配置相仿的 ARM64 和 AMD64 的机器的情况，使用的机器如下：

• Hetzner 的 CPX41(8Core,16G)，价格 22.9EUR/mo
• AWS 的 c5.2xlarge(8Core,16G,100G IO2 磁盘)，折算价格 244.8USD/mo
• AWS 的 c6g.2xlarge(8Core,16G,100G IO2 磁盘)，折算价格 195.84USD/mo
• OracleCloud 的 ARM64 机器（4Core,24G），白嫖的 BennyThink 的，折算价格 0USD/mo
• Vultr CPU Optmized General Purpose 的机器（8Core，16G）白嫖的 BennyThink 的，折算价格 0USD/mo，实际价格 160USD/mo
• Vultr BareMetal 的机器（4Core8Threads，16G）白嫖的 BennyThink 的，折算价格 0USD/mo，实际价格 120USD/mo

为了简单考虑，我直接把 43 个 Query 的总时间作为 Y 轴，版本号从低到高作为 X 轴，每一条线代表一个机器，作图（数值越低表示查询速度越快）：

为此我做了一个站点： https://clickperf.knat.network/ ，有兴趣的话大家可以去看看，上面提供原始数据下载。

从图中可以得到以下信息：

• 配置都为 8Core/16G 的 ARM64 c6g.2xlarge 机器在这个测试上表现比 AMD64 c5.2xlarge 的机器好
• 在三个特定版本 （21.12.2.17-stable,21.11.11.1-stable,21.11.10.1-stable） 上，ARM64 的机器性能有显著提升，AMD64 的机器未观察到类似的提升，原因不明
• 价格为 22.9EUR/mo 的 CPX41 和价格为 195.84USD/mo 的 c6g.2xlarge 测试性能相仿
• 价格为 120USD/mo 的 Vultr Baremetal 和价格为 244.8USD/mo 的 c5.2xlarge 测试性能相仿
• Vultr 的 CPU Optmized General Purpose 机器性能抖动较大
• Vultr 的 Baremetal 机器性能抖动非常小
• 在 ClickHouse 的各个 Release 中，这 43 个 Query 的总时间相对稳定
• AWS 机器的性价比是真的低
• Vultr 机器的性能抖动挺大

另外在跑 Benchmark 的过程中，发现 ARM64 的机器普遍可以用满所有的 CPU 核心，而 AMD64 的机器除了 Hetzner 以外都没法用满所有核心（比如一直只有某 4 个核心是满载，其余核心 30% 左右负载）， BennyThink 怀疑是被宿主机限制，但是这个说法暂时没法得到佐证，也有可能是我构建的镜像有啥问题。

Afterwords

在这里我想非常感谢 BennyThink 大佬给我提供了许多机器用于测试，没有 TA 的帮助，这个测试会失去许多重要数据，也没法因此发现 Vultr 其实也挺拉跨的。

也要感谢 purelind 大佬给我提供了 Oracle 帐号和一些测试上的指导。

还要感谢某个神秘人士帮我制作了 https://clickperf.knat.network/ 并报销了 AWS 账单上的开销，并让我深刻体会了——「没有钱，就没法做科研」的道理。

Appendix

DB::Exception: Memory limit (total) exceeded

在测试中某些请求会失败，然后报错信息类似：

DB::Exception: Received from localhost:9000, 127.0.0.1. 
DB::Exception: Memory limit for query exceeded: would use 9.31 GiB  attempt to allocate chunk of 1048576 bytes , maximum: 9.31 GiB: 

这种时候需要在 clickhouse-user-config.xml 中加入一个 max_memory_usage，类似如下：

<yandex>
    <profiles>
        <default>
            <max_memory_usage>1280000000000</max_memory_usage>
            <log_queries>0</log_queries>
            <log_query_threads>0</log_query_threads>
        </default>
    </profiles>
</yandex>

21.7.x 版本报错

可以參考：https://github.com/knatnetwork/clickhouse-server/runs/6053314472?check_suite_focus=true

#12 11.85 -- SYSTEM_LIBS zlib;ssl;crypto
#12 11.87 -- Dynamic column API support: ON
#12 11.88 -- SYSTEM processor: x86_64
#12 11.88 CMake Error at contrib/mariadb-connector-c/cmake/ConnectorName.cmake:30 (ENDMACRO):
#12 11.88   Flow control statements are not properly nested.
#12 11.88 Call Stack (most recent call first):
#12 11.88   contrib/mariadb-connector-c/CMakeLists.txt:428 (INCLUDE)
#12 11.88 
#12 11.88 
#12 11.88 -- Configuring incomplete, errors occurred!
#12 11.88 See also "/root/ClickHouse/build/CMakeFiles/CMakeOutput.log".
#12 11.88 See also "/root/ClickHouse/build/CMakeFiles/CMakeError.log".
#12 ERROR: process "/bin/sh -c cd /root/ClickHouse && mkdir build && cd build && cmake .. && ninja" did not complete successfully: exit code: 1
------
 > [buildtime 4/5] RUN cd /root/ClickHouse && mkdir build && cd build && cmake .. && ninja:
#12 11.88 -- SYSTEM processor: x86_64
#12 11.88 CMake Error at contrib/mariadb-connector-c/cmake/ConnectorName.cmake:30 (ENDMACRO):
#12 11.88   Flow control statements are not properly nested.
#12 11.88 Call Stack (most recent call first):
#12 11.88   contrib/mariadb-connector-c/CMakeLists.txt:428 (INCLUDE)
#12 11.88 
#12 11.88 
#12 11.88 -- Configuring incomplete, errors occurred!
#12 11.88 See also "/root/ClickHouse/build/CMakeFiles/CMakeOutput.log".
#12 11.88 See also "/root/ClickHouse/build/CMakeFiles/CMakeError.log".
------

以下记录了用于测试的机器的 CPU 信息

CPU 信息

Vultr Bare Metal

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   39 bits physical, 48 bits virtual
CPU(s):                          8
On-line CPU(s) list:             0-7
Thread(s) per core:              2
Core(s) per socket:              4
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           158
Model name:                      Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz
Stepping:                        9
CPU MHz:                         800.337
CPU max MHz:                     4200.0000
CPU min MHz:                     800.0000
BogoMIPS:                        7599.80
Virtualization:                  VT-x
L1d cache:                       128 KiB
L1i cache:                       128 KiB
L2 cache:                        1 MiB
L3 cache:                        8 MiB
NUMA node0 CPU(s):               0-7

Vultr CPU Optmized General Purpose

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   40 bits physical, 48 bits virtual
CPU(s):                          8
On-line CPU(s) list:             0-7
Thread(s) per core:              2
Core(s) per socket:              4
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       AuthenticAMD
CPU family:                      23
Model:                           49
Model name:                      AMD EPYC-Rome Processor
Stepping:                        0
CPU MHz:                         1996.249
BogoMIPS:                        3992.49
Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       128 KiB
L1i cache:                       128 KiB
L2 cache:                        2 MiB
L3 cache:                        16 MiB
NUMA node0 CPU(s):               0-7

CPX41

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   40 bits physical, 48 bits virtual
CPU(s):                          8
On-line CPU(s) list:             0-7
Thread(s) per core:              1
Core(s) per socket:              8
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       AuthenticAMD
CPU family:                      23
Model:                           49
Model name:                      AMD EPYC Processor
Stepping:                        0
CPU MHz:                         2445.406
BogoMIPS:                        4890.81
Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       256 KiB
L1i cache:                       256 KiB
L2 cache:                        4 MiB
L3 cache:                        32 MiB
NUMA node0 CPU(s):               0-7

c5.2xlarge

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   46 bits physical, 48 bits virtual
CPU(s):                          8
On-line CPU(s) list:             0-7
Thread(s) per core:              2
Core(s) per socket:              4
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           85
Model name:                      Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz
Stepping:                        4
CPU MHz:                         2999.998
BogoMIPS:                        5999.99
Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       128 KiB
L1i cache:                       128 KiB
L2 cache:                        4 MiB
L3 cache:                        24.8 MiB
NUMA node0 CPU(s):               0-7

c6g.2xlarge

Architecture:                    aarch64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
CPU(s):                          8
On-line CPU(s) list:             0-7
Thread(s) per core:              1
Core(s) per socket:              8
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       ARM
Model:                           1
Model name:                      Neoverse-N1
Stepping:                        r3p1
BogoMIPS:                        243.75
L1d cache:                       512 KiB
L1i cache:                       512 KiB
L2 cache:                        8 MiB
L3 cache:                        32 MiB
NUMA node0 CPU(s):               0-7

Reference

• https://blog.csdn.net/anyitian/article/details/115390396
• https://clickhouse.com/docs/en/operations/settings/query-complexity/
• https://stackoverflow.com/questions/65316905/clickhouse-dbexception-memory-limit-for-query-exceeded
• Evaluating Graviton 2 for data-intensive applications: an Arm vs Intel comparison