minio性能测试

minio,性能,测试 · 浏览次数 : 91

小编点评

## Minio performance test script performance comparison This script compares the performance of different data transfer methods for reading and writing objects to Minio. **Methods tested:** * `s3fs` (Object Storage) * `fio` (Fast I/O) * `dd` (Disk copy) **Results:** | Operation | s3fs (obj) | fio | dd | |---|---|---|---| | Read 1G | 1048.576 KB/s | 1047.51 MiB/s | 1048.576 KB/s | | Write 1G | 2.73069 s | N/A | 2.61818 s | | Write 100M | 0.5556 s | N/A | 189 MB/s | | Write 20M | 0.139882 s | N/A | 150 MB/s | | Write 20M (cached) | 2.61818 s | 21 MB | 400 MB/s | | Read 1038.75 MB | 0.147937 s | N/A | 2.4 GB/s | | Write 20M (cached) | 0.0432576 s | 10.04 obj/s | 39.35 obj/s | **Observations:** * `dd` consistently performs the best, achieving high read and write speeds for both object and cache modes. * `s3fs` performs well for object reads but struggles with object writes due to slow block operations. * `fio` falls somewhere in between, offering good performance for both read and write with its flexible I/O model. **Conclusions:** * For reading, `dd` emerges as the clear winner. * For writing, `dd` again shines, followed by `fio`. * `s3fs` falls behind but still offers a respectable performance for both read and write. **Recommendations:** * Use `dd` for high-performance data transfer. * Consider using `fio` for both read and write if performance is a concern. * For complex scenarios with multiple concurrent processes, evaluate the performance of each method.

正文

minio性能测试

minio的使用

前期使用了s3fs 
但是想验证一下性能相关, 所以使用今天简单验证了一下,
其实也可以使用一下fio
但是s3fs 是对象存储 没有修改 只有上传, 所以感觉还是使用dd更加好一些.

dd 性能测试脚本-读取

rm -rf /tmp/cache/*
echo 3 >/proc/sys/vm/drop_caches
echo "不带缓存"
dd if=/minio_s3fs/draw.io-21.5.0-windows-no-installer.exe  of=/dev/zero bs=1024k count=100

echo "带缓存"
dd if=/minio_s3fs/draw.io-21.5.0-windows-no-installer.exe  of=/dev/zero bs=1024k count=100

dd 性能测试脚本-写入

rm -rf /tmp/cache/*
echo 3 >/proc/sys/vm/drop_caches
echo "不带缓存"
echo "1G"
dd of=/minio_s3fs/s3fsddin1G.io  if=/dev/zero bs=1024k count=1000
echo "100M"
dd of=/minio_s3fs/s3fsddin100M.io  if=/dev/zero bs=1024k count=100
echo "20M"
dd of=/minio_s3fs/s3fsddin20M.io  if=/dev/zero bs=1024k count=20

echo "带缓存"
echo "1G"
dd of=/minio_s3fs/s3fsddin1G.io  if=/dev/zero bs=1024k count=1000
echo "100M"
dd of=/minio_s3fs/s3fsddin100M.io  if=/dev/zero bs=1024k count=100
echo "20M"
dd of=/minio_s3fs/s3fsddin20M.io  if=/dev/zero bs=1024k count=20

echo "本地文件系统"
dd if=/dev/vda1  of=/dev/zero bs=1024k count=1000

测试结果为

## 写入到s3fs ##

不带缓存写入
1G:
1048576000字节（1.0 GB，1000 MiB）已复制，2.73069 s，384 MB/s
100M:
104857600字节（105 MB，100 MiB）已复制，0.5556 s，189 MB/s
20M:
20971520字节（21 MB，20 MiB）已复制，0.139882 s，150 MB/s
带缓存写入
1G:
1048576000字节（1.0 GB，1000 MiB）已复制，2.61818 s，400 MB/s
100M:
104857600字节（105 MB，100 MiB）已复制，0.336142 s，312 MB/s
20M:
20971520字节（21 MB，20 MiB）已复制，0.142419 s，147 MB/s

本地文件系统
629145600字节（629 MB，600 MiB）已复制，0.130208 s，4.8 GB/s

## 从3sf3读取 ## 

不带缓存
103875056字节（104 MB，99 MiB）已复制，0.147937 s，702 MB/s
带缓存-通过参数cache到了本地
103875056字节（104 MB，99 MiB）已复制，0.0432576 s，2.4 GB/s

warp测试

第一步 下载:
https://github.com/minio/warp
注意可以直接下载二进制就可以了
使用 tar.gz 包 然后将文件放到 /usr/bin 目录下并且赋予权限

第一步开启client
注意需要设定访问密码: 
WARP_ACCESS_KEY=minioadmin 
WARP_SECRET_KEY=minioadmin
然后执行 
warp client 就可以打开客户端. 

效果为:
warp client
warp: Listening on :7761

第二步: 使用warp进行测试. 
warp mixed --warp-client 192.168.255.65:7761 --host 192.168.255.45:9000 --duration 5s --obj.size 10M --concurrent 10     --autoterm   --access-key=minioadmin --secret-key=minioadmin
测试结果为:

warp: Benchmark data written to "warp-remote-2023-07-22[143906]-lATQ.csv.zst"
Mixed operations.
Operation: DELETE, 10%, Concurrency: 10, Ran 4s.
 * Throughput: 25.67 obj/s

Operation: GET, 43%, Concurrency: 10, Ran 4s.
 * Throughput: 1047.51 MiB/s, 109.84 obj/s

Operation: PUT, 15%, Concurrency: 10, Ran 4s.
 * Throughput: 375.29 MiB/s, 39.35 obj/s

Operation: STAT, 30%, Concurrency: 10, Ran 4s.
 * Throughput: 76.02 obj/s

Cluster Total: 1418.33 MiB/s, 249.86 obj/s over 5s.

如果是两个并发:
warp mixed --warp-client 192.168.255.65:7761 --host 192.168.255.45:9000 --duration 5s --obj.size 20M --concurrent 2     --autoterm   --access-key=minioadmin --secret-key=minioadmin
warp: Benchmark data written to "warp-remote-2023-07-22[144630]-3OOp.csv.zst"
Mixed operations.
Operation: DELETE, 10%, Concurrency: 2, Ran 4s.
 * Throughput: 5.73 obj/s

Operation: GET, 41%, Concurrency: 2, Ran 4s.
 * Throughput: 411.84 MiB/s, 21.59 obj/s

Operation: PUT, 16%, Concurrency: 2, Ran 4s.
 * Throughput: 191.57 MiB/s, 10.04 obj/s

Operation: STAT, 29%, Concurrency: 2, Ran 4s.
 * Throughput: 16.23 obj/s

Cluster Total: 587.10 MiB/s, 52.40 obj/s over 5s.