[转帖]如何利用wrarp测试oss性能？

如何,利用,wrarp,测试,oss,性能 · 浏览次数 : 0

小编点评

**使用wrap测试oss性能** **简介** 使用wrap测试oss对象存储的真实性能。 **步骤** 1. **安装docker** 2. **下载和运行warpv0.5.5** 3. **设置配置文件** 4. **运行测试** 5. **分析结果** **配置文件** ``` --no-color disable color theme--debug enable debug output--insecure disable TLS certificate verification--autocompletion install auto-completion for your shell--host value host. Multiple hosts can be specified as a comma separated list. (default: \"127.0.0.1:9000\") [$WARP_HOST]--access-key value Specify access key [$WARP_ACCESS_KEY]--secret-key value Specify secret key [$WARP_SECRET_KEY]--tls Use TLS (HTTPS) for transport [$WARP_TLS]--region value Specify a custom region [$WARP_REGION]--encrypt encrypt/decrypt objects (using server-side encryption with random keys)--bucket value Bucket to use for benchmark data. ALL DATA WILL BE DELETED IN BUCKET! (default: \"warp-benchmark-bucket\")--host-select value Host selection algorithm. Can be "weighed" or "roundrobin" (default: "weighed\")--concurrent value Run this many concurrent operations (default: 20)--noprefix Do not use separate prefix for each thread--prefix value Use a custom prefix for each thread--disable-multipart disable multipart uploads--md5 Add MD5 sum to uploads--storage-class value Specify custom storage class, for instance 'STANDARD' or 'REDUCED_REDUNDANCY'.--objects value Number of objects to upload. (default: 2500)--obj.size value Size of each generated object. Can be a number or 10KiB/MiB/GiB. All sizes are base 2 binary. (default: \"10MiB\")--get-distrib value The amount of GET operations. (default: 45)--stat-distrib value The amount of STAT operations. (default: 30)--put-distrib value The amount of PUT operations. (default: 15)--delete-distrib value The amount of DELETE operations. Must be at least the same as PUT. (default: 10)--obj.generator value Use specific data generator (default: "random\")--obj.randsize Randomize size of objects so they will be up to the specified size--benchdata value Output benchmark+profile data to this file. By default unique filename is generated.--serverprof value Run MinIO server profiling during benchmark; possible values are 'cpu', 'mem', 'block', 'mutex'. ``` **运行测试** ``` warp ``` **分析结果** 测试结果将被写入“warp-mixed-2022-04-18[030853]-nm3V.csv.zst”中。 **总结** 使用wrap测试oss对象存储的性能非常重要。通过设置配置并运行测试，您可以了解对象存储系统的真实性能。

正文

https://zhuanlan.zhihu.com/p/529735003

前言

我们利用mino与ceph rgw搭建好的oss经过多层网络转发，传输速度必定有所折损，这个时候我们使用wrap来测试oss对象存储的真实性能。

利用wrarp测试oss性能

wrarp是minio项目下的一个开源测试工具，wrap会上传一定随机数据到oss对象存储服务器上，统计过程消耗时间得出整个oss性能分析。

使用docker 执行测试

docker pull minio/warp:v0.5.5
docker run --net host minio/warp:v0.5.5 mixed --host=172.16.1.54:7480 \
--access-key=123456 \
--secret-key=123456 \
--autoterm

上述命令使用warpv0.5.5对地址为172.16.1.54:7480的oss服务器进行测试。

相关命令

FLAGS:
--no-color              disable color theme
--debug                 enable debug output
--insecure              disable TLS certificate verification
--autocompletion        install auto-completion for your shell
--host value            host. Multiple hosts can be specified as a comma separated list. (default: "127.0.0.1:9000") [$WARP_HOST]
--access-key value      Specify access key [$WARP_ACCESS_KEY]
--secret-key value      Specify secret key [$WARP_SECRET_KEY]
--tls                   Use TLS (HTTPS) for transport [$WARP_TLS]
--region value          Specify a custom region [$WARP_REGION]
--encrypt               encrypt/decrypt objects (using server-side encryption with random keys)
--bucket value          Bucket to use for benchmark data. ALL DATA WILL BE DELETED IN BUCKET! (default: "warp-benchmark-bucket")
--host-select value     Host selection algorithm. Can be "weighed" or "roundrobin" (default: "weighed")
--concurrent value      Run this many concurrent operations (default: 20)
--noprefix              Do not use separate prefix for each thread
--prefix value          Use a custom prefix for each thread
--disable-multipart     disable multipart uploads
--md5                   Add MD5 sum to uploads
--storage-class value   Specify custom storage class, for instance 'STANDARD' or 'REDUCED_REDUNDANCY'.
--objects value         Number of objects to upload. (default: 2500)
--obj.size value        Size of each generated object. Can be a number or 10KiB/MiB/GiB. All sizes are base 2 binary. (default: "10MiB")
--get-distrib value     The amount of GET operations. (default: 45)
--stat-distrib value    The amount of STAT operations. (default: 30)
--put-distrib value     The amount of PUT operations. (default: 15)
--delete-distrib value  The amount of DELETE operations. Must be at least the same as PUT. (default: 10)
--obj.generator value   Use specific data generator (default: "random")
--obj.randsize          Randomize size of objects so they will be up to the specified size
--benchdata value       Output benchmark+profile data to this file. By default unique filename is generated.
--serverprof value      Run MinIO server profiling during benchmark; possible values are 'cpu', 'mem', 'block', 'mutex' and 'trace'.
--duration value        Duration to run the benchmark. Use 's' and 'm' to specify seconds and minutes. (default: 5m0s)
--autoterm              Auto terminate when benchmark is considered stable.
--autoterm.dur value    Minimum duration where output must have been stable to allow automatic termination. (default: 10s)
--autoterm.pct value    The percentage the last 6/25 time blocks must be within current speed to auto terminate. (default: 7.5)
--noclear               Do not clear bucket before or after running benchmarks. Use when running multiple clients.
--syncstart value       Specify a benchmark start time. Time format is 'hh:mm' where hours are specified in 24h format, server TZ.
--warp-client value     Connect to warp clients and run benchmarks there.
--analyze.dur value     Split analysis into durations of this length. Can be '1s', '5s', '1m', etc.
--analyze.out value     Output aggregated data as to file
--analyze.op value      Only output for this op. Can be GET/PUT/DELETE, etc.
--analyze.host value    Only output for this host.
--analyze.skip value    Additional duration to skip when analyzing data. (default: 0s)
--analyze.v             Display additional analysis data.
--serve value           When running benchmarks open a webserver to fetch results remotely, eg: localhost:7762
--help, -h              show help

结果

Uploading 2500 objects of Random data; 4194304 bytes totalwarp: Starting benchmark in 3s...
warp: Benchmark starting...
Throughput 101.0 objects/s within 7.500000% for 10.262s. Assuming stability. Terminating benchmark.
warp: Saving benchmark data...
warp: Benchmark data written to "warp-mixed-2022-04-18[030853]-nm3V.csv.zst"
Mixed operations.
Operation: DELETE, 10%, Concurrency: 20, Ran 36s.
* Throughput: 33.90 obj/s
Operation: GET, 45%, Concurrency: 20, Ran 36s.
* Throughput: 610.60 MiB/s, 152.65 obj/s
Operation: PUT, 15%, Concurrency: 20, Ran 36s.
* Throughput: 203.71 MiB/s, 50.93 obj/s
Operation: STAT, 30%, Concurrency: 20, Ran 36s.
* Throughput: 101.58 obj/s
Cluster Total: 813.56 MiB/s, 338.78 obj/s over 37s.

结语

以上就是使用wrap测试oss的其中一种办法，你学会了吗？希望读者朋友看完这篇文章后有所启发。