goofys 鲲鹏上面编译挂载与性能测试

goofys,鲲鹏,上面,编译,挂载,性能,测试 · 浏览次数 : 15

小编点评

**goofys编译和性能测试介质使用go进行编译** **步骤 1:下载并编译 goofys 库** - 使用 GitHub 上的 issue 下载并编译 goofys 库: ``` git clone https://github.com/kahing/goofys.git cd goofys make build ``` **步骤 2:设置环境变量** - 设置以下环境变量: ``` GOPROXY=https://goproxy.io GOPATH=/root/goexport GOOFYS_HOME=/root/go/src/github.com/kahing/goofys/go install /root/go/src/github.com/kahing/goofys/PATH=$PATH:/root/go/bin ``` **步骤 3:运行 goofys** - 使用以下命令运行 goofys: ``` goofys [global options] ``` **步骤 4:测试性能** - 使用以下命令测试性能: ``` goofys -Mount an S3 bucket locally ``` **步骤 5:查看性能结果** - 监控性能结果并查看复制速度、错误数量等信息。 **注意:** - 使用 `--debug_fuse` 和 `--debug_s3` 命令选项可以生成更详细的日志信息。 - `--use-content-type` 设置文件扩展名的 Content-Type,并使用 `/etc/mime.types` 文件中的内容设置默认值。 - `--no-implicit-dir` 选项可以强制使用 `goofys mount` 命令创建所有目录树。

正文

goofys 鲲鹏上面编译挂载与性能测试


介质

使用go进行编译.
官网上面有 amd64的介质,但是没有aarch64的介质 
需要自行编译
前几天一直编译失败. 周天在家自己测试了一把,根据github上面issue上面的描述编译成功了

发现的确比s3fs的性能要好, 计划下周进行一下与研发的联合测试工作. 

境内代理

export GOPROXY=https://goproxy.io

编译安装

go install  github.com/kahing/goofys@latest

mkdir -p go/src/github.com/kahing

cd go/src/github.com/kahing/
git clone https://github.com/kahing/goofys.git

也可以手工将zip包挪过来.

export GOPATH=/root/go
export GOOFYS_HOME=/root/go/src/github.com/kahing/goofys/

go install /root/go/src/github.com/kahing/goofys/

PATH=$PATH:/root/go/bin; export PATH
goofys --version
Hope this helps.

挂载

mkdir -p $HOME/.aws
cat >> $HOME/.aws/credentials << EOF
[default]
aws_access_key_id = minioadmin
aws_secret_access_key = minioadmin
EOF

mkdir /minio_goofys
goofys --debug_fuse --debug_s3  --endpoint http://192.168.255.45:9000  miniovoucher /minio_goofys


简单测试结果

写入性能

不带缓存
1G
1048576000字节(1.0 GB,1000 MiB)已复制,2.09474 s,501 MB/s
100M
104857600字节(105 MB,100 MiB)已复制,0.239955 s,437 MB/s
20M
20971520字节(21 MB,20 MiB)已复制,0.0931701 s,225 MB/s
带缓存
1G
1048576000字节(1.0 GB,1000 MiB)已复制,2.09269 s,501 MB/s
100M
104857600字节(105 MB,100 MiB)已复制,0.250175 s,419 MB/s
20M
20971520字节(21 MB,20 MiB)已复制,0.0845809 s,248 MB/s
本地文件系统
629145600字节(629 MB,600 MiB)已复制,0.167972 s,3.7 GB/s

读取性能

不带缓存
103875056字节(104 MB,99 MiB)已复制,0.162691 s,638 MB/s
带缓存
103875056字节(104 MB,99 MiB)已复制,0.0227822 s,4.6 GB/s

帮助效果

NAME:
   goofys - Mount an S3 bucket locally

USAGE:
   goofys [global options] bucket[:prefix] mountpoint

VERSION:
   0.24.0-use `make build' to fill version hash correctly

GLOBAL OPTIONS:
   -o value            Additional system-specific mount options. Be careful!
   --cache value       Directory to use for data cache. Requires catfs and `-o allow_other'. Can also pass in other catfs options (ex: --cache "--free:10%:$HOME/cache") (default: off)
   --dir-mode value    Permission bits for directories. (default: 0755) (default: 493)
   --file-mode value   Permission bits for files. (default: 0644) (default: 420)
   --uid value         UID owner of all inodes. (default: 0)
   --gid value         GID owner of all inodes. (default: 0)
   --endpoint value    The non-AWS endpoint to connect to. Possible values: http://127.0.0.1:8081/
   --profile value     Use a named profile from $HOME/.aws/credentials instead of "default"
   --use-content-type  Set Content-Type according to file extension and /etc/mime.types (default: off)
   --subdomain         Enable subdomain mode of S3

TUNING OPTIONS:
   --cheap                 Reduce S3 operation costs at the expense of some performance (default: off)
   --no-implicit-dir       Assume all directory objects ("dir/") exist (default: off)
   --stat-cache-ttl value  How long to cache StatObject results and inode attributes. (default: 1m0s)
   --type-cache-ttl value  How long to cache name -> file/dir mappings in directory inodes. (default: 1m0s)
   --http-timeout value    Set the timeout on HTTP requests to S3 (default: 30s)

AWS S3 OPTIONS:
   --region value         The region to connect to. Usually this is auto-detected. Possible values: us-east-1, us-west-1, us-west-2, eu-west-1, eu-central-1, ap-southeast-1, ap-southeast-2, ap-northeast-1, sa-east-1, cn-north-1 (default: "us-east-1")
   --requester-pays       Whether to allow access to requester-pays buckets (default: off)
   --storage-class value  The type of storage to use when writing objects. Possible values: REDUCED_REDUNDANCY, STANDARD, STANDARD_IA. (default: "STANDARD")
   --sse                  Enable basic server-side encryption at rest (SSE-S3) in S3 for all writes (default: off)
   --sse-kms key-id       Enable KMS encryption (SSE-KMS) for all writes using this particular KMS key-id. Leave blank to Use the account's CMK - customer master key (default: off)
   --sse-c value          Enable server-side encryption using this base64-encoded key (default: off)
   --acl value            The canned ACL to apply to the object. Possible values: private, public-read, public-read-write, authenticated-read, aws-exec-read, bucket-owner-read, bucket-owner-full-control (default: off)

MISC OPTIONS:
   --help, -h     Print this help text and exit successfully.
   --debug_fuse   Enable fuse-related debugging output.
   --debug_s3     Enable S3-related debugging output.
   -f             Run goofys in foreground.
   --version, -v  print the version

与goofys 鲲鹏上面编译挂载与性能测试相似的内容:

goofys 鲲鹏上面编译挂载与性能测试

# goofys 鲲鹏上面编译挂载与性能测试 ## 介质 ``` 使用go进行编译. 官网上面有 amd64的介质,但是没有aarch64的介质 需要自行编译 前几天一直编译失败. 周天在家自己测试了一把,根据github上面issue上面的描述编译成功了 发现的确比s3fs的性能要好, 计划下周进

[转帖]goofys

Goofys is a high-performance, POSIX-ish Amazon S3 file system written in Go Overview Goofys allows you to mount an S3 bucket as a filey system. It's a

[转帖]使用 goofys 挂载 S3 bucket 为文件系统

https://xie.infoq.cn/article/7f178e0a1315f758d77c6c2bb 背景 公司的 gitlab 目前都是直接存储在物理盘上,为了确保数据不会丢失,需要重复多次备份到不同的地方,备份的过程中会有多次 IO,影响机器的性能,且随着数据量的增长,备份所需的时间也越

[转帖]编译安装goofys挂载Scaleway免费75G对象存储

日常•2022年5月29日 goofys编译 goofys是一个开源的使用Go编写的s3存储桶挂载工具,主打高性能。由于使用Go编写,没有用到什么特别的依赖,自己编译也很容易。截止2022.5.27,官方github仓库貌似一直有提交,但是提供的预编译安装包貌似只到2020年4月,而且只有x86版本

[转帖]gooyfs 的编译 github

https://github.com/kahing/goofys/issues/527 @maobaolong @PengleiShi I had the same issue as you guys, new to git and had to work it out. Here is what