https://openanolis.cn/sig/KeenTune/doc/423701433171625827
准备两台虚拟机，VM A安装wrk，keentuned，keentune-brain，keentune-bench，作为测试机；VM B安装Nginx，作为Nginx服务端，同时安装KeenTune-target，为待测机器。
注：请确保两台虚拟机网络互通
在VM A上安装wrk测试工具
在VM B上安装和启动nginx服务
yum install -y epel-release
yum install -y nginx
vi /etc/nginx/nginx.conf
events {
#worker_connections 1024;
worker_connections 65535;
}
http {
...
#access_log /var/log/nginx/access.log main;
access_log off;
...
}
#go编译器安装
wget https://storage.googleapis.com/golang/go1.15.6.linux-amd64.tar.gz
tar -C /usr/local -xzvf ./go1.15.6.linux-amd64.tar.gz
ln -s /usr/local/go/bin/* /usr/bin/
export PATH=$PATH:/usr/local/go/bin
#验证go编译器
go env
go version
#配置go环境
go env -w GO111MODULE=on
go env -w GOPROXY=https://goproxy.cn,direct
git clone https://codeup.openanolis.cn/codeup/keentune/keentuned.git
cd keentuned
go mod vendor
cd keentuned
./keentuned_install.sh
keentuned
# 安装python36和python36开发包（提供头文件和静态链接库）
yum install python36 python36-devel rust openssl-devel git
# 安装应用所需的Python依赖包
pip3 install tornado==6.1
git clone https://codeup.openanolis.cn/codeup/keentune/keentune_target.git
cd keentune_target
sudo python3 setup.py install
keentune-target
# 安装python36和python36开发包（提供头文件和静态链接库）
yum install python36 python36-devel rust openssl-devel git
# 安装应用所需的Python依赖包
pip3 install tornado==6.1
git clone https://codeup.openanolis.cn/codeup/keentune/keentune_bench.git
cd keentune_bench
python3 setup.py install
keentune-bench
# 安装python36和python36开发包（提供头文件和静态链接库）
yum install python36 python36-devel rust openssl-devel git
# 安装应用所需的Python依赖包，最好先更新一下pip：pip3 install --upgrade pip
pip3 install numpy==1.19.5 POAP==0.1.26 tornado==6.1 hyperopt==0.2.5 ultraopt==0.1.1 bokeh==2.3.2 requests==2.25.1 pySOT==0.3.3 scikit_learn==0.24.2 paramiko==2.7.2 PyYAML==5.4.1
git clone https://codeup.openanolis.cn/codeup/keentune/keentune_brain.git
cd keentune_brain
sudo python3 setup.py install
keentune-brain
nginx_conf参数域使用的是/etc/nginx/nginx.conf文件中定义的nginx配置参数 默认安装路径：/etc/keentune/parameter/nginx.json
更多参数配置文件写法参考《KeenTune自定义调优参数》
定义benchmark分为benchmark脚本和benchmark配置两部分
benchmark脚本中实现benchmark工具的封装，本例中执行wrk命令并解析输出。
执行的wrk命令为
wrk -t 10 -c 300 -d 1 --latency [ip]
选择Latency_90，Latency_99，Requests_sec，Transfer_sec四个benchmark指标作为输出
配置文件中定义了执行benchmark脚本的命令，benchmark脚本的位置以及benchmark脚本输出指标的权重, 本例中为吞吐量调优配置，Requests_sec权重设置为100，其他为0
更多bench配置文件写法参考《KeenTune自定义benchmark》
keentune param tune --param nginx.json --bench bench_wrk_nginx_long.json --job nginx_demo --iteration 10
使用 keentune sensitize collect 命令采集数据
keentune sensitize collect --param sysctl.json --bench bench_wrk_nginx_long.json --data demo --iteration 100
使用 keentune sensitize list 命令可以看到所有可以用于敏感参数识别的数据
>> keentune sensitize list
Get sensitive parameter identification results successfully, and the details displayed in the terminal.
+------------+----------------------+-----------+
|data name |application scenario |algorithm |
+------------+----------------------+-----------+
|nginxtest1 |tuning |TPE |
|nginxtest |tuning |TPE |
|demo |tuning |TPE |
+------------+----------------------+-----------+
使用 keentune sensitize train 命令执行敏感参数识别
>> keentune sensitize train --data demo --output demo --trials 2
[ok] Running Sensitize Train Success.
trials: 2
data: demo
output: demo
see more detailsby log file: "/var/log/keentuned-sensitize-train-1637228899.log"
日志文件中可以看到敏感参数识别结果
| parameter name | sensitivity ratio |
| vm.max_map_count | 0.02636468 |
| vm.min_free_kbytes | 0.024904983 |
| net.ipv4.tcp_fin_timeout | 0.023090081 |
| net.ipv4.tcp_moderate_rcvbuf | 0.023066513 |
| vm.page-cluster | 0.020621613 |
| kernel.shmall | 0.020550262 |
| net.ipv4.conf.all.accept_redirects | 0.019789968 |
| vm.watermark_scale_factor | 0.019286443 |
| kernel.msgmni | 0.019159662 |
| net.ipv4.tcp_syn_retries | 0.017952599 |
| kernel.shmmni | 0.017676342 |
| net.core.netdev_budget | 0.01709693 |
| kernel.dmesg_restrict | 0.016918708 |
| net.ipv4.tcp_ecn | 0.01632732 |
| net.ipv4.tcp_no_metrics_save | 0.016036037 |
| net.ipv4.route.max_size | 0.015378745 |
| net.ipv4.tcp_fastopen | 0.014844485 |
| net.ipv4.tcp_window_scaling | 0.014674037 |
| vm.overcommit_ratio | 0.014470241 |
| net.core.wmem_default | 0.014349915 |
我们可以根据敏感参数筛选的结果对参数进行选择
这一步我们将会把Nginx_conf参数和sysctl经过筛选的参数写到一个参数配置文件中进行参数混合调优（生成参数混合配置文件的操作目前只能手动进行，文件下文已经提供）。 将参数配置文件放到/etc/keentune/parameter目录下，运行参数调优即可。
Nginx参数+敏感内核参数混合参数配置文件：
{
"nginx_conf": {
"worker_rlimit_nofile": {
"desc": "默认值：500",
"dtype": "int",
"range": [
300,
655350
]
},
"worker_connections": {
"desc": "默认值：500",
"dtype": "int",
"range": [
300,
160000
]
},
"multi_accept": {
"desc": "默认值：off",
"dtype": "string",
"options": [
"on",
"off"
]
},
"sendfile": {
"desc": "默认值：on",
"dtype": "string",
"options": [
"on",
"off"
]
},
"tcp_nopush": {
"desc": "默认值：on",
"dtype": "string",
"options": [
"on",
"off"
]
},
"tcp_nodelay": {
"desc": "默认值：on",
"dtype": "string",
"options": [
"on",
"off"
]
},
"keepalive_timeout": {
"desc": "默认值：30",
"dtype": "int",
"range": [
10,
100
]
},
"keepalive_requests": {
"desc": "默认值：100",
"dtype": "int",
"range": [
10,
1000
]
},
"client_body_timeout": {
"desc": "默认值：30",
"dtype": "int",
"range": [
10,
100
]
},
"send_timeout": {
"desc": "默认值：30",
"dtype": "int",
"range": [
10,
100
]
},
"client_body_buffer_size": {
"desc": "默认值：64k",
"dtype": "string",
"options": [
"8k",
"16k",
"32k",
"64k",
"128k",
"526k",
"1024k"
]
},
"client_max_body_size": {
"desc": "默认值：32m",
"dtype": "string",
"options": [
"4m",
"8m",
"16m",
"32m",
"64m",
"128m",
"256m"
]
},
"client_header_buffer_size": {
"desc": "默认值：4k",
"dtype": "string",
"options": [
"2k",
"4k",
"8k",
"16k",
"32k",
"64k"
]
},
"client_header_timeout": {
"desc": "默认值：30",
"dtype": "int",
"range": [
10,
100
]
},
"open_file_cache": {
"desc": "默认值：max=10240 inactive=10s",
"dtype": "string",
"options": [
"max=1024 inactive=10s",
"max=10240 inactive=10s"
]
},
"open_file_cache_valid": {
"desc": "默认值：30s",
"dtype": "string",
"options": [
"10s",
"30s",
"60s",
"90s",
"120s",
"150s"
]
},
"open_file_cache_min_uses": {
"desc": "默认值：2",
"dtype": "int",
"range": [
2,
10
]
},
"gzip": {
"desc": "默认值：off",
"dtype": "string",
"options": [
"on",
"off"
]
},
"gzip_min_length": {
"desc": "默认值：1024",
"dtype": "int",
"range": [
1024,
102400
]
},
"gzip_buffers": {
"desc": "默认值：4 4k",
"dtype": "string",
"options": [
"4 4k",
"4 8k",
"8 8k",
"16 8k",
"16 16k",
"16 32k"
]
},
"gzip_comp_level": {
"desc": "默认值：5",
"dtype": "int",
"range": [
1,
9
]
},
"ssl_session_cache": {
"desc": "默认值：shared:SSL:1m",
"dtype": "string",
"options": [
"off",
"none",
"builtin:1000",
"shared:SSL:1m"
]
},
"ssl_session_timeout": {
"desc": "默认值：5m",
"dtype": "string",
"options": [
"2m",
"5m",
"10m",
"20m"
]
},
"proxy_connect_timeout": {
"desc": "默认值：5",
"dtype": "int",
"range": [
1,
50
]
},
"proxy_read_timeout": {
"desc": "默认值：60",
"dtype": "int",
"range": [
6,
600
]
},
"proxy_send_timeout": {
"desc": "默认值：60",
"dtype": "int",
"range": [
6,
600
]
},
"ssl_buffer_size": {
"desc": "默认值：64",
"dtype": "string",
"options": [
"64",
"128",
"526",
"4k",
"64k",
"256k",
"1m",
"4m"
]
}
},
"sysctl": {
"vm.max_map_count":{
"desc":"Defines the maximum memory area that a process can have.",
"dtype":"int",
"range":[
100000,
10000000
],
"step":100000
},
"vm.min_free_kbytes":{
"desc":"Size of memory reserved in each memory area, in KB.",
"dtype":"int",
"range":[
10240,
1024000
],
"step":10240
},
"net.ipv4.tcp_fin_timeout":{
"desc":"Maximum duration for a socket to remain in the FIN_WAIT_2 state.",
"dtype":"int",
"range":[
1,
120
],
"step":1
},
"net.ipv4.tcp_moderate_rcvbuf":{
"desc":"Whether to adjust the receive buffer when receiving data 0: no adjustment 1: yes",
"dtype":"string",
"needrestart":"false",
"options":[
"0",
"1"
]
},
"vm.page-cluster":{
"desc":"Number of pages written to the swap partition each time.",
"dtype":"int",
"range":[
0,
8
],
"step":1
},
"kernel.shmall":{
"desc":"The total amount of shared memory available on the system in bytes",
"dtype":"int",
"range":[
1073741824,
8589934592
],
"step":1073741824
},
"net.ipv4.conf.all.accept_redirects":{
"desc":"Receives and sends ICMP redirection messages. The default value is True for hosts and False for routers. 0: disabled 1: yes",
"dtype":"string",
"needrestart":"false",
"options":[
"0",
"1"
]
},
"vm.watermark_scale_factor":{
"desc":"Controls the radical degree of the kswapd process, that is, how much memory needs to be released for the system (NUMA node) from wakeup to hibernation.",
"dtype":"int",
"range":[
10,
1000
],
"step":10
},
"kernel.msgmni":{
"desc":"System Message Queue Length",
"dtype":"int",
"range":[
8000,
128000
],
"step":8000
},
"net.ipv4.tcp_syn_retries":{
"desc":"Number of times that the local device retransmits TCP SYN packets due to timeout. The value cannot be greater than 255. This parameter is valid only for outgoing connections. For incoming connections, this parameter is controlled by tcp_retries1.",
"dtype":"int",
"range":[
3,
64
],
"step":1
},
"kernel.shmmni":{
"desc":"Maximum number of shared memory segments in the system.",
"dtype":"int",
"range":[
1024,
16384
],
"step":1024
},
"net.core.netdev_budget":{
"desc":"Number of network packets processed in each software interrupt",
"dtype":"int",
"range":[
100,
1000
],
"step":100
},
"kernel.dmesg_restrict":{
"desc":"Which users are restricted from viewing syslogs? 0: no restriction 1: Only privileged users can view the information.",
"dtype":"string",
"needrestart":"false",
"options":[
"0",
"1"
]
},
"net.ipv4.tcp_ecn":{
"desc":"Indicates whether to enable the direct TCP congestion notification function. 0: disabled; 1: enabled",
"dtype":"string",
"needrestart":"false",
"options":[
"0",
"1",
"2"
]
},
"net.ipv4.tcp_no_metrics_save":{
"desc":"After a TCP connection is closed, the saved parameters can be used to initialize the connection when the same connection is created next time as long as dst_entry is valid.",
"dtype":"string",
"needrestart":"false",
"options":[
"0",
"1"
]
},
"net.ipv4.route.max_size":{
"desc":"Maximum number of entries in the routing cache. If the number of entries in the routing cache exceeds the maximum, the old entries are cleared.",
"dtype":"int",
"range":[
67108864,
2080374784
],
"step":67108864
},
"net.ipv4.tcp_fastopen":{
"desc":"Whether to enable the TCP quick open mode is to avoid the three-way handshake of hot requests. This greatly improves the performance in the scenario where small objects are moved.",
"dtype":"string",
"needrestart":"false",
"options":[
"1",
"2",
"4"
]
},
"net.ipv4.tcp_window_scaling":{
"desc":"The window scaling defined in RFC 1323 is enabled. To support a TCP window larger than 64 KB, this parameter must be set to 1. The maximum size of a TCP window is 1 GB. This parameter takes effect only when both parties of a TCP connection are enabled.",
"dtype":"string",
"needrestart":"false",
"options":[
"0",
"1"
]
},
"vm.overcommit_ratio":{
"desc":"When overcommit_memory is set to 2, the percentage of physical RAM that is considered is set.",
"dtype":"int",
"range":[
0,
100
],
"step":10
},
"net.core.wmem_default":{
"desc":"Default TCP send window size (bytes)",
"dtype":"int",
"range":[
8192,
1048576
],
"step":8192
}
}
}