[转帖]k8s nginx-ingress上的配置优化

k8s,nginx,ingress,配置,优化 · 浏览次数 : 0

小编点评

**优化配置部分** ```yaml access-log-path: /var/log/nginx/access.log disable-access-log: "false" disable-ipv6: "true" disable-ipv6-dns: "true" enable-modsecurity: "false" enable-multi-accept: "true" enable-opentracing: "true" error-log-level: notice error-log-path: /var/log/nginx/error.log generate-request-id: "true" keep-alive: "60" keep-alive-requests: "10000" load-balance: round_robin log-format-escape-json: "true" max-worker-connections: "65535" max-worker-open-files: "10240" nginx-status-ipv4-whitelist: 0.0.0.0 reuse-port: "true" upstream-keepalive-connections: "200" upstream-keepalive-requests: "100" upstream-keepalive-timeout: "60" worker-processes: "4" ``` **其他建议** * 使用 `nginx-ingress-controller` 的默认配置，无需进行过多设置。 * 可以根据实际情况调整 `access-log-path`、`error-log-path`、`upstream_keepalive_connections` 等参数。 * 在 `initContainers` 中设置 `sysctl` 参数可以优化内核参数。

正文

https://cloud.tencent.com/developer/article/1932188?areaSource=&traceId=

自建K8s上，如果部署了Nginx-Ingress，通常一些默认的参数有些可能需要优化下以便提升它的性能（阿里云之类的云厂商提供的Ingress是优化过的）。

我下面是自建的测试K8s上部署的ingress，这里贴下优化的地方：

kubectl get cm -n ingress-nginx

NAME                              DATA   AGE
ingress-controller-leader-nginx   0      6d19h
ingress-nginx-controller          0      6d19h
nginx-ingress-controller          22     5d20h

1、修改些nginx的常用参数

kubectl get cm -n ingress-nginx nginx-ingress-controller -oyaml 下面是data的内容，部分可以再根据实际情况修改（例如日志文件的路径）

apiVersion: v1
data:
  access-log-path: /var/log/nginx/access.log
  disable-access-log: "false"
  disable-ipv6: "true"
  disable-ipv6-dns: "true"
  enable-modsecurity: "false"
  enable-multi-accept: "true"
  enable-opentracing: "true"
  error-log-level: notice
  error-log-path: /var/log/nginx/error.log
  generate-request-id: "true"
  keep-alive: "60"
  keep-alive-requests: "10000"
  load-balance: round_robin
  log-format-escape-json: "true"
  max-worker-connections: "65535"
  max-worker-open-files: "10240"
  nginx-status-ipv4-whitelist: 0.0.0.0
  reuse-port: "true"
  upstream-keepalive-connections: "200"
  upstream-keepalive-requests: "100"
  upstream-keepalive-timeout: "60"
  worker-processes: "4"
kind: ConfigMap
.... 其余部分忽略....

2、修改下nginx容器的内核参数

kubectl get deployments -n ingress-nginx -oyaml 注意这里加了个initContainers的配置，主要是修改一些内核参数，更适配Nginx的使用场景

        dnsPolicy: ClusterFirst
        initContainers:
        - command:
          - /bin/sh
          - -c
          - |
            mount -o remount rw /proc/sys
            sysctl -w net.core.somaxconn=65535
            sysctl -w net.ipv4.tcp_tw_reuse=1
            sysctl -w net.ipv4.ip_local_port_range="1024 65535"
            sysctl -w fs.file-max=1048576
            sysctl -w fs.inotify.max_user_instances=16384
            sysctl -w fs.inotify.max_user_watches=524288
            sysctl -w fs.inotify.max_queued_events=16384
          image: busybox
          imagePullPolicy: IfNotPresent
          name: init-sysctl
          resources: {}
          securityContext:
            capabilities:
              add:
              - SYS_ADMIN
              drop:
              - ALL
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
        nodeSelector:
          kubernetes.io/os: linux
        restartPolicy: Always
        schedulerName: default-scheduler

参考：

https://zhuanlan.zhihu.com/p/212620792 
https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/ 
https://www.nginx.com/blog/tuning-nginx/   
https://mp.weixin.qq.com/s/NAwz4dlsPuJnqfWYBHkfGg