[转帖]nginx如何实现负载均衡、限流、缓存、黑白名单和灰度发布

nginx,如何,实现,负载,均衡,限流,缓存,黑白,名单,灰度,发布 · 浏览次数 : 0

小编点评

```lua -- shared state for blacklist ip_blacklist = 1m -- server for listening on port 80 server { listen 80; location / { access_by_lua_file lua/ip_blacklist.lua; proxy_pass http://$group; } lua { -- script for listening on port 80 -- and setting blacklist entry local redis_host = \"127.0.0.1\" local redis_port = 6379 local redis_pwd = 123456 local redis_db = 2-- connection timeout for redis in ms. local redis_connection_timeout = 100-- a set key for blacklist entries local redis_key = \"ip_blacklist\"-- cache lookups for this many seconds local cache_ttl = 60-- end configuration -- update blacklist from redis every cache_ttl seconds if last_update_time == nil or last_update_time < ( ngx.now() - cache_ttl ) then local redis = require \"resty.redis\"; local red = redis:new(); red:set_timeout(redis_connection_timeout); local ok, err = red:connect(redis_host, redis_port); if not ok then ngx.log(ngx.ERR, "Redis connection error while connect: " .. err) else local ok, err = red:auth(redis_pwd) if not ok then ngx.log(ngx.ERR, "Redis password error while auth: " .. err) else local new_ip_blacklist, err = red:smembers(redis_key); if err then ngx.log(ngx.ERR, "Redis read error while retrieving ip_blacklist: " .. err) else ngx.log(ngx.ERR, "Get data success:\" .. new_ip_blacklist) -- replace the locally stored ip_blacklist with the updated values: ip_blacklist:flush_all() for index, banned_ip in ipairs(new_ip_blacklist) do ip_blacklist:set(banned_ip, true) end -- update time ip_blacklist:set("last_update_time", ngx.now()) end end end end } } -- second server for listening on port 80 server { set $group default; if ($remote_addr ~ \"192.168.119.1\") { set $group host1; } if ($remote_addr ~ \"192.168.119.2\") { set $group host2; } else { server { set $group default; if ($remote_addr ~ \"192.168.119.2\") { set $group host2; } } } lua { -- script for listening on port 80 -- and setting blacklist entry local redis_host = \"127.0.0.1\" local redis_port = 6379 local redis_pwd = 123456 local redis_db = 2-- connection timeout for redis in ms. local redis_connection_timeout = 100-- a set key for blacklist entries local redis_key = \"ip_blacklist\"-- cache lookups for this many seconds local cache_ttl = 60-- end configuration -- update blacklist from redis every cache_ttl seconds if last_update_time == nil or last_update_time < ( ngx.now() - cache_ttl ) then local redis = require \"resty.redis\"; local red = redis:new(); red:set_timeout(redis_connection_timeout); local ok, err = red:connect(redis_host, redis_port); if not ok then ngx.log(ngx.ERR, "Redis connection error while connect: " .. err) else local ok, err = red:auth(redis_pwd) if not ok then ngx.log(ngx.ERR, "Redis password error while auth: " .. err) else local new_ip_blacklist, err = red:smembers(redis_key); if err then ngx.log(ngx.ERR, "Redis read error while retrieving ip_blacklist: " .. err) else ngx.log(ngx.ERR, "Get data success:\" .. new_ip_blacklist) -- replace the locally stored ip_blacklist with the updated values: ip_blacklist:flush_all() for index, banned_ip in ipairs(new_ip_blacklist) do ip_blacklist:set(banned_ip, true) end -- update time ip_blacklist:set("last_update_time", ngx.now()) end end end end } } ```

正文

https://zhuanlan.zhihu.com/p/464491494

挺好的文章.

nginx负载均衡配置

1.负载均衡配置

http {
    upstream real_server {
       server 192.168.1.100:8082 weight=1;  #轮询服务器和访问权重
       server 192.168.1.100:8084 weight=2;
    }
    server {
        listen  80;
        location / {
            proxy_pass http://real_server;
        }
    }
}

2.失败重试配置

 upstream real_server {
    server 192.168.1.100:8082 weight=1 max_fails=2 fail_timeout=60s;  #轮询服务器和访问权重
    server 192.168.1.100:8084 weight=2 max_fails=2 fail_timeout=60s;
 }

在fail_timeout时间内失败了max_fails次请求后,认为上游服务器不可用,就会将服务地址剔除掉,fail_timeout时间后会再次将服务器加入存活列表进行重试。

nginx限流配置

limit_req_zone指令设置参数

limit_req_zone $binary_remote_addr zone=mylimit:10m rate=10r/s;

参数说明

limit_req_zone定义在http块中，$binary_remote_addr表示保存客户端IP地址的二进制形式。

Zone定义IP状态及URL访问频率的共享内存区域。zone=keyword标识区域的名字，以及冒号后面跟区域大小。16000个IP地址的状态信息约1MB，例子区域可以存储160000个IP地址。

Rate定义最大请求速率。示例中速率不能超过每秒10个请求。

设置限流

location / {
        limit_req zone=user burst=20 nodelay;
        proxy_pass http://real_server;
}

burs排队大小,nodelay不限制单个请求间的时间。具体使用可以查看高并发场景如何使用nginx实现限流-实战篇

不限流白名单

geo $limit {
  default              1;
  192.168.1.0/24  0;
}
map $limit $limit_key {
  1 $binary_remote_addr;
  0 "";
}
limit_req_zone $limit_key zone=mylimit:10m rate=1r/s;
location / {
        limit_req zone=mylimit burst=1 nodelay;
        proxy_pass http://real_server;
}

该配置说明 192.168.1.0/24网段的ip访问是不限流的,其它限流。ip后面数字的含义

24表示子网掩码:255.255.255.0

16表示子网掩码:255.255.0.0

8表示子网掩码:255.0.0.0

nginx缓存设置

1.浏览器缓存静态资源缓存用expire

location ~*  .(jpg|jpeg|png|gif|ico|css|js)$ {
   expires 2d;
}

Response Header中添加了Expires和Cache-Control

所谓的静态资源一般包括一直不变的图像,如网站的logo,js、css静态文件还有可下载的内容，媒体文件

协商缓存(add_header ETag/Last-Modified value)包括html文件,经常替换的图片,经常需要修改的js、css文件和基本不变的api接口

不需要缓存包括用户隐私等敏感数据,用户经常变动的api接口

2.代理层缓存

//缓存路径，inactive表示缓存的时间，到期之后将会把缓存清理
proxy_cache_path /data/cache/nginx/ levels=1:2 keys_zone=cache:512m inactive = 1d max_size=8g;

location / {
    location ~ \.(htm|html)?$ {
        proxy_cache cache;
        proxy_cache_key    $uri$is_args$args;  //以此变量值做HASH，作为KEY
        //HTTP响应首部可以看到X-Cache字段，内容可以有HIT,MISS,EXPIRES等等
        add_header X-Cache $upstream_cache_status;
        proxy_cache_valid 200 10m;
        proxy_cache_valid any 1m;
        proxy_pass  http://real_server;
        proxy_redirect     off;
    }
    location ~ .*\.(gif|jpg|jpeg|bmp|png|ico|txt|js|css)$ {
        root /data/webapps/edc;
        expires      3d;
        add_header Static Nginx-Proxy;
    }
}

在本地磁盘创建一个文件目录,根据我们的配置把请求资源以k(key自定义,这边用url的hash值)->v形式缓存到目录里,并根据需求对内容设置缓存时长,比如状态码为200缓存10分钟,其余的缓存1分钟等待。要清理缓存可以借助purger的功能。如果ab测试/个性化需求时应禁用浏览器缓存,否则会因为缓存导致误差。

nginx黑名单

方式一

location / {
    deny  192.168.1.1;
    deny 192.168.1.0/24;
    allow 10.1.1.0/16;
    allow 2001:0db8::/32;
    deny  all;
}

方式二 lua+redis动态黑名单(openresty)

yum install yum-utils
yum-config-manager --add-repo https://openresty.org/package/centos/openresty.repo
yum install openresty
yum install openresty-resty
查看
yum --disablerepo="*" --enablerepo="openresty" list available
运行
service openresty start

配置(/usr/local/openresty/nginx/conf/nginx.conf)

lua_shared_dict ip_blacklist 1m;
server {
    listen  80;
    location / {
        access_by_lua_file lua/ip_blacklist.lua;
        proxy_pass http://real_server;
    }
}

lua脚本编写(ip_blacklist.lua)

local redis_host    = "127.0.0.1"
local redis_port    = 6379
local redis_pwd     = 123456
local redis_db = 2

-- connection timeout for redis in ms.
local redis_connection_timeout = 100

-- a set key for blacklist entries
local redis_key     = "ip_blacklist"

-- cache lookups for this many seconds
local cache_ttl     = 60

-- end configuration

local ip                = ngx.var.remote_addr
local ip_blacklist      = ngx.shared.ip_blacklist
local last_update_time  = ip_blacklist:get("last_update_time");

-- update ip_blacklist from Redis every cache_ttl seconds:
if last_update_time == nil or last_update_time < ( ngx.now() - cache_ttl ) then

  local redis = require "resty.redis";
  local red = redis:new();

  red:set_timeout(redis_connect_timeout);

  local ok, err = red:connect(redis_host, redis_port);
  if not ok then
    ngx.log(ngx.ERR, "Redis connection error while connect: " .. err);
  else
    local ok, err = red:auth(redis_pwd)
    if not ok then
      ngx.log(ngx.ERR, "Redis password error while auth: " .. err);
    else
        local new_ip_blacklist, err = red:smembers(redis_key);
        if err then
            ngx.log(ngx.ERR, "Redis read error while retrieving ip_blacklist: " .. err);
        else
        ngx.log(ngx.ERR, "Get data success:" .. new_ip_blacklist)
          -- replace the locally stored ip_blacklist with the updated values:
            ip_blacklist:flush_all();
          for index, banned_ip in ipairs(new_ip_blacklist) do
            ip_blacklist:set(banned_ip, true);
          end
          -- update time
          ip_blacklist:set("last_update_time", ngx.now());
      end
    end
  end
end

if ip_blacklist:get(ip) then
  ngx.log(ngx.ERR, "Banned IP detected and refused access: " .. ip);
  return ngx.exit(ngx.HTTP_FORBIDDEN);
end

nginx灰度发布

1.根据cookie实现灰度发布

根据cooke查询version值，根据version跳转到对应的host,如果没有匹配上的就跳转到默认配置。

upstream host1 {
   server 192.168.2.100:2001 weight=1;  #轮询服务器和访问权重
   server 192.168.2.100:2002 weight=2;
}
upstream host2 {
   server 192.168.1.100:1111  max_fails=1 fail_timeout=60;
}
upstream default {
   server 192.168.1.100:1111  max_fails=1 fail_timeout=60;
}
map $COOKIE_version $group {
   ~*v1$ host1;
   ~*v2$ host2;
   default default;
}
lua_shared_dict ip_blacklist 1m;
server {
    listen  80;
    location / {
        access_by_lua_file lua/ip_blacklist.lua;
        proxy_pass http://$group;
    }
}

2.根据来路ip实现灰度发布

server {
  set $group default;
  if ($remote_addr ~ "192.168.119.1") {
      set $group host1;
  }
  if ($remote_addr ~ "192.168.119.2") {
      set $group host2;
  }