Nginx 几乎是每个互联网公司的接入层标配,但很多初中级运维的状态是:
这一篇要解决的是“合格运维对 Nginx 应有的知识地图”。覆盖:架构、配置、模块、HTTPS、反向代理、负载均衡、限流、日志、调优、监控、高可用、平滑升级、典型排障,最后落到 6 套生产级配置示例。
写作目标:
nginx.confNginx 是事件驱动的高性能 Web 服务器。核心模型是:
worker 的几个关键点:
accept_mutex)worker_connections)事件驱动模型的优点:
事件驱动模型的限制:
Nginx 用信号控制:
nginx -s stop
:发 SIGTERM,优雅停止nginx -s quit
:发 SIGQUIT,worker 处理完所有连接再退出nginx -s reload
:发 SIGHUP,master 重新加载配置,旧 worker 处理完连接后退出nginx -s reopen
:发 SIGUSR1,重新打开日志(logrotate 配合使用)kill -USR2
:热升级,新 master 启动kill -WINCH
:通知旧 master 关闭 worker热升级流程:
bash
# 备份旧 binary cp /usr/sbin/nginx /usr/sbin/nginx.old # 用新 binary 替换 cp /usr/local/nginx/sbin/nginx /usr/sbin/nginx # 触发热升级 kill -USR2 # 新 master 启动,旧 master 改名为 .old,worker 也变 .old # 旧 worker 继续处理现有连接,新 worker 接受新连接 # 等所有旧连接结束 kill -WINCH # 关闭旧 master kill -QUIT # 如果新版本有问题,回滚 kill -HUP # 新版本会平滑退出,回到旧版本处理 Nginx 配置是层级化的,每段都有自己作用域:
user、worker_processes、error_log、pid、worker_rlimit_nofileworker_connections、multi_accept、useinclude、default_type、log_format、access_log、sendfile、tcp_nopush、tcp_nodelay、keepalive_timeout、gzip、upstreamlisten、server_name、root、location、access_log、error_page、return、rewriteproxy_pass、try_files、root、alias、index、limit_req、auth_basicserver、ip_hash、least_conn、keepalive、health_check变量与执行顺序:
$ 开头,例如 $remote_addr、$http_user_agent、$arg_namehttp、server、location、if、map、geo、set 中被赋值http → server → location → if,子级覆盖父级Nginx 模块分为:
main、events、http 等基础指令http_core、http_log、http_gzip、http_ssl、http_proxy、http_upstream、http_rewrite、http_limit_req、http_limit_conn、http_auth_request、http_secure_link、http_stub_status、http_v2、http_slicestream_core、stream_proxy、stream_upstream、stream_ssl、stream_limit_conn、stream_log查看编译参数:
bash
nginx -V 2>&1 | tr ' ' ' ' | grep -E '^--' 第三方模块:
nginx
user www www; worker_processes auto; worker_rlimit_nofile65535; pid /var/run/nginx.pid; error_log /var/log/nginx/error.log warn; events { worker_connections10240; multi_accepton; useepoll; } http { include mime.types; default_type application/octet-stream; charset utf-8; server_tokensoff; client_max_body_size50m; client_body_buffer_size128k; client_header_buffer_size1k; large_client_header_buffers48k; sendfileon; tcp_nopushon; tcp_nodelayon; keepalive_timeout65; keepalive_requests1000; types_hash_max_size2048; server_names_hash_bucket_size128; proxy_http_version1.1; proxy_set_header Connection ""; log_format main '$remote_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent" ' 'rt=$request_time uct="$upstream_connect_time" ' 'urt="$upstream_response_time"'; access_log /var/log/nginx/access.log main buffer=32k flush=5s; open_file_cache max=10240 inactive=20s; open_file_cache_valid30s; open_file_cache_min_uses2; open_file_cache_errorson; gzipon; gzip_min_length1k; gzip_comp_level5; gzip_types text/plain text/css application/json application/javascript application/xml; include /etc/nginx/conf.d/*.conf; } nginx
server { listen80; server_name static.example.com; root /data/www/static; index index.html; location / { try_files$uri$uri/ =404; } location~* .(jpg|jpeg|png|gif|webp|avif)$ { expires30d; add_header Cache-Control "public, immutable"; access_logoff; } location~* .(css|js)$ { expires7d; access_logoff; } location~* .(mp4|webm|m4a)$ { mp4; mp4_buffer_size4m; mp4_max_buffer_size10m; } } 关键指令:
try_files $uri $uri/ =404
:依次尝试文件、目录、最后返回 404expires 30d
:浏览器缓存 30 天mp4
:启用 MP4 模块支持伪流access_log off
:关掉静态资源日志nginx
upstream backend { ip_hash; server10.0.0.1:8080 max_fails=3 fail_timeout=30s weight=1; server10.0.0.2:8080 max_fails=3 fail_timeout=30s weight=1; server10.0.0.3:8080 max_fails=3 fail_timeout=30s weight=1; keepalive64; keepalive_timeout60s; keepalive_requests1000; } server { listen80; server_name app.example.com; location / { proxy_pass http://backend; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_http_version1.1; proxy_set_header Connection ""; proxy_connect_timeout5s; proxy_send_timeout30s; proxy_read_timeout30s; proxy_bufferingon; proxy_buffer_size16k; proxy_buffers832k; proxy_busy_buffers_size64k; } } 负载均衡算法:
round-robin
(默认):轮询weight
:权重ip_hash
:按 IP 哈希,固定上游least_conn
:最少连接least_time
:最少响应时间(商业版)random
:随机hash $key consistent
:一致性哈希url_hash
:按 URL 哈希(第三方模块)健康检查:
max_fails、fail_timeouthealth_check(商业版)或 ngx_http_upstream_check_module(开源)nginx
server { listen443 ssl http2; server_name secure.example.com; ssl_certificate /etc/nginx/ssl/fullchain.pem; ssl_certificate_key /etc/nginx/ssl/privkey.pem; ssl_session_cache shared10m; ssl_session_timeout1d; ssl_session_ticketson; ssl_protocols TLSv1.2 TLSv1.3; ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384; ssl_prefer_server_cipherson; ssl_dhparam /etc/nginx/ssl/dhparam.pem; add_header Strict-Transport-Security "max-age=63072000" always; add_header X-Frame-Options "SAMEORIGIN"; add_header X-Content-Type-Options "nosniff"; add_header X-XSS-Protection "1; mode=block"; } 证书自动续签(Let’s Encrypt):
bash
# 安装 certbot apt install -y certbot python3-certbot-nginx # 申请证书 certbot --nginx -d example.com -d www.example.com # 自动续签(systemd timer) systemctl enable certbot.timer systemctl start certbot.timer # 手动测试 certbot renew --dry-run nginx
http { limit_req_zone$binary_remote_addr zone=perip:10m rate=10r/s; limit_req_zone$server_name zone=perserver:10m rate=1000r/s; limit_req_zone$http_x_api_key zone=perkey:10m rate=100r/s; server { location /api/ { limit_req zone=perip burst=20 nodelay; limit_req_status429; } } } 参数:
$binary_remote_addr
:按 IP$server_name
:按 server$http_x_api_key
:按 API Keyrate=10r/s
:每秒 10 个burst=20
:突发 20nodelay
:超过立即返回 429,不排队nginx
http { limit_conn_zone $binary_remote_addr zone=perip_conn:10m; server { location /download/ { limit_conn perip_conn 5; limit_conn_status 503; } } } nginx
http { geo$blocked { default0; 10.0.0.0/8 0; 192.168.1.0/24 1; 1.2.3.4 1; } map$http_user_agent$is_bot { default0; ~*bot1; ~*spider1; ~*crawler1; } server { if ($blocked) { return403; } if ($is_bot) { return403; } } } nginx
location /admin/ { allow10.0.0.0/8; allow192.168.0.0/16; deny all; auth_basic"Admin Area"; auth_basic_user_file /etc/nginx/.htpasswd; } location /api/ { auth_request /auth; } location = /auth { internal; proxy_pass http://auth-service/verify; proxy_set_header X-Original-URI $request_uri; } nginx
# 强制 HTTPS server { listen80; server_name example.com www.example.com; return301 https://$host$request_uri; } # 隐藏 .php 扩展 location / { rewrite ^/(.*).php$ /$1last; } # 旧 URL 跳转 location /old/path { return301 /new/path; } # 反盗链 location~* .(jpg|png|gif)$ { valid_referersnoneblocked example.com *.example.com; if ($invalid_referer) { return403; } }
return 与 rewrite 的区别:
return
直接返回响应,不再处理后续指令rewrite
改写 URI 后重新匹配 locationnginx
log_format json_combined escape=json '{' '"time":"$time_iso8601",' '"remote_addr":"$remote_addr",' '"request":"$request",' '"status":$status,' '"body_bytes_sent":$body_bytes_sent,' '"request_time":$request_time,' '"upstream_response_time":"$upstream_response_time",' '"upstream_addr":"$upstream_addr",' '"http_referer":"$http_referer",' '"http_user_agent":"$http_user_agent"' '}'; access_log /var/log/nginx/access.json json_combined buffer=64k flush=5s; 日志切割(logrotate):
bash
cat > /etc/logrotate.d/nginx <<'EOF' /var/log/nginx/*.log { daily missingok rotate 30 compress delaycompress notifempty create 0640 www-data adm sharedscripts prerotate if [ -d /etc/logrotate.d/httpd-prerotate ]; then run-parts /etc/logrotate.d/httpd-prerotate; fi endscript postrotate [ -f /var/run/nginx.pid ] && kill -USR1 $(cat /var/run/nginx.pid) endscript } EOF 关键参数:
worker_processes auto
:与 CPU 核数相同worker_rlimit_nofile 65535
:单 worker 最大文件描述符worker_connections 10240
:单 worker 最大连接数multi_accept on
:一次 accept 多个连接use epoll
:使用 epollsendfile on
:零拷贝发送文件tcp_nopush on
:合并小包tcp_nodelay on
:Nagle 算法关闭keepalive_timeout 65
:长连接保持keepalive_requests 1000
:单连接最大请求数gzip on
:开启 gzipgzip_comp_level 5
:压缩级别open_file_cache
:文件描述符缓存worker_cpu_affinity
:CPU 亲和性CPU 亲和性:
nginx
worker_processes 8; worker_cpu_affinity 00000001 00000010 00000100 00001000 00010000 00100000 01000000 10000000; nginx
upstream backend { server 10.0.0.1:8080 max_fails=3 fail_timeout=30s; server 10.0.0.2:8080 max_fails=3 fail_timeout=30s; health_check interval=5s fails=3 passes=2 uri=/health; }
max_fails=3:连续 3 次失败标记为 downfail_timeout=30s:30s 后重新探测health_check 是商业版功能,开源版用 ngx_http_upstream_check_module
两台 Nginx 主备,VIP 在主机器上。
主:
nginx
vrrp_script chk_nginx { script"/usr/local/bin/check_nginx.sh" interval 2 weight -20 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass SECRET } virtual_ipaddress { 10.0.0.100/24 } track_script { chk_nginx } } 主备 Nginx,DNS 解析到主;主故障时通过脚本切换 DNS(一般使用 api TTL 较短的方式)。
LVS 四层负载,后端是多个 Nginx。
server_tokens offlimit_except GET POST { deny all; }limit_req + limit_conngeo 或 mapsecure_link 或 valid_referersmap $http_user_agent + ifclient_max_body_sizelarge_client_header_bufferslimit_reqsecure_link 防盗链:
nginx
location /download/ { secure_link $arg_md5,$arg_expires; secure_link_md5 "SECRET$uri$arg_expires"; if ($secure_link = "") { return 403; } if ($secure_link = "0") { return 410; } root /data/www; } bash
# 备份 cp /usr/sbin/nginx /usr/sbin/nginx.bak # 编译新版本到不同目录 ./configure --prefix=/usr/local/nginx-new make make install # 用 nginx -V 拿到原编译参数再编译 nginx -V 2>&1 | grep configure # 复制新 binary 替换 cp /usr/local/nginx-new/sbin/nginx /usr/sbin/nginx # 触发热升级 kill -USR2 # 等旧 worker 处理完 sleep 10 kill -WINCH # 关掉旧 master kill -QUIT # 回滚 kill -HUP nginx -V 编译参数和当前配置ss -s、netstat -s 看连接数nginx.confbash
apt update apt install -y nginx nginx -v systemctl enable nginx systemctl start nginx bash
yum install -y epel-release yum install -y nginx systemctl enable nginx systemctl start nginx bash
# 安装依赖 yum install -y gcc make pcre-devel zlib-devel openssl-devel # 下载 wget http://nginx.org/download/nginx-1.24.0.tar.gz tar zxf nginx-1.24.0.tar.gz cd nginx-1.24.0 # 配置 ./configure --prefix=/usr/local/nginx --user=www --group=www --with-http_stub_status_module --with-http_ssl_module --with-http_v2_module --with-http_gzip_static_module --with-http_realip_module --with-http_secure_link_module --with-http_sub_module --with-stream --with-stream_ssl_module --with-pcre --with-pcre-jit --add-module=/path/to/nginx-module-vts # 编译安装 make -j$(nproc) make install # 添加 systemd 单元 cat > /etc/systemd/system/nginx.service <<'EOF' [Unit] Description=The NGINX HTTP and reverse proxy server After=network.target [Service] Type=forking PIDFile=/usr/local/nginx/logs/nginx.pid ExecStartPre=/usr/local/nginx/sbin/nginx -t ExecStart=/usr/local/nginx/sbin/nginx ExecReload=/bin/kill -s HUP $MAINPID ExecStop=/bin/kill -s QUIT $MAINPID PrivateTmp=true [Install] WantedBy=multi-user.target EOF systemctl daemon-reload systemctl enable nginx systemctl start nginx bash
# 推荐结构 /etc/nginx/ ├── nginx.conf # 主配置 ├── mime.types ├── conf.d/ # 业务 server │ ├── default.conf │ ├── api.conf │ └── static.conf ├── snippets/ # 复用片段 │ ├── ssl-params.conf │ ├── proxy-params.conf │ └── security-headers.conf ├── ssl/ # 证书 │ ├── example.com.crt │ └── example.com.key └── logrotate.d/ └── nginx 主配置:
nginx
user www www; worker_processes auto; worker_rlimit_nofile65535; pid /var/run/nginx.pid; error_log /var/log/nginx/error.log warn; events { worker_connections10240; multi_accepton; useepoll; } http { include mime.types; default_type application/octet-stream; charset utf-8; server_tokensoff; sendfileon; tcp_nopushon; tcp_nodelayon; keepalive_timeout65; keepalive_requests1000; client_max_body_size50m; client_body_buffer_size128k; client_header_buffer_size1k; large_client_header_buffers48k; include /etc/nginx/conf.d/*.conf; include /etc/nginx/sites-enabled/*; } 操作目的:让一个域名支持 HTTPS。
步骤:
bash
# 申请证书 certbot certonly --webroot -w /var/www/html -d example.com -d www.example.com # 证书位置 ls -la /etc/letsencrypt/live/example.com/ # fullchain.pem privkey.pem
/etc/nginx/conf.d/example.com.conf:
nginx
server { listen80; server_name example.com www.example.com; return301 https://$host$request_uri; } server { listen443 ssl http2; server_name example.com www.example.com; ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; include /etc/nginx/snippets/ssl-params.conf; root /var/www/example.com; index index.html; location / { try_files$uri$uri/ =404; } }
/etc/nginx/snippets/ssl-params.conf:
nginx
ssl_protocols TLSv1.2 TLSv1.3; ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384; ssl_prefer_server_cipherson; ssl_session_cache shared10m; ssl_session_timeout1d; ssl_session_ticketson; ssl_staplingon; ssl_stapling_verifyon; resolver1.1.1.18.8.8.8 valid=300s; resolver_timeout5s; add_header Strict-Transport-Security "max-age=63072000" always; add_header X-Frame-Options "SAMEORIGIN" always; add_header X-Content-Type-Options "nosniff" always; add_header X-XSS-Protection "1; mode=block" always; add_header Referrer-Policy "no-referrer-when-downgrade" always; 生效:
bash
nginx -t systemctl reload nginx 验证:
bash
curl -vI https://example.com nmap --script ssl-enum-ciphers -p 443 example.com openssl s_client -connect example.com:443 -servername example.com nginx
upstream backend { least_conn; server10.0.0.1:8080 max_fails=3 fail_timeout=30s; server10.0.0.2:8080 max_fails=3 fail_timeout=30s; server10.0.0.3:8080 max_fails=3 fail_timeout=30s; keepalive64; keepalive_timeout60s; } server { listen80; server_name api.example.com; access_log /var/log/nginx/api.example.com.access.log main; location / { proxy_pass http://backend; include /etc/nginx/snippets/proxy-params.conf; } location /api/v1/health { access_logoff; return200'{"status":"ok"}'; add_header Content-Type application/json; } }
/etc/nginx/snippets/proxy-params.conf:
nginx
proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Port $server_port; proxy_connect_timeout5s; proxy_send_timeout30s; proxy_read_timeout30s; proxy_bufferingon; proxy_buffer_size16k; proxy_buffers832k; proxy_busy_buffers_size64k; proxy_next_upstreamerror timeout http_502 http_503 http_504; proxy_next_upstream_tries3; proxy_next_upstream_timeout10s; nginx
http { limit_req_zone $binary_remote_addr zone=global:10m rate=100r/s; server { location / { limit_req zone=global burst=200 nodelay; limit_req_status 429; } } } nginx
http { limit_req_zone $binary_remote_addr zone=login:10m rate=5r/m; server { location /api/login { limit_req zone=login burst=3 nodelay; proxy_pass http://backend; } } } nginx
http { limit_req_zone$http_x_api_key zone=apikey:10m rate=1000r/s; limit_req_zone$arg_token zone=token:10m rate=10r/s; server { location /api/ { if ($http_x_api_key) { set$limit_zone apikey; } if ($arg_token) { set$limit_zone token; } limit_req zone=$limit_zone burst=20 nodelay; } } } nginx
server { listen80; server_name files.example.com; root /data/files; access_log /var/log/nginx/files.access.log; location / { limit_rate10m; # 单连接 10MB/s limit_rate_after10m; # 头 10MB 不限速 limit_conn perip 2; # 单 IP 最多 2 个连接 limit_conn perserver 1000; # 服务器总连接 1000 } } nginx
upstream websocket { server10.0.0.1:8081; server10.0.0.2:8081; } server { listen80; server_name ws.example.com; location / { proxy_pass http://websocket; proxy_http_version1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_set_header Host $host; proxy_read_timeout600s; # 长连接超时 proxy_send_timeout600s; } } nginx
upstream grpc_backend { server10.0.0.1:50051; server10.0.0.2:50051; keepalive32; } server { listen443 ssl http2; server_name grpc.example.com; ssl_certificate /etc/nginx/ssl/fullchain.pem; ssl_certificate_key /etc/nginx/ssl/privkey.pem; location / { grpc_pass grpc://grpc_backend; grpc_set_header Host $host; grpc_connect_timeout5s; } } nginx
stream { upstream mysql_backend { server10.0.0.1:3306 max_fails=3 fail_timeout=30s; server10.0.0.2:3306 max_fails=3 fail_timeout=30s; } server { listen3306; proxy_pass mysql_backend; proxy_connect_timeout5s; proxy_timeout600s; } upstream dns_backend { server8.8.8.8:53; server1.1.1.1:53; } server { listen53 udp; proxy_pass dns_backend; } } nginx
server { listen 127.0.0.1:80; server_name 127.0.0.1; location /nginx_status { stub_status on; access_log off; allow 127.0.0.1; deny all; } } 查看:
bash
curl http://127.0.0.1/nginx_status # Active connections: 291 # server accepts handled requests # 16630948 16630948 31070465 # Reading: 6 Writing: 179 Waiting: 106
Reading + Writing + Waiting 之和应小于 worker_connections * worker_processes。
nginx
http { vhost_traffic_status_zone; vhost_traffic_status_filter_by_hoston; vhost_traffic_status_filter_by_upstreamon; server { location /status { vhost_traffic_status_display; vhost_traffic_status_display_format html; allow127.0.0.1; deny all; } } }
使用 nginx_exporter 或 nginx-prometheus-exporter:
bash
# 启动 nginx-prometheus-exporter nginx-prometheus-exporter -nginx.scrape-uri=http://127.0.0.1:80/nginx_status -web.listen-address=:9113
或使用 nginxinc/nginx-prometheus-exporter,配合 vts 模块。
Prometheus 配置:
yaml
scrape_configs: - job_name: nginx static_configs: - targets: ['nginx-host:9113'] scrape_interval: 15s 现象:worker 进程 CPU 100%。
初步判断:某个请求在 worker 里死循环或密集计算。
命令:
bash
# 查进程 top -H -p # 找到 CPU 最高的线程 ID(十进制) # 转十六进制 printf'%x ' # 看 perf perf top -p # 看 strace strace -p # 看 nginx 错误日志 tail -f /var/log/nginx/error.log 可能原因:
try_files
配合 deep nested 目录gzip
大文件修复:
gzip 启用范围现象:accesslog 502/504 突增。
初步判断:Nginx 连不上后端或后端响应超时。
命令:
bash
# 查后端状态 curl -v http://10.0.0.1:8080/health # 看 nginx errorlog tail -f /var/log/nginx/error.log # 关键关键字:connect() failed (110: Connection timed out) # upstream prematurely closed connection # no live upstreams # 看后端日志 ssh 10.0.0.1 'tail -n 200 -f /var/log/app/app.log' # 看后端进程 ssh 10.0.0.1 'ps -ef | grep -E "java|node|python" | grep -v grep' 可能原因:
修复:
现象:df -h 看到 /var/log/nginx 占满。
命令:
bash
du -sh /var/log/nginx ls -la /var/log/nginx/ 可能原因:
修复:
truncate -s 0 /var/log/nginx/access.log; nginx -s reopen
现象:ss -s 显示 ESTABLISHED 几十万个。
命令:
bash
ss -s ss -tan 'sport = :80' | awk '{print $6}' | sort | uniq -c # 看每个 IP 的连接数 ss -tan 'sport = :80' | awk 'NR>1 {print $6}' | sort | uniq -c | sort -nr | head 可能原因:
keepalive_timeout
设得太大修复:
keepalive_timeoutkeepalive_requests操作目的:升级 Nginx 到新版本而不中断服务。
bash
# 1. 备份 cp /usr/sbin/nginx /usr/sbin/nginx.bak # 2. 编译新版本到临时目录 ./configure --prefix=/tmp/nginx-new [options] make # 注意:不要 make install # 3. 复制新 binary cp /tmp/nginx-new/objs/nginx /usr/sbin/nginx.new # 4. 替换 cp /usr/sbin/nginx.new /usr/sbin/nginx # 5. 触发热升级 kill -USR2 $(cat /var/run/nginx.pid) # 6. 等待 sleep 10 # 7. 优雅关闭旧 worker kill -WINCH $(cat /var/run/nginx.pid.oldbin) # 8. 关掉旧 master kill -QUIT $(cat /var/run/nginx.pid.oldbin) # 9. 验证 nginx -v ps -ef | grep nginx bash
# 改之前先备份 cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak.$(date +%Y%m%d_%H%M%S) # 验证 nginx -t # reload nginx -s reload # 出问题立刻回滚 cp /etc/nginx/nginx.conf.bak.YYYYMMDD_HHMMSS /etc/nginx/nginx.conf nginx -t nginx -s reload bash
# 编译 ./configure --add-module=/path/to/ModSecurity-nginx make make install nginx
modsecurity on; modsecurity_rules_file /etc/nginx/modsec/main.conf; 通过 DNS 切换或 IP 回源切换到云 WAF。
nginx
location / { if ($request_method !~ ^(GET|POST|HEAD)$) { return 405; } if ($http_user_agent ~* (sqlmap|nikto|acunetix|nmap)) { return 403; } if ($query_string ~* union.*select) { return 403; } if ($http_cookie ~* "(../|..\)") { return 403; } }
使用 nginx-ingress-controller:
bash
# 安装 helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm install ingress-nginx ingress-nginx/ingress-nginx Ingress 资源:
yaml
apiVersion: networking.k8s.io/v1 kind:Ingress metadata: name:api annotations: nginx.ingress.kubernetes.io/proxy-body-size:"50m" nginx.ingress.kubernetes.io/rate-limit:"100" nginx.ingress.kubernetes.io/rate-limit-window:"1m" nginx.ingress.kubernetes.io/limit-rps:"10" spec: ingressClassName:nginx tls: -hosts: -api.example.com secretName:example-tls rules: -host:api.example.com http: paths: -path:/ pathType:Prefix backend: service: name:api-service port: number:80 bash
nginx # 启动 nginx -t # 测试配置 nginx -T # 测试并打印完整配置 nginx -s reload # 重新加载配置 nginx -s reopen # 重新打开日志 nginx -s stop # 停止(SIGTERM) nginx -s quit # 优雅停止(SIGQUIT) nginx -V # 编译信息 nginx -v # 版本 nginx -h # 帮助 bash
ps -ef | grep nginx pgrep -af nginx ss -tlnp | grep -E ':80|:443' lsof -i :80 bash
# 测试配置 nginx -t nginx -t -c /etc/nginx/nginx.conf # 查看最终生效配置 nginx -T | less # 格式化输出 nginx -T 2>&1 | grep -E '^s*#' | grep -v '^s*#s*$' | head -20 bash
# ab ab -n 1000 -c 100 http://example.com/ ab -k -n 10000 -c 200 -H 'Connection: keep-alive' https://example.com/ # wrk wrk -t4 -c200 -d30s http://example.com/ wrk -t4 -c200 -d30s -H 'Host: example.com' https://example.com/ # hey hey -n 5000 -c 200 http://example.com/ # vegeta echo"GET http://example.com/" | vegeta attack -duration=30s -rate=1000 | vegeta report -type=hist[0,50ms,100ms,200ms,500ms,1s,2s,5s] # 自带 stub_status curl http://127.0.0.1/nginx_status bash
# 状态码分布 awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -nr # 耗时 Top 20 awk '{print $NF, $0}' /var/log/nginx/access.log | sort -nr | head -20 # 5xx 错误 awk '$9 ~ /^5/' /var/log/nginx/access.log # 客户端 IP 分布 awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head # UA 分布 awk -F'"''{print $6}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head # URL Top 20 awk '{print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head -20 # 实时观察 tail -f /var/log/nginx/access.log | awk '{print $1, $9, $7, $NF}' bash
# 看错误日志 tail -f /var/log/nginx/error.log # 看连接 ss -tan 'sport = :80' | wc -l # 看每个 worker 状态 curl http://127.0.0.1/nginx_status # 看 CPU top -H -p $(pgrep -f 'nginx: worker' | head -1) # 看网络 ss -s netstat -s bash
nginx -t && nginx -s reload # 如果出错,自动回滚到上次成功的配置不会发生 # 一定要先 nginx -t 验证 bash
# 升级 kill -USR2 sleep 10 kill -WINCH kill -QUIT # 回滚 kill -HUP nginx
server { listen80; server_name static.example.com; root /data/www/static; index index.html; charset utf-8; access_log /var/log/nginx/static.access.log; location / { try_files$uri$uri/ =404; } location~* .(gif|jpg|jpeg|png|webp|avif|svg|ico)$ { expires30d; add_header Cache-Control "public, immutable"; access_logoff; } location~* .(css|js|woff2?|ttf|eot)$ { expires7d; access_logoff; } location~* .(mp4|webm|m4a|m4v|mov)$ { mp4; mp4_buffer_size4m; mp4_max_buffer_size10m; } } nginx
upstream backend { least_conn; server10.0.0.1:8080 max_fails=3 fail_timeout=30s; server10.0.0.2:8080 max_fails=3 fail_timeout=30s; server10.0.0.3:8080 max_fails=3 fail_timeout=30s; keepalive64; } server { listen80; server_name api.example.com; return301 https://$host$request_uri; } server { listen443 ssl http2; server_name api.example.com; ssl_certificate /etc/letsencrypt/live/api.example.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/api.example.com/privkey.pem; include /etc/nginx/snippets/ssl-params.conf; access_log /var/log/nginx/api.access.log main; error_log /var/log/nginx/api.error.log warn; client_max_body_size50m; location /health { access_logoff; return200'OK'; } location / { proxy_pass http://backend; include /etc/nginx/snippets/proxy-params.conf; } } nginx
http { limit_req_zone$binary_remote_addr zone=perip:10m rate=10r/s; limit_req_zone$server_name zone=perserver:10m rate=1000r/s; limit_conn_zone$binary_remote_addr zone=perip_conn:10m; server { listen80; server_name app.example.com; location / { limit_req zone=perserver burst=2000 nodelay; limit_conn perip_conn 50; proxy_pass http://backend; } location /api/login { limit_req zone=perip burst=5 nodelay; limit_req_status429; add_header Retry-After 5s; proxy_pass http://backend; } } } nginx
server { listen80; server_name cdn.example.com; root /data/www/cdn; location~* .(jpg|jpeg|png|gif|webp|avif)$ { valid_referersnoneblocked example.com *.example.com; if ($invalid_referer) { return403; } expires30d; add_header Cache-Control "public, immutable"; } } nginx
upstream backend_v1 { server10.0.0.1:8080; server10.0.0.2:8080; } upstream backend_v2 { server10.0.0.3:8080; } server { listen80; server_name app.example.com; # 按 cookie 灰度 if ($cookie_gray = "v2") { proxy_pass http://backend_v2; break; } # 按 IP 灰度 set$gray0; if ($remote_addr~ "^10.0.0.[0-9]+$") { set$gray1; } if ($arg_gray = "v2") { set$gray1; } if ($gray = 1) { proxy_pass http://backend_v2; } # 默认 v1 location / { proxy_pass http://backend_v1; include /etc/nginx/snippets/proxy-params.conf; } } nginx
stream { log_format stream_log '$remote_addr [$time_local] ' '$protocol $status $bytes_sent $bytes_received ' '$session_time "$upstream_addr"'; access_log /var/log/nginx/stream.access.log stream_log; upstream mysql { server10.0.0.1:3306 max_fails=3 fail_timeout=30s; server10.0.0.2:3306 max_fails=3 fail_timeout=30s; } server { listen3306; proxy_pass mysql; proxy_connect_timeout5s; proxy_timeout600s; } upstream dns_backend { server8.8.8.8:53; server1.1.1.1:53; } server { listen53 udp; proxy_pass dns_backend; proxy_timeout5s; } }
默认 combined 格式:
text
192.168.1.10 - - [29/Jun/202623:45 +0800] "GET /api/user/1 HTTP/1.1" 200 1234 "-" "Mozilla/5.0 ..." 字段含义:
$remote_addr
:客户端 IP$remote_user
:HTTP 认证用户[$time_local]
:本地时间"$request"
:请求行$status
:响应码$body_bytes_sent
:响应字节数"$http_referer"
:Referer"$http_user_agent"
:User-Agent自定义字段:
$request_time
:请求总耗时$upstream_response_time
:上游响应耗时$upstream_addr
:上游地址$upstream_status
:上游状态$upstream_connect_time
:上游连接耗时$upstream_header_time
:上游响应头耗时$connection
:连接序号$connection_requests
:连接上的请求数$pipe
:是否 pipeline$scheme
:HTTP/HTTPS$request_id
:请求唯一 ID| 关键字 | 含义 | 处理建议 |
|---|---|---|
connect() failed (110: Connection timed out) |
上游连接超时 | 排查网络或上游 |
connect() failed (111: Connection refused) |
上游拒绝 | 上游未启动 |
upstream prematurely closed connection |
上游断连 | 上游崩溃或主动断 |
no live upstreams |
所有 upstream down | 检查 upstream 状态 |
client sent invalid header line |
客户端发非法头 | 客户端 bug 或攻击 |
client sent too large entity |
请求体过大 |
调大 client_max_body_size |
SSL_CTX_use_PrivateKey_file() failed |
私钥文件有问题 | 检查证书 |
BIO_new_file() failed |
证书文件读取失败 | 检查路径权限 |
worker process exited on signal 9 |
worker 被 SIGKILL | OOM Killer 触发 |
worker process is shutting down |
优雅退出中 | 正常 |
an upstream response is buffered to a temporary file |
响应缓冲到磁盘 | 加 proxy buffer |
| 指标 | 来源 | 健康范围 | 异常表现 |
|---|---|---|---|
| Active connections | stub_status | < worker_connections × workers | 持续 > 80% |
| Reading | stub_status | < 100 | > 500 表示 worker 处理慢 |
| Writing | stub_status | < 100 | > 500 表示上游写慢 |
| Waiting | stub_status | < 1000 | 持续 > 1 万为连接打满 |
nginx_http_requests_total |
exporter | 业务相关 | 突增为流量暴涨 |
nginx_http_connections{state="active"} |
exporter | < workers × connections | 持续高 |
nginx_upstream_response_time_seconds |
vts | 业务相关 | 突增为上游慢 |
nginx_upstream_response_status |
vts | 业务相关 | 5xx 突增 |
nginx_server_response_time_seconds |
vts | 业务相关 | 突增 |
worker_connectionsulimit -n调高:
bash
# /etc/security/limits.conf * soft nofile 65535 * hard nofile 65535 * soft nproc 65535 * hard nproc 65535 bash
# systemd 单元 [Service] LimitNOFILE=65535 LimitNPROC=65535 upstream_addr、upstream_status、upstream_response_timeconnect() failed 还是 upstream prematurely closedmtr、tcpping、curl -vproxy_read_timeout 是否合理client_max_body_sizeserver.tomcat.max-http-form-post-sizeallow
/ deny 规则误伤auth_basic
配置问题secure_link
时间戳过期root
/ alias 路径错try_files
不匹配location
优先级问题large_client_header_buffers
调大worker_rlimit_nofile
超限accept_mutex
死锁error_log
看具体错误nginx -s reload
前必须 nginx -t,配置错误 reload 后 master 仍能接收连接,但 worker 不会接新连接,造成表面正常实际不服务client_max_body_size
不设或过大会被攻击者发大文件耗尽磁盘worker_connections
设得过大但 ulimit -n 没调高,worker 启动就报错keepalive_timeout
设得过大会让攻击者占满长连接stub_status 到公网会泄露内部状态if
指令在 location 块内使用有“邪恶 if”陷阱,能用 map 或 set 就别用 ifproxy_set_header Connection "" 会让上游不保持长连接proxy_buffering off
会让上游响应立刻发到客户端,但同时占更多内存try_files 配错成 $uri/ =404 之前的部分,会让所有请求落空autoindex on
会让目录可遍历,泄露信息geo
模块加载超过 7 万条记录会启动失败gzip_types
写错 MIME 会被攻击者上传非法格式文件add_header
在 if 内只对当前 if 生效,redirect 中丢失头worker_rlimit_nofile
必须小于 ulimit -nbash
nginx -t nginx -T | less bash
systemctl status nginx ss -tlnp | grep -E ':80|:443' curl -I http://localhost/ bash
curl -vI https://example.com openssl s_client -connect example.com:443 -servername example.com nmap --script ssl-enum-ciphers -p 443 example.com bash
wrk -t4 -c200 -d30s http://localhost/ ab -n 1000 -c 100 http://localhost/ bash
curl http://127.0.0.1/nginx_status curl http://127.0.0.1:9113/metrics bash
tail -f /var/log/nginx/access.log tail -f /var/log/nginx/error.log bash
# 修改前备份 cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak.$(date +%Y%m%d_%H%M%S) cp -r /etc/nginx/conf.d /etc/nginx/conf.d.bak.$(date +%Y%m%d_%H%M%S) # 出问题立刻回滚 cp /etc/nginx/nginx.conf.bak.YYYYMMDD_HHMMSS /etc/nginx/nginx.conf nginx -t nginx -s reload bash
# 保留 nginx.bak kill -QUIT $(cat /var/run/nginx.pid) cp /usr/sbin/nginx.bak /usr/sbin/nginx systemctl start nginx bash
# 用 git 管理 cd /etc/nginx git init git add . git commit -m "init" git log git diff HEAD~1 # 出问题 git checkout HEAD~1 nginx -t nginx -s reload server_tokensautoindexssiStrict-Transport-Securityclient_max_body_size 上限worker_rlimit_nofileLimitNOFILEaccess_log 与 error_logstub_status 或 vts 给监控Restart=alwaysLimitCORE=infinity 用于 core dumpWorkingDirectory 与 RuntimeDirectoryNginx 看似简单,做深了要懂 11 件事:
合格运维的标准,是把上面 11 件事都能在生产环境用 30 分钟内完成:
剩下的是熟练度问题。
最后留一个清单,按这个清单自己跑一遍,能跑通就基本合格:
如果都做不到,绕回来看文章。
全部0条评论
快来发表一下你的评论吧 !