Web Archives - 第2页共3页 - 论野生技术&二次元

用 OpenResty 写了一个 SNI 代理

on 2016 年 6 月 6 日

0 14011 转为繁体

推荐 OpenResty 加上 stream 模块和 ngx_stream_lua_module 模块。在 1.9.15.1 上测试通过。

SNI Proxy based on stream-lua-nginx-module
https://github.com/fffonion/lua-resty-sniproxy
18 forks.
83 stars.
2 open issues.

Recent commits:

doc(readme) fix example nginx configuration, Wangchong Zhou
release: 0.22, Wangchong Zhou
fix: proxy protocol on openresty 1.17.8, Wangchong Zhou
release: 0.21, Wangchong Zhou
proxy protocol support, Wangchong Zhou

示例配置：

stream {
    lua_resolver 8.8.8.8;
    init_worker_by_lua_block {
        sni_rules = { 
            ["www.google.com"] = {"www.google.com", 443},
            ["www.facebook.com"] = {"9.8.7.6", 443},
            ["twitter.com"] = {"1.2.3.4"},
            [".+.twitter.com"] = {nil, 443}
        }   
    }

    server {
            error_log /var/log/nginx/sniproxy-error.log error;
            listen 443;
            content_by_lua_block {
                    local sni = require("resty.sniproxy")
                    local sp = sni:new()
                    sp:run()
            }   
    }
}

stream {

lua_resolver 8.8.8.8;

init_worker_by_lua_block {

sni_rules = {

["www.google.com"] = {"www.google.com", 443},

["www.facebook.com"] = {"9.8.7.6", 443},

["twitter.com"] = {"1.2.3.4"},

[".+.twitter.com"] = {nil, 443}

}

server {

error_log /var/log/nginx/sniproxy-error.log error;

listen 443;

content_by_lua_block {

local sni = require("resty.sniproxy")

local sp = sni:new()

sp:run()

}

A Lua table sni_rules should be defined in the init_worker_by_lua_block directive.

The key can be either whole host name or regular expression. Use . for a default host name. If no entry is matched, connection will be closed.

The value is a table containing host name and port. If host is set to nil, the server_name in SNI will be used. If the port is not defined or set to nil, 443 will be used.

Rules are applied with the priority as its occurrence sequence in the table. In the example above, twitter.com will match the third rule rather than the fourth.

If the protocol version is less than TLSv1 (eg. SSLv3, SSLv2), connection will be closed, since SNI extension is not supported in these versions.

Let’s Encrypt集中化管理

on 2016 年 4 月 7 日

9 11775 转为繁体

Let’s Encrypt的证书签发原理实际上和传统的PKI一样，只不过自动化完成了生成CSR和私钥、提交CSR、取回证书的过程。

此外还要验证域名所属，这一部分和传统的签发机构是一样的，不过传统的签发机构还允许我们使用域名whois中填写的邮箱来验证，而Letsencrypt貌似只能通过http challenge的方式来验证。即和验证服务器约定一个uri和随机字符串，验证服务器请求这一uri，如果得到的内容和约定的随机字符串相同，则验证通过。如图所示：

letsencrypt_howitworks

（官网上抄的）

这意味着我们得在每台部署https的前端的负载均衡服务器上都装一个letencrypt工具。有没有什么集中化管理的办法的呢？

实际上，由于challenge的uri的有规律，我们可以将前端服务器收到的这类请求代理到同一台专门用来签发、更新证书的服务器上。如图所示：

letsencrypt_howitworks_proxypass

当在服务器B上发起域名a.example.com新的签发请求后，Let’s Encrypt的签发服务器返回一个challange uri (8303)和response (ed98)。
服务器B使用webroot插件将这个uri和response写入本地磁盘上对应的文件。
Let’s Encrypt的签发服务器为了验证example.com的所属，查询到example.com指向前端服务器A，于是发送一个HTTP请求/.well-known/acme-challenge/8303到服务器A
服务器A反代这一请求到服务器B
B读取刚才第二步时写入到response，返回到A；A返回到Let’s Encrypt的签发服务器
验证成功，发证！

然后，我们只要从服务器A上取回存储在B上到证书就可以了。可以在B上做一个RESTful的api。注意要配置allow和deny。

A服务器（前端）的nginx配置如下：

server {
    # 其他的location
    # location { ..... }

    location ~ /.well-known {
        proxy_pass http://B.example.com:23333;
    }
}

server {

# 其他的location

# location { ..... }

location ~ /.well-known {

proxy_pass http://B.example.com:23333;

}

B服务器的nginx配置如下：

server {
    listen 23333;
    server_name B.example.com;
    location ~ /.well-known {
        root /tmp/letsencrypt;
        allow IP-OF-A;
        deny all;
    }
}

server {

listen 23333;

server_name B.example.com;

location ~ /.well-known {

root /tmp/letsencrypt;

allow IP-OF-A;

deny all;

}

然后在B上运行：

./letsencrypt-auto --webroot -w /tmp/letsencrypt -d exmaple.com;

1	./letsencrypt-auto --webroot -w /tmp/letsencrypt -d exmaple.com;

搭建一个不被审查的串流站点

on 2015 年 12 月 18 日

Web
作死

17 48892 转为繁体

原标题：被害妄想症该如何生存

先看配置：

server {
        listen 80 default_server;
        listen [::]:80 default_server;
        listen 443 ssl default_server;

        root /usr/share/nginx/html;

        ssl_certificate certs/default.pem;
        ssl_certificate_key certs/default-nopass.key;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH;
        ssl_prefer_server_ciphers on;
}

server {
	listen 443 ssl spdy;

	ssl_certificate certs/real-cert.pem;
	ssl_certificate_key certs/real-cert-nopass.key;
	ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
	ssl_ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH;
	ssl_prefer_server_ciphers on;

	root /usr/share/nginx/html;

	server_name example.com;
 
	location / {
                root /path/to/your/files;
                if (-d $request_filename) {
                        return 404;
                }
		try_files $uri $uri/ =404;
		charset utf-8; 
	}

               
        location ~ .*/$ {
                root /path/to/your/files;
                access_by_lua '
                        local coo = ngx.var.cookie_coo
                        if coo ~= "coo" then
                                ngx.exit(404)
                        end
                ';
                autoindex on;
                autoindex_exact_size off;
        }

}

server {

listen 80 default_server;

listen [::]:80 default_server;

listen 443 ssl default_server;

root /usr/share/nginx/html;

ssl_certificate certs/default.pem;

ssl_certificate_key certs/default-nopass.key;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

ssl_ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH;

ssl_prefer_server_ciphers on;

}

server {

listen 443 ssl spdy;

ssl_certificate certs/real-cert.pem;

ssl_certificate_key certs/real-cert-nopass.key;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

ssl_ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH;

ssl_prefer_server_ciphers on;

root /usr/share/nginx/html;

server_name example.com;

location / {

root /path/to/your/files;

if (-d $request_filename) {

return 404;

}

try_files $uri $uri/ =404;

charset utf-8;

}

location ~ .*/$ {

root /path/to/your/files;

access_by_lua '

local coo = ngx.var.cookie_coo

if coo ~= "coo" then

ngx.exit(404)

end

autoindex on;

autoindex_exact_size off;

}

假设：

审查机关拥有运营商级别的入侵检测设备（比如GFW）

说明：

全站使用https，关闭SSLv3，关闭弱加密组件
default_server开启80端口，使用自签名证书；真实需要访问的域名(example.com)必须使用有效的证书，或者在本地信任根证书。注意example.com不能开启80端口，且与default_server使用的证书不能相同。不要使用泛域名证书。这是为了防止审查机关通过直连IP查看返回的证书中的Common Name来得到真实域名。这样配置之后，直连IP https://xxx.xxx.xxx.xxx默认是返回自签名证书，无法得到真实example.com。
选择性开启autoindex，通过cookie鉴别。注意也可以通过HTTP Basic Authenication认证。对匹配文件夹的uri（”/”结尾）做认证，示例中只有带cookie coo=coo的请求才会返回autoindex，否则返回404。
在location /中禁用目录末尾自动加斜杠，因为如果自动加斜杠，审查机关可以通过暴力猜测出服务器上有哪些目录确实存在（返回了301到末尾加/的url）。方法是if (-d $request_filename)返回404。

PHP大发好

on 2015 年 11 月 30 日

5 54821 转为繁体

博客现在用PHP7了，我再也不用装HHVM了，开心。

你们有没有觉得加载速度变快了~~或者变慢了~~呢

436919cbgw1eyiyw10kcxg203c03cqe1

nginx 批量配置同步

on 2015 年 11 月 5 日

0 71472 转为繁体

在编译了lua-nginx-module的nginx上，可以方便地使用shared dict特性，在不reload配置文件的情况下实现配置同步。

由于shared dict使用一块共享内存，因此所有worker均可读写，也就不存在一致性的问题。

使用shared dict

nginx/openresty的一些记录

on 2015 年 10 月 16 日

28 41225 转为繁体

日志

屏蔽user-agent并屏蔽日志

if ($http_user_agent ~* (^badbot/useragent$)) {
    rewrite (.*) /badbot break;
}

location = /badbot {
    access_log off;
    return 403;
}

if ($http_user_agent ~* (^badbot/useragent$)) {

rewrite (.*) /badbot break;

}

location = /badbot {

access_log off;

return 403;

}

不屏蔽user-agent（允许其访问），但屏蔽日志

server {
    set $is_spider 1;
    if ($http_user_agent ~* "bot|spider|Bot|Disqus|WebIndex|YunGuanCe") {
        set $is_spider 0;
    }
    access_log /var/log/nginx/access.log combined if=$is_spider;
}

server {

set $is_spider 1;

if ($http_user_agent ~* "bot|spider|Bot|Disqus|WebIndex|YunGuanCe") {

set $is_spider 0;

}

access_log /var/log/nginx/access.log combined if=$is_spider;

}

按uri屏蔽日志（可以和上面的按user-agent用同一个变量来同时过滤uri和user-agent)

map $request $loggable {
    ~/favicon.ico 0;
    ~/images/* 0;
    ~/js/* 0;
    ~/css/* 0;
    default 1;
}

server {
    access_log /var/log/nginx/access.log combined if=$loggable;
}

map $request $loggable {

~/favicon.ico 0;

~/images/* 0;

~/js/* 0;

~/css/* 0;

default 1;

}

server {

access_log /var/log/nginx/access.log combined if=$loggable;

}

Header

按mime type设置缓存时间

map $sent_http_content_type $cacheable_types {
    "text/css"    "max-age=14400";
    "text/plain"  "max-age=86400";
    "image/jpeg"  "max-age=86400";
    "image/png"   "max-age=86400";
    "image/gif"   "max-age=86400";
    "image/x-icon" "max-age=86400";
    "application/x-7z-compressed" "max-age=864000";
    "application/x-javascript" "max-age=86400";
    "application/json" "max-age=86400";
    "application/x-bittorrent" "max-age=864000";
    default       "";
}

server {
    location / {
        root   "/var/www/html/";           
        add_header "Cache-Control" $cacheable_types;
    }
}

map $sent_http_content_type $cacheable_types {

"text/css" "max-age=14400";

"text/plain" "max-age=86400";

"image/jpeg" "max-age=86400";

"image/png" "max-age=86400";

"image/gif" "max-age=86400";

"image/x-icon" "max-age=86400";

"application/x-7z-compressed" "max-age=864000";

"application/x-javascript" "max-age=86400";

"application/json" "max-age=86400";

"application/x-bittorrent" "max-age=864000";

default "";

}

server {

location / {

root "/var/www/html/";

add_header "Cache-Control" $cacheable_types;

}

防攻击

简单的无状态cookie challenge（需要lua-nginx-module)

crawlers块中可以手动填写要屏蔽的IP

将其中的s改成随机字符串＋时间戳可以变成有状态版本（需使用redis/memcached/shared memory存储生成的随机字符串）

将set-cookie改成通过js生成cookie可以变成javascript challenge，注意要在js里加上浏览器上下文判断，如var cookie=location.protocol?cookie:””; 或者DOM操作

这里有个更高级的输验证码的示例

geo $crawlers{
    1.2.3.0/24 1;
    4.5.6.7 1;
    default 0;
}
server {
    location ~ \.php {
        set $_c 0;
        if ($http_user_agent ~* "bad.guy/123") {
            set $_c 1;
        }            
        if ($crawlers){
            set $_c 1;
        }
        if ($_c){
            access_log     off;
            access_by_lua_block {
                local expires = 90
                local s = ngx.time()
                local cob = tonumber(ngx.var.cookie_cob)
                if cob == nil then cob=0 end
                local coa = ngx.md5("saltsalt" .. ngx.var.remote_addr .. "saltsalt" .. cob)
                if s - cob > expires or ngx.var.cookie_coa ~= coa then
                    coa = ngx.md5("saltsalt" .. ngx.var.remote_addr .. "saltsalt" .. s)
                    ngx.header["Set-Cookie"] = {"coa=" .. coa .."; path=/; domain=.example.com; HttpOnly",
                        "cob=" .. s.."; path=/; domain=.example.com; HttpOnly"}
                    ngx.header["Content-Type"] = "text/html"
                    ngx.say("<meta http-equiv ='refresh' content='0'>");
                    ngx.exit(200)
                end
            }
        }
        # fastcgi_pass
    }
}

geo $crawlers{

1.2.3.0/24 1;

4.5.6.7 1;

default 0;

}

server {

location ~ \.php {

set $_c 0;

if ($http_user_agent ~* "bad.guy/123") {

set $_c 1;

}

if ($crawlers){

set $_c 1;

}

if ($_c){

access_log off;

access_by_lua_block {

local expires = 90

local s = ngx.time()

local cob = tonumber(ngx.var.cookie_cob)

if cob == nil then cob=0 end

local coa = ngx.md5("saltsalt" .. ngx.var.remote_addr .. "saltsalt" .. cob)

if s - cob > expires or ngx.var.cookie_coa ~= coa then

coa = ngx.md5("saltsalt" .. ngx.var.remote_addr .. "saltsalt" .. s)

ngx.header["Set-Cookie"] = {"coa=" .. coa .."; path=/; domain=.example.com; HttpOnly",

"cob=" .. s.."; path=/; domain=.example.com; HttpOnly"}

ngx.header["Content-Type"] = "text/html"

ngx.say("<meta http-equiv ='refresh' content='0'>");

ngx.exit(200)

end

}

# fastcgi_pass

}

其他

植入cookie

~~需要注意的是使用ngx.time()产生秒级的时间，用来做随机数种子可能会冲突，因此建议加上另外的随机变量（如下面的例子用的是客户端的ip）~~ 可以使用ngx.now()产生毫秒精度时间

local modname = "util"
local _M = { _VERSION = '0.01' }
local mt = { __index = _M }

function _M.random_str(l, seed, r)
    local s = r or 'abcdefghijklmnhopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789=+-'
 
    local ret =''
	-- os.time() is precise to second
	math.randomseed(os.time() * 1000 + ngx.crc32_short(seed or ""))
    for i=1 ,l do
        local pos = math.random(1, string.len(s))
        ret = ret .. string.sub(s, pos, pos)
    end
 
    return ret
end

function _M.plant_cookie(k, h)
	local ck = require "resty.cookie"
	local cookie, err = ck:new()
	if not cookie then
		ngx.log(ngx.ERR, "ERROR PLANTING COOKIE", err)
		return
	end
	
	local _, err = cookie:get(k)
	if _ then
		return
	end
	local ok, err = cookie:set({
		key = k, value = _M.random_str(32, ngx.var.remote_addr, '0123456789abcdef'), path = "/",
		domain = h or ngx.var.http_host,
		httponly = true,
		expires= 'Thu, 31-Dec-37 23:55:55 GMT',
		max_age = 2147483647
	})

end

return _M

local modname = "util"

local _M = { _VERSION = '0.01' }

local mt = { __index = _M }

function _M.random_str(l, seed, r)

local s = r or 'abcdefghijklmnhopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789=+-'

local ret =''

-- os.time() is precise to second

math.randomseed(os.time() * 1000 + ngx.crc32_short(seed or ""))

for i=1 ,l do

local pos = math.random(1, string.len(s))

ret = ret .. string.sub(s, pos, pos)

end

return ret

end

function _M.plant_cookie(k, h)

local ck = require "resty.cookie"

local cookie, err = ck:new()

if not cookie then

ngx.log(ngx.ERR, "ERROR PLANTING COOKIE", err)

return

end

local _, err = cookie:get(k)

if _ then

return

end

local ok, err = cookie:set({

key = k, value = _M.random_str(32, ngx.var.remote_addr, '0123456789abcdef'), path = "/",

domain = h or ngx.var.http_host,

httponly = true,

expires= 'Thu, 31-Dec-37 23:55:55 GMT',

max_age = 2147483647

})

end

return _M

location / {
    access_by_lua '
        local _ = require("util")
        _.plant_cookie("cookie_name",".example.come")
    ';
}

location / {

access_by_lua '

local _ = require("util")

_.plant_cookie("cookie_name",".example.come")

}

Category Archives

用 OpenResty 写了一个 SNI 代理

Let’s Encrypt集中化管理

搭建一个不被审查的串流站点

PHP大发好

nginx 批量配置同步

使用shared dict

nginx/openresty的一些记录

日志

Header

防攻击

其他