Web Archives - 第2頁共3頁 - 論野生技術&二次元

用 OpenResty 寫了一個 SNI 代理

on 2016 年 6 月 6 日

0 14018 轉為簡體

推薦 OpenResty 加上 stream 模塊和 ngx_stream_lua_module 模塊。在 1.9.15.1 上測試通過。

SNI Proxy based on stream-lua-nginx-module
https://github.com/fffonion/lua-resty-sniproxy
18 forks.
83 stars.
2 open issues.

Recent commits:

doc(readme) fix example nginx configuration, Wangchong Zhou
release: 0.22, Wangchong Zhou
fix: proxy protocol on openresty 1.17.8, Wangchong Zhou
release: 0.21, Wangchong Zhou
proxy protocol support, Wangchong Zhou

示例配置：

stream {
    lua_resolver 8.8.8.8;
    init_worker_by_lua_block {
        sni_rules = { 
            ["www.google.com"] = {"www.google.com", 443},
            ["www.facebook.com"] = {"9.8.7.6", 443},
            ["twitter.com"] = {"1.2.3.4"},
            [".+.twitter.com"] = {nil, 443}
        }   
    }

    server {
            error_log /var/log/nginx/sniproxy-error.log error;
            listen 443;
            content_by_lua_block {
                    local sni = require("resty.sniproxy")
                    local sp = sni:new()
                    sp:run()
            }   
    }
}

stream {

lua_resolver 8.8.8.8;

init_worker_by_lua_block {

sni_rules = {

["www.google.com"] = {"www.google.com", 443},

["www.facebook.com"] = {"9.8.7.6", 443},

["twitter.com"] = {"1.2.3.4"},

[".+.twitter.com"] = {nil, 443}

}

server {

error_log /var/log/nginx/sniproxy-error.log error;

listen 443;

content_by_lua_block {

local sni = require("resty.sniproxy")

local sp = sni:new()

sp:run()

}

A Lua table sni_rules should be defined in the init_worker_by_lua_block directive.

The key can be either whole host name or regular expression. Use . for a default host name. If no entry is matched, connection will be closed.

The value is a table containing host name and port. If host is set to nil, the server_name in SNI will be used. If the port is not defined or set to nil, 443 will be used.

Rules are applied with the priority as its occurrence sequence in the table. In the example above, twitter.com will match the third rule rather than the fourth.

If the protocol version is less than TLSv1 (eg. SSLv3, SSLv2), connection will be closed, since SNI extension is not supported in these versions.

Let’s Encrypt集中化管理

on 2016 年 4 月 7 日

9 11781 轉為簡體

Let’s Encrypt的證書籤發原理實際上和傳統的PKI一樣，只不過自動化完成了生成CSR和私鑰、提交CSR、取回證書的過程。

此外還要驗證域名所屬，這一部分和傳統的簽發機構是一樣的，不過傳統的簽發機構還允許我們使用域名whois中填寫的郵箱來驗證，而Letsencrypt貌似只能通過http challenge的方式來驗證。即和驗證服務器約定一個uri和隨機字符串，驗證服務器請求這一uri，如果得到的內容和約定的隨機字符串相同，則驗證通過。如圖所示：

letsencrypt_howitworks

（官網上抄的）

這意味着我們得在每台部署https的前端的負載均衡服務器上都裝一個letencrypt工具。有沒有什麼集中化管理的辦法的呢？

實際上，由於challenge的uri的有規律，我們可以將前端服務器收到的這類請求代理到同一台專門用來簽發、更新證書的服務器上。如圖所示：

letsencrypt_howitworks_proxypass

當在服務器B上發起域名a.example.com新的簽發請求後，Let’s Encrypt的簽發服務器返回一個challange uri (8303)和response (ed98)。
服務器B使用webroot插件將這個uri和response寫入本地磁盤上對應的文件。
Let’s Encrypt的簽發服務器為了驗證example.com的所屬，查詢到example.com指向前端服務器A，於是發送一個HTTP請求/.well-known/acme-challenge/8303到服務器A
服務器A反代這一請求到服務器B
B讀取剛才第二步時寫入到response，返回到A；A返回到Let’s Encrypt的簽發服務器
驗證成功，發證！

然後，我們只要從服務器A上取回存儲在B上到證書就可以了。可以在B上做一個RESTful的api。注意要配置allow和deny。

A服務器（前端）的nginx配置如下：

server {
    # 其他的location
    # location { ..... }

    location ~ /.well-known {
        proxy_pass http://B.example.com:23333;
    }
}

server {

# 其他的location

# location { ..... }

location ~ /.well-known {

proxy_pass http://B.example.com:23333;

}

B服務器的nginx配置如下：

server {
    listen 23333;
    server_name B.example.com;
    location ~ /.well-known {
        root /tmp/letsencrypt;
        allow IP-OF-A;
        deny all;
    }
}

server {

listen 23333;

server_name B.example.com;

location ~ /.well-known {

root /tmp/letsencrypt;

allow IP-OF-A;

deny all;

}

然後在B上運行：

./letsencrypt-auto --webroot -w /tmp/letsencrypt -d exmaple.com;

1	./letsencrypt-auto --webroot -w /tmp/letsencrypt -d exmaple.com;

搭建一個不被審查的串流站點

on 2015 年 12 月 18 日

Web
作死

17 48900 轉為簡體

原標題：被害妄想症該如何生存

先看配置：

server {
        listen 80 default_server;
        listen [::]:80 default_server;
        listen 443 ssl default_server;

        root /usr/share/nginx/html;

        ssl_certificate certs/default.pem;
        ssl_certificate_key certs/default-nopass.key;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH;
        ssl_prefer_server_ciphers on;
}

server {
	listen 443 ssl spdy;

	ssl_certificate certs/real-cert.pem;
	ssl_certificate_key certs/real-cert-nopass.key;
	ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
	ssl_ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH;
	ssl_prefer_server_ciphers on;

	root /usr/share/nginx/html;

	server_name example.com;
 
	location / {
                root /path/to/your/files;
                if (-d $request_filename) {
                        return 404;
                }
		try_files $uri $uri/ =404;
		charset utf-8; 
	}

               
        location ~ .*/$ {
                root /path/to/your/files;
                access_by_lua '
                        local coo = ngx.var.cookie_coo
                        if coo ~= "coo" then
                                ngx.exit(404)
                        end
                ';
                autoindex on;
                autoindex_exact_size off;
        }

}

server {

listen 80 default_server;

listen [::]:80 default_server;

listen 443 ssl default_server;

root /usr/share/nginx/html;

ssl_certificate certs/default.pem;

ssl_certificate_key certs/default-nopass.key;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

ssl_ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH;

ssl_prefer_server_ciphers on;

}

server {

listen 443 ssl spdy;

ssl_certificate certs/real-cert.pem;

ssl_certificate_key certs/real-cert-nopass.key;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

ssl_ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH;

ssl_prefer_server_ciphers on;

root /usr/share/nginx/html;

server_name example.com;

location / {

root /path/to/your/files;

if (-d $request_filename) {

return 404;

}

try_files $uri $uri/ =404;

charset utf-8;

}

location ~ .*/$ {

root /path/to/your/files;

access_by_lua '

local coo = ngx.var.cookie_coo

if coo ~= "coo" then

ngx.exit(404)

end

autoindex on;

autoindex_exact_size off;

}

假設：

審查機關擁有運營商級別的入侵檢測設備（比如GFW）

說明：

全站使用https，關閉SSLv3，關閉弱加密組件
default_server開啟80端口，使用自簽名證書；真實需要訪問的域名(example.com)必須使用有效的證書，或者在本地信任根證書。注意example.com不能開啟80端口，且與default_server使用的證書不能相同。不要使用泛域名證書。這是為了防止審查機關通過直連IP查看返回的證書中的Common Name來得到真實域名。這樣配置之後，直連IP https://xxx.xxx.xxx.xxx默認是返回自簽名證書，無法得到真實example.com。
選擇性開啟autoindex，通過cookie鑒別。注意也可以通過HTTP Basic Authenication認證。對匹配文件夾的uri（”/”結尾）做認證，示例中只有帶cookie coo=coo的請求才會返回autoindex，否則返回404。
在location /中禁用目錄末尾自動加斜杠，因為如果自動加斜杠，審查機關可以通過暴力猜測出服務器上有哪些目錄確實存在（返回了301到末尾加/的url）。方法是if (-d $request_filename)返回404。

PHP大發好

on 2015 年 11 月 30 日

5 54830 轉為簡體

博客現在用PHP7了，我再也不用裝HHVM了，開心。

你們有沒有覺得加載速度變快了~~或者變慢了~~呢

436919cbgw1eyiyw10kcxg203c03cqe1

nginx 批量配置同步

on 2015 年 11 月 5 日

0 71482 轉為簡體

在編譯了lua-nginx-module的nginx上，可以方便地使用shared dict特性，在不reload配置文件的情況下實現配置同步。

由於shared dict使用一塊共享內存，因此所有worker均可讀寫，也就不存在一致性的問題。

使用shared dict

nginx/openresty的一些記錄

on 2015 年 10 月 16 日

28 41232 轉為簡體

日誌

屏蔽user-agent並屏蔽日誌

if ($http_user_agent ~* (^badbot/useragent$)) {
    rewrite (.*) /badbot break;
}

location = /badbot {
    access_log off;
    return 403;
}

if ($http_user_agent ~* (^badbot/useragent$)) {

rewrite (.*) /badbot break;

}

location = /badbot {

access_log off;

return 403;

}

不屏蔽user-agent（允許其訪問），但屏蔽日誌

server {
    set $is_spider 1;
    if ($http_user_agent ~* "bot|spider|Bot|Disqus|WebIndex|YunGuanCe") {
        set $is_spider 0;
    }
    access_log /var/log/nginx/access.log combined if=$is_spider;
}

server {

set $is_spider 1;

if ($http_user_agent ~* "bot|spider|Bot|Disqus|WebIndex|YunGuanCe") {

set $is_spider 0;

}

access_log /var/log/nginx/access.log combined if=$is_spider;

}

按uri屏蔽日誌（可以和上面的按user-agent用同一個變量來同時過濾uri和user-agent)

map $request $loggable {
    ~/favicon.ico 0;
    ~/images/* 0;
    ~/js/* 0;
    ~/css/* 0;
    default 1;
}

server {
    access_log /var/log/nginx/access.log combined if=$loggable;
}

map $request $loggable {

~/favicon.ico 0;

~/images/* 0;

~/js/* 0;

~/css/* 0;

default 1;

}

server {

access_log /var/log/nginx/access.log combined if=$loggable;

}

Header

按mime type設置緩存時間

map $sent_http_content_type $cacheable_types {
    "text/css"    "max-age=14400";
    "text/plain"  "max-age=86400";
    "image/jpeg"  "max-age=86400";
    "image/png"   "max-age=86400";
    "image/gif"   "max-age=86400";
    "image/x-icon" "max-age=86400";
    "application/x-7z-compressed" "max-age=864000";
    "application/x-javascript" "max-age=86400";
    "application/json" "max-age=86400";
    "application/x-bittorrent" "max-age=864000";
    default       "";
}

server {
    location / {
        root   "/var/www/html/";           
        add_header "Cache-Control" $cacheable_types;
    }
}

map $sent_http_content_type $cacheable_types {

"text/css" "max-age=14400";

"text/plain" "max-age=86400";

"image/jpeg" "max-age=86400";

"image/png" "max-age=86400";

"image/gif" "max-age=86400";

"image/x-icon" "max-age=86400";

"application/x-7z-compressed" "max-age=864000";

"application/x-javascript" "max-age=86400";

"application/json" "max-age=86400";

"application/x-bittorrent" "max-age=864000";

default "";

}

server {

location / {

root "/var/www/html/";

add_header "Cache-Control" $cacheable_types;

}

防攻擊

簡單的無狀態cookie challenge（需要lua-nginx-module)

crawlers塊中可以手動填寫要屏蔽的IP

將其中的s改成隨機字符串＋時間戳可以變成有狀態版本（需使用redis/memcached/shared memory存儲生成的隨機字符串）

將set-cookie改成通過js生成cookie可以變成javascript challenge，注意要在js里加上瀏覽器上下文判斷，如var cookie=location.protocol?cookie:””; 或者DOM操作

這裡有個更高級的輸驗證碼的示例

geo $crawlers{
    1.2.3.0/24 1;
    4.5.6.7 1;
    default 0;
}
server {
    location ~ \.php {
        set $_c 0;
        if ($http_user_agent ~* "bad.guy/123") {
            set $_c 1;
        }            
        if ($crawlers){
            set $_c 1;
        }
        if ($_c){
            access_log     off;
            access_by_lua_block {
                local expires = 90
                local s = ngx.time()
                local cob = tonumber(ngx.var.cookie_cob)
                if cob == nil then cob=0 end
                local coa = ngx.md5("saltsalt" .. ngx.var.remote_addr .. "saltsalt" .. cob)
                if s - cob > expires or ngx.var.cookie_coa ~= coa then
                    coa = ngx.md5("saltsalt" .. ngx.var.remote_addr .. "saltsalt" .. s)
                    ngx.header["Set-Cookie"] = {"coa=" .. coa .."; path=/; domain=.example.com; HttpOnly",
                        "cob=" .. s.."; path=/; domain=.example.com; HttpOnly"}
                    ngx.header["Content-Type"] = "text/html"
                    ngx.say("<meta http-equiv ='refresh' content='0'>");
                    ngx.exit(200)
                end
            }
        }
        # fastcgi_pass
    }
}

geo $crawlers{

1.2.3.0/24 1;

4.5.6.7 1;

default 0;

}

server {

location ~ \.php {

set $_c 0;

if ($http_user_agent ~* "bad.guy/123") {

set $_c 1;

}

if ($crawlers){

set $_c 1;

}

if ($_c){

access_log off;

access_by_lua_block {

local expires = 90

local s = ngx.time()

local cob = tonumber(ngx.var.cookie_cob)

if cob == nil then cob=0 end

local coa = ngx.md5("saltsalt" .. ngx.var.remote_addr .. "saltsalt" .. cob)

if s - cob > expires or ngx.var.cookie_coa ~= coa then

coa = ngx.md5("saltsalt" .. ngx.var.remote_addr .. "saltsalt" .. s)

ngx.header["Set-Cookie"] = {"coa=" .. coa .."; path=/; domain=.example.com; HttpOnly",

"cob=" .. s.."; path=/; domain=.example.com; HttpOnly"}

ngx.header["Content-Type"] = "text/html"

ngx.say("<meta http-equiv ='refresh' content='0'>");

ngx.exit(200)

end

}

# fastcgi_pass

}

其他

植入cookie

~~需要注意的是使用ngx.time()產生秒級的時間，用來做隨機數種子可能會衝突，因此建議加上另外的隨機變量（如下面的例子用的是客戶端的ip）~~ 可以使用ngx.now()產生毫秒精度時間

local modname = "util"
local _M = { _VERSION = '0.01' }
local mt = { __index = _M }

function _M.random_str(l, seed, r)
    local s = r or 'abcdefghijklmnhopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789=+-'
 
    local ret =''
	-- os.time() is precise to second
	math.randomseed(os.time() * 1000 + ngx.crc32_short(seed or ""))
    for i=1 ,l do
        local pos = math.random(1, string.len(s))
        ret = ret .. string.sub(s, pos, pos)
    end
 
    return ret
end

function _M.plant_cookie(k, h)
	local ck = require "resty.cookie"
	local cookie, err = ck:new()
	if not cookie then
		ngx.log(ngx.ERR, "ERROR PLANTING COOKIE", err)
		return
	end
	
	local _, err = cookie:get(k)
	if _ then
		return
	end
	local ok, err = cookie:set({
		key = k, value = _M.random_str(32, ngx.var.remote_addr, '0123456789abcdef'), path = "/",
		domain = h or ngx.var.http_host,
		httponly = true,
		expires= 'Thu, 31-Dec-37 23:55:55 GMT',
		max_age = 2147483647
	})

end

return _M

local modname = "util"

local _M = { _VERSION = '0.01' }

local mt = { __index = _M }

function _M.random_str(l, seed, r)

local s = r or 'abcdefghijklmnhopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789=+-'

local ret =''

-- os.time() is precise to second

math.randomseed(os.time() * 1000 + ngx.crc32_short(seed or ""))

for i=1 ,l do

local pos = math.random(1, string.len(s))

ret = ret .. string.sub(s, pos, pos)

end

return ret

end

function _M.plant_cookie(k, h)

local ck = require "resty.cookie"

local cookie, err = ck:new()

if not cookie then

ngx.log(ngx.ERR, "ERROR PLANTING COOKIE", err)

return

end

local _, err = cookie:get(k)

if _ then

return

end

local ok, err = cookie:set({

key = k, value = _M.random_str(32, ngx.var.remote_addr, '0123456789abcdef'), path = "/",

domain = h or ngx.var.http_host,

httponly = true,

expires= 'Thu, 31-Dec-37 23:55:55 GMT',

max_age = 2147483647

})

end

return _M

location / {
    access_by_lua '
        local _ = require("util")
        _.plant_cookie("cookie_name",".example.come")
    ';
}

location / {

access_by_lua '

local _ = require("util")

_.plant_cookie("cookie_name",".example.come")

}

Category Archives

用 OpenResty 寫了一個 SNI 代理

Let’s Encrypt集中化管理

搭建一個不被審查的串流站點

PHP大發好

nginx 批量配置同步

使用shared dict

nginx/openresty的一些記錄

日誌

Header

防攻擊

其他