Lua Archives - 第2页共2页 - 论野生技术&二次元

cidr.me地理位置查询

on 2016 年 7 月 26 日

2 16322 转为繁体

基于OpenResty ，MaxMind GeoIP数据库和从bgp.he.net生成的ASN数据库，因为没有经纬度的需求所以没有显示。下文有源代码的链接，如果需要可以自行修改加上经纬度或者将输出变为JSON等。

这里有chrome插件

使用方法

查询当前IPv4地址

$ curl http://cidr.me/ip
x.x.x.x

查询当前IPv6地址

$ curl http://ipv6.cidr.me/ip
x.x.x.x

查询当前IP的PTR记录（反向DNS），地理位置和ASN

$ curl http://cidr.me/
x.x.x.x
Country, City
ASN number

查询当前IP的PTR记录

$ curl http://cidr.me/rdns
x-x-x-x.com

查询指定IP的地理位置

$ curl http://cidr.me/74.125.203.199
74.125.203.199 th-in-f199.1e100.net
United States, California, Mountain View
AS15169 Google Inc.

查询域名的地理位置

$ curl http://cidr.me/www.google.com.hk
172.217.3.195 sea15s12-in-f3.1e100.net
United States, California, Mountain View
AS15169 Google LLC

2607:f8b0:400a:809::2003 sea15s12-in-x03.1e100.net
United States
AS15169 Google LLC

查询域名的IP

$ curl http://cidr.me/www.google.com.hk/ip
74.125.203.199
2607:f8b0:400a:809::2003

查询域名的IP和CNAME（如果存在）

$ curl http://cidr.me/www.youtube.com/dns
youtube-ui.l.google.com 172.217.3.174
172.217.3.206
216.58.193.78
216.58.217.46
2607:f8b0:400a:808::200e

源代码

在这里https://gist.github.com/fffonion/44e5fb59e2a8f0efba5c1965c6043584

需要ngx_http_geoip_module, echo-nginx-module, lua-nginx-module，安装libgeoip，并将maxmind的旧版geoip数据库放在/usr/share/GeoIP

通过bgp.he.net生成ASN数据库

用 OpenResty 写了一个 SNI 代理

on 2016 年 6 月 6 日

0 14018 转为繁体

功能类似于dlundquist/sniproxy

推荐 OpenResty 加上 stream 模块和 ngx_stream_lua_module 模块。在 1.9.15.1 上测试通过。

SNI Proxy based on stream-lua-nginx-module
https://github.com/fffonion/lua-resty-sniproxy
18 forks.
83 stars.
2 open issues.

Recent commits:

doc(readme) fix example nginx configuration, Wangchong Zhou
release: 0.22, Wangchong Zhou
fix: proxy protocol on openresty 1.17.8, Wangchong Zhou
release: 0.21, Wangchong Zhou
proxy protocol support, Wangchong Zhou

示例配置：

stream {
    lua_resolver 8.8.8.8;
    init_worker_by_lua_block {
        sni_rules = { 
            ["www.google.com"] = {"www.google.com", 443},
            ["www.facebook.com"] = {"9.8.7.6", 443},
            ["twitter.com"] = {"1.2.3.4"},
            [".+.twitter.com"] = {nil, 443}
        }   
    }

    server {
            error_log /var/log/nginx/sniproxy-error.log error;
            listen 443;
            content_by_lua_block {
                    local sni = require("resty.sniproxy")
                    local sp = sni:new()
                    sp:run()
            }   
    }
}

stream {

lua_resolver 8.8.8.8;

init_worker_by_lua_block {

sni_rules = {

["www.google.com"] = {"www.google.com", 443},

["www.facebook.com"] = {"9.8.7.6", 443},

["twitter.com"] = {"1.2.3.4"},

[".+.twitter.com"] = {nil, 443}

}

server {

error_log /var/log/nginx/sniproxy-error.log error;

listen 443;

content_by_lua_block {

local sni = require("resty.sniproxy")

local sp = sni:new()

sp:run()

}

A Lua table sni_rules should be defined in the init_worker_by_lua_block directive.

The key can be either whole host name or regular expression. Use . for a default host name. If no entry is matched, connection will be closed.

The value is a table containing host name and port. If host is set to nil, the server_name in SNI will be used. If the port is not defined or set to nil, 443 will be used.

Rules are applied with the priority as its occurrence sequence in the table. In the example above, twitter.com will match the third rule rather than the fourth.

If the protocol version is less than TLSv1 (eg. SSLv3, SSLv2), connection will be closed, since SNI extension is not supported in these versions.

nginx 批量配置同步

on 2015 年 11 月 5 日

0 71482 转为繁体

在编译了lua-nginx-module的nginx上，可以方便地使用shared dict特性，在不reload配置文件的情况下实现配置同步。

由于shared dict使用一块共享内存，因此所有worker均可读写，也就不存在一致性的问题。

使用shared dict

nginx/openresty的一些记录

on 2015 年 10 月 16 日

28 41233 转为繁体

日志

屏蔽user-agent并屏蔽日志

if ($http_user_agent ~* (^badbot/useragent$)) {
    rewrite (.*) /badbot break;
}

location = /badbot {
    access_log off;
    return 403;
}

if ($http_user_agent ~* (^badbot/useragent$)) {

rewrite (.*) /badbot break;

}

location = /badbot {

access_log off;

return 403;

}

不屏蔽user-agent（允许其访问），但屏蔽日志

server {
    set $is_spider 1;
    if ($http_user_agent ~* "bot|spider|Bot|Disqus|WebIndex|YunGuanCe") {
        set $is_spider 0;
    }
    access_log /var/log/nginx/access.log combined if=$is_spider;
}

server {

set $is_spider 1;

if ($http_user_agent ~* "bot|spider|Bot|Disqus|WebIndex|YunGuanCe") {

set $is_spider 0;

}

access_log /var/log/nginx/access.log combined if=$is_spider;

}

按uri屏蔽日志（可以和上面的按user-agent用同一个变量来同时过滤uri和user-agent)

map $request $loggable {
    ~/favicon.ico 0;
    ~/images/* 0;
    ~/js/* 0;
    ~/css/* 0;
    default 1;
}

server {
    access_log /var/log/nginx/access.log combined if=$loggable;
}

map $request $loggable {

~/favicon.ico 0;

~/images/* 0;

~/js/* 0;

~/css/* 0;

default 1;

}

server {

access_log /var/log/nginx/access.log combined if=$loggable;

}

Header

按mime type设置缓存时间

map $sent_http_content_type $cacheable_types {
    "text/css"    "max-age=14400";
    "text/plain"  "max-age=86400";
    "image/jpeg"  "max-age=86400";
    "image/png"   "max-age=86400";
    "image/gif"   "max-age=86400";
    "image/x-icon" "max-age=86400";
    "application/x-7z-compressed" "max-age=864000";
    "application/x-javascript" "max-age=86400";
    "application/json" "max-age=86400";
    "application/x-bittorrent" "max-age=864000";
    default       "";
}

server {
    location / {
        root   "/var/www/html/";           
        add_header "Cache-Control" $cacheable_types;
    }
}

map $sent_http_content_type $cacheable_types {

"text/css" "max-age=14400";

"text/plain" "max-age=86400";

"image/jpeg" "max-age=86400";

"image/png" "max-age=86400";

"image/gif" "max-age=86400";

"image/x-icon" "max-age=86400";

"application/x-7z-compressed" "max-age=864000";

"application/x-javascript" "max-age=86400";

"application/json" "max-age=86400";

"application/x-bittorrent" "max-age=864000";

default "";

}

server {

location / {

root "/var/www/html/";

add_header "Cache-Control" $cacheable_types;

}

防攻击

简单的无状态cookie challenge（需要lua-nginx-module)

crawlers块中可以手动填写要屏蔽的IP

将其中的s改成随机字符串＋时间戳可以变成有状态版本（需使用redis/memcached/shared memory存储生成的随机字符串）

将set-cookie改成通过js生成cookie可以变成javascript challenge，注意要在js里加上浏览器上下文判断，如var cookie=location.protocol?cookie:””; 或者DOM操作

这里有个更高级的输验证码的示例

geo $crawlers{
    1.2.3.0/24 1;
    4.5.6.7 1;
    default 0;
}
server {
    location ~ \.php {
        set $_c 0;
        if ($http_user_agent ~* "bad.guy/123") {
            set $_c 1;
        }            
        if ($crawlers){
            set $_c 1;
        }
        if ($_c){
            access_log     off;
            access_by_lua_block {
                local expires = 90
                local s = ngx.time()
                local cob = tonumber(ngx.var.cookie_cob)
                if cob == nil then cob=0 end
                local coa = ngx.md5("saltsalt" .. ngx.var.remote_addr .. "saltsalt" .. cob)
                if s - cob > expires or ngx.var.cookie_coa ~= coa then
                    coa = ngx.md5("saltsalt" .. ngx.var.remote_addr .. "saltsalt" .. s)
                    ngx.header["Set-Cookie"] = {"coa=" .. coa .."; path=/; domain=.example.com; HttpOnly",
                        "cob=" .. s.."; path=/; domain=.example.com; HttpOnly"}
                    ngx.header["Content-Type"] = "text/html"
                    ngx.say("<meta http-equiv ='refresh' content='0'>");
                    ngx.exit(200)
                end
            }
        }
        # fastcgi_pass
    }
}

geo $crawlers{

1.2.3.0/24 1;

4.5.6.7 1;

default 0;

}

server {

location ~ \.php {

set $_c 0;

if ($http_user_agent ~* "bad.guy/123") {

set $_c 1;

}

if ($crawlers){

set $_c 1;

}

if ($_c){

access_log off;

access_by_lua_block {

local expires = 90

local s = ngx.time()

local cob = tonumber(ngx.var.cookie_cob)

if cob == nil then cob=0 end

local coa = ngx.md5("saltsalt" .. ngx.var.remote_addr .. "saltsalt" .. cob)

if s - cob > expires or ngx.var.cookie_coa ~= coa then

coa = ngx.md5("saltsalt" .. ngx.var.remote_addr .. "saltsalt" .. s)

ngx.header["Set-Cookie"] = {"coa=" .. coa .."; path=/; domain=.example.com; HttpOnly",

"cob=" .. s.."; path=/; domain=.example.com; HttpOnly"}

ngx.header["Content-Type"] = "text/html"

ngx.say("<meta http-equiv ='refresh' content='0'>");

ngx.exit(200)

end

}

# fastcgi_pass

}

其他

植入cookie

~~需要注意的是使用ngx.time()产生秒级的时间，用来做随机数种子可能会冲突，因此建议加上另外的随机变量（如下面的例子用的是客户端的ip）~~ 可以使用ngx.now()产生毫秒精度时间

local modname = "util"
local _M = { _VERSION = '0.01' }
local mt = { __index = _M }

function _M.random_str(l, seed, r)
    local s = r or 'abcdefghijklmnhopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789=+-'
 
    local ret =''
	-- os.time() is precise to second
	math.randomseed(os.time() * 1000 + ngx.crc32_short(seed or ""))
    for i=1 ,l do
        local pos = math.random(1, string.len(s))
        ret = ret .. string.sub(s, pos, pos)
    end
 
    return ret
end

function _M.plant_cookie(k, h)
	local ck = require "resty.cookie"
	local cookie, err = ck:new()
	if not cookie then
		ngx.log(ngx.ERR, "ERROR PLANTING COOKIE", err)
		return
	end
	
	local _, err = cookie:get(k)
	if _ then
		return
	end
	local ok, err = cookie:set({
		key = k, value = _M.random_str(32, ngx.var.remote_addr, '0123456789abcdef'), path = "/",
		domain = h or ngx.var.http_host,
		httponly = true,
		expires= 'Thu, 31-Dec-37 23:55:55 GMT',
		max_age = 2147483647
	})

end

return _M

local modname = "util"

local _M = { _VERSION = '0.01' }

local mt = { __index = _M }

function _M.random_str(l, seed, r)

local s = r or 'abcdefghijklmnhopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789=+-'

local ret =''

-- os.time() is precise to second

math.randomseed(os.time() * 1000 + ngx.crc32_short(seed or ""))

for i=1 ,l do

local pos = math.random(1, string.len(s))

ret = ret .. string.sub(s, pos, pos)

end

return ret

end

function _M.plant_cookie(k, h)

local ck = require "resty.cookie"

local cookie, err = ck:new()

if not cookie then

ngx.log(ngx.ERR, "ERROR PLANTING COOKIE", err)

return

end

local _, err = cookie:get(k)

if _ then

return

end

local ok, err = cookie:set({

key = k, value = _M.random_str(32, ngx.var.remote_addr, '0123456789abcdef'), path = "/",

domain = h or ngx.var.http_host,

httponly = true,

expires= 'Thu, 31-Dec-37 23:55:55 GMT',

max_age = 2147483647

})

end

return _M

location / {
    access_by_lua '
        local _ = require("util")
        _.plant_cookie("cookie_name",".example.come")
    ';
}

location / {

access_by_lua '

local _ = require("util")

_.plant_cookie("cookie_name",".example.come")

}

Lua用setmetatable返回默认值为function时的暗坑

on 2015 年 5 月 30 日

0 119079 转为繁体

想在api服务器里实现一个acl的功能，对某些请求（需要登录，需要检查appkey，需要限制频次等）做限制，对某些起始状态（比如登陆）或者终结状态（比如报错）的请求放行。

因为lua里木有switch case，因此通过一个acl_list的table去查找规则，因为需要限制的请求种类比较多，就把rule_check_token当成默认值了，一看是是这么写的：

local _M = {
    ACL_PASS = 0,
    ACL_DENIED = 1,
}

function _M.rule_always_pass(self)
    return _M.ACL_PASS
end

function _M.rule_check_token(query)
    if query == nil then
        return _M.ACL_DENIED
    end
    local token = query['token'] or ngx.req.get_headers()['Authorization']
    local uid = query['uid']
    -- ...
end

_M.acl_list = setmetatable({
    ['error'] = _M.rule_always_pass,
    ['user/login'] = _M.rule_always_pass,
}, {
    __index = _M.rule_check_token
})

return _M

local _M = {

ACL_PASS = 0,

ACL_DENIED = 1,

}

function _M.rule_always_pass(self)

return _M.ACL_PASS

end

function _M.rule_check_token(query)

if query == nil then

return _M.ACL_DENIED

end

local token = query['token'] or ngx.req.get_headers()['Authorization']

local uid = query['uid']

-- ...

end

_M.acl_list = setmetatable({

['error'] = _M.rule_always_pass,

['user/login'] = _M.rule_always_pass,

}, {

__index = _M.rule_check_token

})

return _M

然而却会在local token= xxxx那一行报stack overflow，想了半天也发现哪里有无限递归，因为query传进来的是http请求的query string解析出的键值对表。

把query打印出来一看，发现竟然是这个模块本身……

仔细看了文档才知道，原来__index后面的值是一个function时，lua会调用这个function去获得不存在的键，并且第一个参数是模块本身（即_M，一个table）。在这个例子里：

调用一个非默认规则的api
lua调用了rule_check_token
参数query被传入了_M本身
运行到local token= xxxx这一行
这个table里(_M)又没有token这个键
回到2

所以就死循环了

所以要好好看文档

解决方法是可以套一个function

 __index = function() return _M.rule_check_token end

1	__index = function() return _M.rule_check_token end

贴一个打印table的工具，方便调试：

function print_r(tbl, pref)
    for k,v in pairs(tbl) do 
        if type(v) == 'table' then
            print((pref or "|-"), k, "\t", "<table>")
            _print_r(v, "  "..(pref or "|-"))
        else
            if type(v) == 'function' then
                v = "   <function>"
            end
            print((pref or "|-"), k, "\t", v)
        end
    end
end

function print_r(tbl, pref)

for k,v in pairs(tbl) do

if type(v) == 'table' then

print((pref or "|-"), k, "\t", "<table>")

_print_r(v, " "..(pref or "|-"))

else

if type(v) == 'function' then

v = " <function>"

end

print((pref or "|-"), k, "\t", v)

end

可以打印出如下形式：

|-_VERSION      0.01
|-rule_always_pass         <function>
|-ACL_DENIED    1
|-acl_list      <table>
  |-error          <function>
  |-user/login     <function>
|-ACL_PASS      0
|-rule_check_token         <function>

|-_VERSION 0.01

|-rule_always_pass <function>

|-ACL_DENIED 1

|-acl_list <table>

|-error <function>

|-user/login <function>

|-ACL_PASS 0

|-rule_check_token <function>

使用openresty请自行改写成local function和ngx.say的形式

Category Archives

cidr.me地理位置查询

使用方法

源代码

用 OpenResty 写了一个 SNI 代理

nginx 批量配置同步

使用shared dict

nginx/openresty的一些记录

日志

Header

防攻击

其他

Lua用setmetatable返回默认值为function时的暗坑