- https://tyk.io/
- 阿里云API网关： https://help.aliyun.com/product/9194896_apigateway.html
- AWS API网关：https://aws.amazon.com/cn/api-gateway/
为什么API网关也能成为一种服务： http://www.d1net.com/cloud/vendors/359931.html
关于httpdns
概要
httpdns就是通过http的方式进行域名解析；阿里云有提供httpdns服务（目前2016-4-9 还处于公测阶段，需要申请公测资格才能用）。
传统dns解析存在的问题：
- 由于传统dns解析采用udp方式，而且是明文，所以
- 可能会被篡改
- 可能会被投毒（什么叫“投毒”？且看下文）
httpdns的实现原理：
- 通过http的方式进行域名解析，如：
12curl http://203.107.1.1/100000/d?host=www.aliyun.com{"host":"www.aliyun.com","ips":["140.205.63.8"],"ttl":274} - 通过设置http host header的方式发起http请求，如：
1curl -H"Host: www.aliyun.com" http://140.205.115.67/
问题：
- httpdns server的域名如何解析？
httpdns server 不通过域名提供服务，（只能）直接通过一个固定IP提供服务；服务提供商有足够的手段让这个IP不会出现宕机，而且世界各地（或者是全国各地）访问都能很快（就近访问），这个IP在世界上存在多份。 - 如何保证连接的这个ip是没被劫持呢？
- 或许可以给这个IP买个https证书，https证书提供商是可以给IP颁发证书的（比较少见），不过可能贵的多
优点：（参考文档： https://www.aliyun.com/product/httpdns?spm=5176.7960203.223922.4.2JohAO）
- 防劫持
- 避免dns服务商莫名其妙的cache，可以做到立即修改理解生效
- 精准调度
123456对于地址：http://203.107.1.1/100000/d?host=www.aliyun.com&ip=111.111.11.111其中：100000是你的账户ID，host参数是你要解析的域名，ip参数是你的来源IP（用来做精准调度的）；你可以在你的账户中设置允许解析的域名，以及要解析到的地址，甚至可以根据来源IP解析到不同的IP，做到精准调度
问题：
- https请求问题及解决方案： https://help.aliyun.com/document_detail/httpdns/practice/https.html?spm=5176.7947101.220063.5.9DtXGS
- 这里回避了单个IP上多个https证书的问题，或许也可以通过钩子解决，但是不一定存在这样的钩子； ssl握手的时候，必须告知正确的host信息，否则，服务器无法知道该返回哪个证书，而不仅仅是验证证书的问题
- httpdns只适用于app、client的情况，对于web浏览器是不行的，还好现在大家都是在玩app了；
- 对于app里面适用webview的情况需要特殊处理一下：https://help.aliyun.com/document_detail/httpdns/practice/webview.html?spm=5176.dochttpdns/user-guide/resolve-stats.6.127.VFTz3N；
- 对于通过react native来开发app的情况不知道是否方便使用
dns投毒：
因为域名解析是udp的，只要我已dns server的ip不断地向你的机器发送域名解析结果响应的话，你可能就会认为确实是dns server响应的结果；当然，有几个关键点需要注意：
- 用户确实需要解析某域名（一般来讲，解析完了会cache一段时间的）
- 伪造的响应里面是需要包含用户正在查询的域名的，域名不匹配自然不会被接受
- 用户发起请求的端口号（2字节）是随机的，响应的数据包需要能匹配上这个端口，否则不会被接受
- 用户发起的请求有一个随机的请求标识（2字节），响应的数据包需要能匹配上这个请求标识，否则不会被接受
- 我咋感觉这个难度其实并不算小啊，除非你是个中间人
httpdns示例：
更多参考：
全局精确流量调度新思路-HttpDNS服务详解： http://www.zmke.com/i/8705.html
渗透测试：内网DNS投毒技术劫持会话：http://www.freebuf.com/articles/web/43157.html
http://www.onedns.net/ 需要特殊的dns客户端
加密的dns： https://www.opendns.com/about/innovations/dnscrypt/ 需要特殊的dns客户端
wireshark分析dns协议： http://blog.csdn.net/hunanchenxingyu/article/details/21488291
dns协议格式： http://cjhust.blog.163.com/blog/static/1758271572014111875652363/
关于shell一些小知识点
关于 [[]] 和 [] 的区别
实例1：
说明：
[] 和 [[]] 是不同的语法；
对于 [
事实上， bash 中没有 [] 语法， [ 是一个命令，出于美观考虑，要求有一个 ] 与之配对； 所以 [ 的前后是要有空格的
对于 [[
[[]] 是bash的内建语法
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
[[ expression ]] Return a status of 0 or 1 depending on the evaluation of the conditional expression expression. Expressions are composed of the primaries described below under CONDITIONAL EXPRESSIONS. Word splitting and pathname expansion are not performed on the words between the [[ and ]]; tilde expansion, parameter and variable expansion, arithmetic expansion, command substitution, process substitution, and quote removal are performed. Conditional operators such as -f must be unquoted to be recognized as primaries. When the == and != operators are used, the string to the right of the operator is consid- ered a pattern and matched according to the rules described below under Pattern Matching. If the shell option nocasematch is enabled, the match is performed without regard to the case of alphabetic characters. The return value is 0 if the string matches (==) or does not match (!=) the pattern, and 1 otherwise. Any part of the pattern may be quoted to force it to be matched as a string. An additional binary operator, =~, is available, with the same precedence as == and !=. When it is used, the string to the right of the operator is considered an extended regu- lar expression and matched accordingly (as in regex(3)). The return value is 0 if the string matches the pattern, and 1 otherwise. If the regular expression is syntactically incorrect, the conditional expression's return value is 2. If the shell option nocase- match is enabled, the match is performed without regard to the case of alphabetic charac- ters. Substrings matched by parenthesized subexpressions within the regular expression are saved in the array variable BASH_REMATCH. The element of BASH_REMATCH with index 0 is the portion of the string matching the entire regular expression. The element of BASH_REMATCH with index n is the portion of the string matching the nth parenthesized subexpression. |
- 如果是 == ，则右边被视为 Pattern； 如果是 =~ ，则右边视为正则； 也就是说，左边和右边是不等价的
- [[]] 允许内部 与、或、非； 如： [[ 1 == 1 && 2 == 2 ]]; 而 [] 则不可以，只能： [ 1 == 1 ] && [ 2 == 2 ]
bash基本概念
- bash中有保留字(reserved word)和元字符(meta charactor)的区别的，这个很重要
- 关键字：
1! case do done elif else esac fi for function if in select then until while { } time [[ ]] - 元字符：
1| & ; ( ) < > space tab - xx
- 关键字：
- 对于关键字，和其他非元字符的字符都需要用空白字符分开，否则，关键字就识别不出来了； 因为元字符是被特殊对待的，所以，关键字挨着元字符是允许的，所以，如果分不清关键字和元字符的话，就不知道什么时候需要用空白符分隔，什么时候不需要空白符分隔
man bash
关于元字符、保留字
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
DEFINITIONS The following definitions are used throughout the rest of this document. blank A space or tab. word A sequence of characters considered as a single unit by the shell. Also known as a token. name A word consisting only of alphanumeric characters and underscores, and beginning with an alphabetic character or an underscore. Also referred to as an identifier. metacharacter A character that, when unquoted, separates words. One of the following: | & ; ( ) < > space tab control operator A token that performs a control function. It is one of the following symbols: || & && ; ;; ( ) | <newline> RESERVED WORDS Reserved words are words that have a special meaning to the shell. The following words are rec- ognized as reserved when unquoted and either the first word of a simple command (see SHELL GRAM- MAR below) or the third word of a case or for command: ! case do done elif else esac fi for function if in select then until while { } time [[ ]] |
关于 花括弧
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
Brace Expansion Brace expansion is a mechanism by which arbitrary strings may be generated. This mechanism is similar to pathname expansion, but the filenames generated need not exist. Patterns to be brace expanded take the form of an optional preamble, followed by either a series of comma-separated strings or a sequence expression between a pair of braces, followed by an optional postscript. The preamble is prefixed to each string contained within the braces, and the postscript is then appended to each resulting string, expanding left to right. Brace expansions may be nested. The results of each expanded string are not sorted; left to right order is preserved. For example, a{d,c,b}e expands into `ade ace abe'. A sequence expression takes the form {x..y}, where x and y are either integers or single charac- ters. When integers are supplied, the expression expands to each number between x and y, inclu- sive. When characters are supplied, the expression expands to each character lexicographically between x and y, inclusive. Note that both x and y must be of the same type. Brace expansion is performed before any other expansions, and any characters special to other expansions are preserved in the result. It is strictly textual. Bash does not apply any syn- tactic interpretation to the context of the expansion or the text between the braces. A correctly-formed brace expansion must contain unquoted opening and closing braces, and at least one unquoted comma or a valid sequence expression. Any incorrectly formed brace expansion is left unchanged. A { or , may be quoted with a backslash to prevent its being considered part of a brace expression. To avoid conflicts with parameter expansion, the string ${ is not con- sidered eligible for brace expansion. This construct is typically used as shorthand when the common prefix of the strings to be gener- ated is longer than in the above example: mkdir /usr/local/src/bash/{old,new,dist,bugs} or chown root /usr/{ucb/{ex,edit},lib/{ex?.?*,how_ex}} Brace expansion introduces a slight incompatibility with historical versions of sh. sh does not treat opening or closing braces specially when they appear as part of a word, and preserves them in the output. Bash removes braces from words as a consequence of brace expansion. For exam- ple, a word entered to sh as file{1,2} appears identically in the output. The same word is out- put as file1 file2 after expansion by bash. If strict compatibility with sh is desired, start bash with the +B option or disable brace expansion with the +B option to the set command (see SHELL BUILTIN COMMANDS below). |
命令替换
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
Command Substitution Command substitution allows the output of a command to replace the command name. There are two forms: $(command) or `command` Bash performs the expansion by executing command and replacing the command substitution with the standard output of the command, with any trailing newlines deleted. Embedded newlines are not deleted, but they may be removed during word splitting. The command substitution $(cat file) can be replaced by the equivalent but faster $(< file). When the old-style backquote form of substitution is used, backslash retains its literal meaning except when followed by $, `, or \. The first backquote not preceded by a backslash terminates the command substitution. When using the $(command) form, all characters between the parenthe- ses make up the command; none are treated specially. Command substitutions may be nested. To nest when using the backquoted form, escape the inner backquotes with backslashes. If the substitution appears within double quotes, word splitting and pathname expansion are not performed on the results. |
docker虚拟机设置固定IP
参考文章： http://www.tuicool.com/articles/v2yQ7bA
创建网桥(可以写成一个脚本)：
|
1 2 3 4 5 |
br_name=docker brctl addbr $br_name ip addr add 192.168.33.2/24 dev $br_name ip link set $br_name up brctl addif $br_name eth0 |
创建容器
|
1 |
docker create --net none -i -t docker.io/blalor/centos /bin/bash |
稍微整理一下原作者的命令，写一个脚本：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 |
#!/bin/bash cid=$1 #containerid ip=$2 #192.168.199.200/24 gateway=$3 #192.168.199.1 br_name=$4 #br1 eth=$5 #eth1 if [[ "$1" == "-h" || "$1" == "--help" ]]; then echo "sh $0 cid ip gateway brname eth";exit 1 fi pid=$(docker inspect -f '{{.State.Pid}}' $cid) if [[ "$pid" == "" ]]; then echo "container $cid is not exists";exit 2 fi if [[ $pid == 0 ]]; then echo "container $cid is not running" echo -n "starting $cid..."; docker start $cid if [[ $? != 0 ]];then echo "[ fail ]";exit 3 fi echo "[ OK ]" pid=$(docker inspect -f '{{.State.Pid}}' $cid) fi # set up netns netns_dir=/var/run/netns [ -d $netns_dir ] || mkdir -p /var/run/netns # garbage collect for f in $netns_dir/*;do _pid=$(basename $f) if [ "*" == "$f" ]; then break;fi if [ "$pid" == "$_pid" ]; then echo "network is allready yet";exit 4 fi if ! kill -0 $_pid 2>/dev/null;then ip netns delete $_pid fi done ln -s /proc/$pid/ns/net /var/run/netns/$pid # set up bridge ip link add q$pid type veth peer name r$pid brctl addif $br_name q$pid ip link set q$pid up # set up docker interface ip link set r$pid netns $pid ip netns exec $pid ip link set dev r$pid name $eth ip netns exec $pid ip link set $eth up ip netns exec $pid ip addr add $ip dev $eth ip netns exec $pid ip route add default via $gateway |
注： 最新版本的iproute （至少在iproute-3.10.0-54.el7.x86_64 中）支持 -n 选项，功能为：
|
1 2 3 4 5 6 7 8 |
-n, -netns <NETNS> switches ip to the specified network namespace NETNS. Actually it just simplifies executing of: ip netns exec NETNS ip [ OPTIONS ] OBJECT { COMMAND | help } to ip -n[etns] NETNS [ OPTIONS ] OBJECT { COMMAND | help } |
至少 iproute-3.10.0-21.el7.x86_64 中还没有该选项， 也就是说
|
1 |
ip netns exec 7819 ip addr show |
等价于
|
1 |
ip -n 7819 addr show |
如下br1是我做的网桥：
当docker容器stop时，这里的q7443也就消失了，但是网络名字空间中依然会存在； 可以通过 ip netns delete 来删除
一种情况，由于我的docker宿主机是virtualbox做的虚拟机，virtualbox虚拟机使用的桥接模式，所以，我这么做出来的docker虚拟机是访问不了外网的；因为，尽管docker虚拟机可以往docker宿主机的网卡上写数据，virtualbox的宿主机上的后台进程也不会发送该数据的（至少不会接受响应的数据）。
其实，有一个叫做pipework的工具可以很方便地给容器设置ip； 但是都没有提供卸载容器网卡的功能，有一种比较方便的办法: nsenter 进去 ifdown 就可以了
关于网桥
为什么virtualbox网卡桥接模式下在宿主机上看不到网桥？
virtualbox在后台启动了一个进程，很牛B，能从指定的网卡读取数据和写入数据，如此，就可以做到将某个虚拟机要发送出去的数据写入到指定网卡，而且网卡上收到的传给某个虚拟机（IP）的数据转发给指定虚拟机； 就这么简单。
参考： http://superuser.com/questions/594550/how-does-bridged-networking-work-in-virtualbox
dnsmasq 的一个小配置
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
listen-address=172.16.10.4 # If you want dnsmasq to provide only DNS service on an interface, # configure it as shown above, and then use the following line to # disable DHCP on it. #no-dhcp-interface= # On systems which support it, dnsmasq binds the wildcard address, # even when it is listening on only some interfaces. It then discards # requests that it shouldn't reply to. This has the advantage of # working even when interfaces come and go and change address. If you # want dnsmasq to really bind only the interfaces it is listening on, # uncomment this option. About the only time you may need this is when # running another nameserver on the same machine. bind-interfaces |
一个cobbler错误
在做一个yum源镜像的时候，遇到一个错误： cobbler reposync –only centos-7-aliyun-x86_64-epel
Error: xz compression not available
解决办法：
yum install pyliblzma
PHP中sessionid的那点儿事儿
PHP中默认的sessionid为PHPSESSID;
当然，也可以通过
string session_name ([ string $name ] )
来修改， 官方说明：
name
The session name references the name of the session, which is used in cookies and URLs (e.g. PHPSESSID). It should contain only alphanumeric characters; it should be short and descriptive (i.e. for users with enabled cookie warnings). If name is specified, the name of the current session is changed to its value.
疑问： 只能包含字母和数字？
参考源码：
- 不能全数字
- 当然，sessionid毕竟是个cookie name，也会遵循cookie name的检查（参看： http://phpor.net/blog/post/1122 )
- 没有发现哪里体现“只能包含字母和数字”，实测也发现，其实 dot、_ 都是可以的
php-redis benchmark
脚本：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 |
<?php $pid = posix_getpid(); $host = '10.10.11.123'; $port = '6379'; $auth = 'thepassword'; $total = $slow = $fail = 0; $arr = array(); $t_start = microtime(1); for($i = 0; $i < $argv[1]; $i++) { $total++; $time_start = microtime(1); $r = new Redis(); $r->connect($host, $port, 1); $r->auth($auth); $r->select(3); $key = "test_${pid}_$i"; $val = "data_$i"; $r->set($key, $val); $v = $r->get($key); if ($v !== $val) { $fail++;continue; } $r->del($key); $time_end = microtime(1); $time_use = 1000 * ($time_end - $time_start); if ($time_use < 3) { @$arr["<3"]++; } else if ($time_use < 10) { @$arr["<10"]++; } else if ($time_use < 14) { @$arr["<14"]++; } else if ($time_use < 20) { @$arr["<20"]++; } else if ($time_use < 50) { @$arr["<50"]++; } else if ($time_use > 100) { @$arr[">100"]++; } } $t_end = microtime(1); echo "total: $total\t"; echo "fail: $fail\n"; echo "time use: ". (($t_end - $t_start) * 100) ."ms\n"; ksort($arr); printf("%4s: %6s %s(%%)\n", "ms", "count", "percent"); foreach($arr as $k=>$v) { printf("%4s: %6d %.2f%%\n", $k, $v, $v/$total*100); } |
阿里云的1G标准的redis，当同时40个这样的PHP进程压测的话，会有8个进程最终出现select时 1s 超时异常； 但是人家给的指标是支持 300并发的
关于syslog协议
在使用rsyslog的时候，一般来讲，如果消息中含有换行符的话，这条消息会以换行符为分隔，视为多条消息；即： rsyslog协议是基于行处理的，而且，默认单个消息大小为2k（rsyslogd v5），rsyslogd v8默认单个消息大小为8k。
问题： 一般来讲，程序出错时的堆栈信息都是多行的，这种情况该如何处理呢？
办法1： 先将堆栈信息中的换行替换成其它字符
办法2： 其实rsyslog是可以支持换行的，只是不是想换就换的，在structure data中可以小心地换行
关于syslog协议有两个rfc：
其中：
rfc3164 定义的syslog协议是比较原始的用法，不支持消息的换行
rfc5424 是所谓的version为1的syslog协议，支持 structure data；就是这个structure data中是可以包含换行的； 当然，要严格遵照定义的格式的哦
两种协议格式解析的golang实现： https://github.com/jeromer/syslogparser
rsyslog (rsyslog-8.17.0)源码学习：
- runtime/stream.c 中是这么解析的
翻译下：
mode=0 单行模式，遇到回车算做消息结束
mode=1 空行分隔模式，遇到空行算作消息结束（就是相邻消息之间要已空行分隔）
mode=2 非空开头模式， 只要下一行空白（应该也包含tab吧）开头就算作上个消息的一部分，不视为新消息
不幸的是，目前发现只有imfile模块才支持这三种模式之间的选择，更不幸的时，imfile模块使用inotify配置时，mode=2有bug（我测试时不是有bug，是根本不能用）