关于docker
- docker create -it 其中 -it选项,如果没有的话,docker exec 进去,top时会报 TERM environment variable not set. 错误 ; 其实没有关系,手动 export TERM=xterm-256color 就可以了;其实, -it 选项和1号进程为 /bin/bash 是密切相关的; 如果1号进程不是 /bin/bash 的话,-it选项没有屁用
- docker 的CMD 其实是1号进程,一定不能立即就会退出的进程,如: service sshd start 就不行,如下脚本就可以:
123456#!/bin/bash# this is a start shell for containnerservice sshd startwhile :; dosleep 10000done
- 容器可以自杀: 容器内进程使用root kill -9 1 默认是杀不死1号进程的,但是:
当strace -p 1 的时候,kill -9 1 是可以杀死1号进程的 (难道说这是个bug?)
因为docker stop时是先给1号进程发送sigterm,是在不行才kill -9 ; 所以,1号进程最好能合理处理sigterm信号,最后退出;然而,容器内的root进程是有权限给1号进程发送sigterm信号的,所以,容器自杀其实是很方便的 - 改进版的1号进程
myinit.c
123456789#include <stdlib.h>#include <unistd.h>int main(int ac, char **av) {while(1){sleep(10000);}return 0;}
gcc -o myinit myinit.c
myinit.sh
1234#!/bin/bash# this is a start shell for containnerservice sshd startexec /sbin/myinit
- 其实还不够
你会慢慢发现,ssh连接失败,因为:sshd接到连接请求之后,检查是否session太多了(默认最大10个),超过了就不伺候了,直接断开; 为什么会太多了?sshd每次接到连接请求,会fork => fork (两次fork)出来一个子进程处理请求,就因为是两次fork,这个新的sshd进程便“过继”给了1号进程;但是上面的1号进程还不够成熟,不会带孩子,当这个新的sshd进程退出的时候,就需要1号进程来做一些处理(如:清理进程表项),但是1号进程还不会,于是新的sshd进程会处于 <defunct>状态;而sshd的祖先还以为该session还没结束,就不再接受新的请求了,解决办法:添加信号处理,如下:
1234567891011#include <stdlib.h>#include <unistd.h>#include <signal.h>int main(int ac, char **av) {signal(SIGCHLD, SIG_IGN);while(1){sleep(10000);}return 0;}
参考资料:
https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/
https://github.com/phusion/baseimage-docker/blob/rel-0.9.16/image/bin/my_init
dockerd 提供了init 和init-path的选项,允许配置是否启动一个指定的(默认的)init进程, 该进程就是为了转发信号和收割僵尸进程 的 - 上面sshd对于连接请求是两次fork的说法不太对,见下图:
99、134进程并没有立即退出(不过,一旦意外退出,则 101、136将会被1号进程收养)
还有一种情况为:sshd被restart了,这时候30号进程退出,99、134被1号进程收养,如下:
由于我重启了sshd,再次连接就会失败,如下:
为什么呢?
稍后再查,至少可以确定的是,启动容器时候启动的sshd是可以正常服务的,登录进去重启的sshd是不能正常服务的
see also : https://docs.docker.com/engine/examples/running_ssh_service/
关于docker的内存限制:
- 如果每个docker容器限制最大使用4G,则可能会使用超过4G,(似乎是使用了交换分区了)
- 如果宿主机有64G内存,则允许docker容器内存总和大于64G,(这个不会也和交换分区有关吧)
- 文档说–memory-swap=-1 就禁止使用交换分区了,但是测试发现还是使用了1倍内存的交换分区
缅怀一下阿里云云引擎
学习一下人家下线产品是咋写的:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
阿里云云引擎ACE产品转型下线公告 尊敬的阿里云用户: 您好。 自ACE产品上线以来,我们致力于为用户提供更好的服务以及体验。 今日,我们非常遗憾的通知您,由于产品体系升级,阿里云云引擎ACE产品将于2016年5月12日整体下线,届时ACE产品(包括扩展服务)将全部停止服务(整体下线安排见后)。我们推荐您使用云服务器ECS、弹性web托管、容器服务等其他云计算产品。 在此期间,ACE产品团队将会: 1、 如选择使用ECS或弹性web托管服务,我们将提供相应的代金券; 2、 在用户管理控制台为您提供数据下载的快速通道,您可通过此快速通道进行数据下载\备份以及迁移的工作(如您选择阿里云其他产品的,阿里云并将提供对应的迁移方案); 3、 针对未到期的包年包月ACE产品,阿里云将启动退款程序(具体方案将通过站内信或邮件发送给您)。 请在ACE服务到期日之前安排好数据,逾期将无法找回。 如您在下线过程中遇到问题,可直接从阿里云用户中心—工单管理—提交工单-云引擎ACE提交工单。 因产品体系升级给您带来不便,我们深表歉意。 再次感谢您对阿里云的支持! 阿里云计算有限公司 2016年4月6日 常用Q&A 1、云引擎ACE为什么要下线? 答:您好,非常抱歉,由于产品体系升级,业务调整影响,云引擎ACE将不再提供服务。 2、已买了云引擎ACE的包年包月某某版本,在2016年4月6日以后服务还未到期,产品下线了怎么办? 答:您好,已购用户一方面我们会根据您购买ACE的不同语言针对性的推荐阿里云其它产品并提供对应的迁移方案,另一方面会启动退款程序,已购用户的邮箱或者站内信、短信4月6日会统一收到通知,请收到的用户及时查到您的手机、邮箱或阿里云账户中心 3、已买了按量付费云引擎ACE,产品下线了怎么办? 答:您好,一方面我们会根据您购买ACE的不同语言针对性的推荐阿里云其它产品并提供对应的迁移方案,另一方面我们已争取替代产品的最大化代金券,已发放至您的账号,您可以直接用来购买新产品 4、产品下线了,我的应用数据怎么办 答:您好,5月12正式下线之前,我们会一直提供数据下载快速通道,您可以随时做好数据下线和备份工作,若您选择阿里云其他产品,我们提供推荐产品的迁移方案,您可直接根据方案迁移数据 5、我的应用正好在你产品下线期间到期了怎么办? 答:您好,为了保证用户体验,在4月6日-5月12日期间到期的客户可以继续使用应用直至5月12日产品正式下线为止。 6、你们产品下线,推荐我购买其他产品有没有优惠? 答:您好,我们已争取其他产品的最大化优惠代金券,已发放至您的账号,您可以直接用来购买新产品,代金券有效期为30天,请及时使用 。 7、下线期限,包年包月超过流量套餐配置外流量还收费吗? 答:您好,4月6日-5月12日针对超套餐的流量系统仍计费,超出部分按照¥0.80/GB收费,基本版和普通版每个月50GB免费公网流出流量,流入流量免费,专业版每个月80GB免费公网流出流量,流入流量免费 |
关于API网关
- https://tyk.io/
- 阿里云API网关: https://help.aliyun.com/product/9194896_apigateway.html
- AWS API网关:https://aws.amazon.com/cn/api-gateway/
为什么API网关也能成为一种服务: http://www.d1net.com/cloud/vendors/359931.html
关于httpdns
概要
httpdns就是通过http的方式进行域名解析;阿里云有提供httpdns服务(目前2016-4-9 还处于公测阶段,需要申请公测资格才能用)。
传统dns解析存在的问题:
- 由于传统dns解析采用udp方式,而且是明文,所以
- 可能会被篡改
- 可能会被投毒(什么叫“投毒”?且看下文)
httpdns的实现原理:
- 通过http的方式进行域名解析,如:
12curl http://203.107.1.1/100000/d?host=www.aliyun.com{"host":"www.aliyun.com","ips":["140.205.63.8"],"ttl":274} - 通过设置http host header的方式发起http请求,如:
1curl -H"Host: www.aliyun.com" http://140.205.115.67/
问题:
- httpdns server的域名如何解析?
httpdns server 不通过域名提供服务,(只能)直接通过一个固定IP提供服务;服务提供商有足够的手段让这个IP不会出现宕机,而且世界各地(或者是全国各地)访问都能很快(就近访问),这个IP在世界上存在多份。 - 如何保证连接的这个ip是没被劫持呢?
- 或许可以给这个IP买个https证书,https证书提供商是可以给IP颁发证书的(比较少见),不过可能贵的多
优点:(参考文档: https://www.aliyun.com/product/httpdns?spm=5176.7960203.223922.4.2JohAO)
- 防劫持
- 避免dns服务商莫名其妙的cache,可以做到立即修改理解生效
- 精准调度
123456对于地址:http://203.107.1.1/100000/d?host=www.aliyun.com&ip=111.111.11.111其中:100000是你的账户ID,host参数是你要解析的域名,ip参数是你的来源IP(用来做精准调度的);你可以在你的账户中设置允许解析的域名,以及要解析到的地址,甚至可以根据来源IP解析到不同的IP,做到精准调度
问题:
- https请求问题及解决方案: https://help.aliyun.com/document_detail/httpdns/practice/https.html?spm=5176.7947101.220063.5.9DtXGS
- 这里回避了单个IP上多个https证书的问题,或许也可以通过钩子解决,但是不一定存在这样的钩子; ssl握手的时候,必须告知正确的host信息,否则,服务器无法知道该返回哪个证书,而不仅仅是验证证书的问题
- httpdns只适用于app、client的情况,对于web浏览器是不行的,还好现在大家都是在玩app了;
- 对于app里面适用webview的情况需要特殊处理一下:https://help.aliyun.com/document_detail/httpdns/practice/webview.html?spm=5176.dochttpdns/user-guide/resolve-stats.6.127.VFTz3N;
- 对于通过react native来开发app的情况不知道是否方便使用
dns投毒:
因为域名解析是udp的,只要我已dns server的ip不断地向你的机器发送域名解析结果响应的话,你可能就会认为确实是dns server响应的结果;当然,有几个关键点需要注意:
- 用户确实需要解析某域名(一般来讲,解析完了会cache一段时间的)
- 伪造的响应里面是需要包含用户正在查询的域名的,域名不匹配自然不会被接受
- 用户发起请求的端口号(2字节)是随机的,响应的数据包需要能匹配上这个端口,否则不会被接受
- 用户发起的请求有一个随机的请求标识(2字节),响应的数据包需要能匹配上这个请求标识,否则不会被接受
- 我咋感觉这个难度其实并不算小啊,除非你是个中间人
httpdns示例:
更多参考:
全局精确流量调度新思路-HttpDNS服务详解: http://www.zmke.com/i/8705.html
渗透测试:内网DNS投毒技术劫持会话:http://www.freebuf.com/articles/web/43157.html
http://www.onedns.net/ 需要特殊的dns客户端
加密的dns: https://www.opendns.com/about/innovations/dnscrypt/ 需要特殊的dns客户端
wireshark分析dns协议: http://blog.csdn.net/hunanchenxingyu/article/details/21488291
dns协议格式: http://cjhust.blog.163.com/blog/static/1758271572014111875652363/
关于shell一些小知识点
关于 [[]] 和 [] 的区别
实例1:
说明:
[] 和 [[]] 是不同的语法;
对于 [
事实上, bash 中没有 [] 语法, [ 是一个命令,出于美观考虑,要求有一个 ] 与之配对; 所以 [ 的前后是要有空格的
对于 [[
[[]] 是bash的内建语法
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
[[ expression ]] Return a status of 0 or 1 depending on the evaluation of the conditional expression expression. Expressions are composed of the primaries described below under CONDITIONAL EXPRESSIONS. Word splitting and pathname expansion are not performed on the words between the [[ and ]]; tilde expansion, parameter and variable expansion, arithmetic expansion, command substitution, process substitution, and quote removal are performed. Conditional operators such as -f must be unquoted to be recognized as primaries. When the == and != operators are used, the string to the right of the operator is consid- ered a pattern and matched according to the rules described below under Pattern Matching. If the shell option nocasematch is enabled, the match is performed without regard to the case of alphabetic characters. The return value is 0 if the string matches (==) or does not match (!=) the pattern, and 1 otherwise. Any part of the pattern may be quoted to force it to be matched as a string. An additional binary operator, =~, is available, with the same precedence as == and !=. When it is used, the string to the right of the operator is considered an extended regu- lar expression and matched accordingly (as in regex(3)). The return value is 0 if the string matches the pattern, and 1 otherwise. If the regular expression is syntactically incorrect, the conditional expression's return value is 2. If the shell option nocase- match is enabled, the match is performed without regard to the case of alphabetic charac- ters. Substrings matched by parenthesized subexpressions within the regular expression are saved in the array variable BASH_REMATCH. The element of BASH_REMATCH with index 0 is the portion of the string matching the entire regular expression. The element of BASH_REMATCH with index n is the portion of the string matching the nth parenthesized subexpression. |
- 如果是 == ,则右边被视为 Pattern; 如果是 =~ ,则右边视为正则; 也就是说,左边和右边是不等价的
- [[]] 允许内部 与、或、非; 如: [[ 1 == 1 && 2 == 2 ]]; 而 [] 则不可以,只能: [ 1 == 1 ] && [ 2 == 2 ]
bash基本概念
- bash中有保留字(reserved word)和元字符(meta charactor)的区别的,这个很重要
- 关键字:
1! case do done elif else esac fi for function if in select then until while { } time [[ ]] - 元字符:
1| & ; ( ) < > space tab - xx
- 关键字:
- 对于关键字,和其他非元字符的字符都需要用空白字符分开,否则,关键字就识别不出来了; 因为元字符是被特殊对待的,所以,关键字挨着元字符是允许的,所以,如果分不清关键字和元字符的话,就不知道什么时候需要用空白符分隔,什么时候不需要空白符分隔
man bash
关于元字符、保留字
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
DEFINITIONS The following definitions are used throughout the rest of this document. blank A space or tab. word A sequence of characters considered as a single unit by the shell. Also known as a token. name A word consisting only of alphanumeric characters and underscores, and beginning with an alphabetic character or an underscore. Also referred to as an identifier. metacharacter A character that, when unquoted, separates words. One of the following: | & ; ( ) < > space tab control operator A token that performs a control function. It is one of the following symbols: || & && ; ;; ( ) | <newline> RESERVED WORDS Reserved words are words that have a special meaning to the shell. The following words are rec- ognized as reserved when unquoted and either the first word of a simple command (see SHELL GRAM- MAR below) or the third word of a case or for command: ! case do done elif else esac fi for function if in select then until while { } time [[ ]] |
关于 花括弧
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
Brace Expansion Brace expansion is a mechanism by which arbitrary strings may be generated. This mechanism is similar to pathname expansion, but the filenames generated need not exist. Patterns to be brace expanded take the form of an optional preamble, followed by either a series of comma-separated strings or a sequence expression between a pair of braces, followed by an optional postscript. The preamble is prefixed to each string contained within the braces, and the postscript is then appended to each resulting string, expanding left to right. Brace expansions may be nested. The results of each expanded string are not sorted; left to right order is preserved. For example, a{d,c,b}e expands into `ade ace abe'. A sequence expression takes the form {x..y}, where x and y are either integers or single charac- ters. When integers are supplied, the expression expands to each number between x and y, inclu- sive. When characters are supplied, the expression expands to each character lexicographically between x and y, inclusive. Note that both x and y must be of the same type. Brace expansion is performed before any other expansions, and any characters special to other expansions are preserved in the result. It is strictly textual. Bash does not apply any syn- tactic interpretation to the context of the expansion or the text between the braces. A correctly-formed brace expansion must contain unquoted opening and closing braces, and at least one unquoted comma or a valid sequence expression. Any incorrectly formed brace expansion is left unchanged. A { or , may be quoted with a backslash to prevent its being considered part of a brace expression. To avoid conflicts with parameter expansion, the string ${ is not con- sidered eligible for brace expansion. This construct is typically used as shorthand when the common prefix of the strings to be gener- ated is longer than in the above example: mkdir /usr/local/src/bash/{old,new,dist,bugs} or chown root /usr/{ucb/{ex,edit},lib/{ex?.?*,how_ex}} Brace expansion introduces a slight incompatibility with historical versions of sh. sh does not treat opening or closing braces specially when they appear as part of a word, and preserves them in the output. Bash removes braces from words as a consequence of brace expansion. For exam- ple, a word entered to sh as file{1,2} appears identically in the output. The same word is out- put as file1 file2 after expansion by bash. If strict compatibility with sh is desired, start bash with the +B option or disable brace expansion with the +B option to the set command (see SHELL BUILTIN COMMANDS below). |
命令替换
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
Command Substitution Command substitution allows the output of a command to replace the command name. There are two forms: $(command) or `command` Bash performs the expansion by executing command and replacing the command substitution with the standard output of the command, with any trailing newlines deleted. Embedded newlines are not deleted, but they may be removed during word splitting. The command substitution $(cat file) can be replaced by the equivalent but faster $(< file). When the old-style backquote form of substitution is used, backslash retains its literal meaning except when followed by $, `, or \. The first backquote not preceded by a backslash terminates the command substitution. When using the $(command) form, all characters between the parenthe- ses make up the command; none are treated specially. Command substitutions may be nested. To nest when using the backquoted form, escape the inner backquotes with backslashes. If the substitution appears within double quotes, word splitting and pathname expansion are not performed on the results. |
docker虚拟机设置固定IP
参考文章: http://www.tuicool.com/articles/v2yQ7bA
创建网桥(可以写成一个脚本):
1 2 3 4 5 |
br_name=docker brctl addbr $br_name ip addr add 192.168.33.2/24 dev $br_name ip link set $br_name up brctl addif $br_name eth0 |
创建容器
1 |
docker create --net none -i -t docker.io/blalor/centos /bin/bash |
稍微整理一下原作者的命令,写一个脚本:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 |
#!/bin/bash cid=$1 #containerid ip=$2 #192.168.199.200/24 gateway=$3 #192.168.199.1 br_name=$4 #br1 eth=$5 #eth1 if [[ "$1" == "-h" || "$1" == "--help" ]]; then echo "sh $0 cid ip gateway brname eth";exit 1 fi pid=$(docker inspect -f '{{.State.Pid}}' $cid) if [[ "$pid" == "" ]]; then echo "container $cid is not exists";exit 2 fi if [[ $pid == 0 ]]; then echo "container $cid is not running" echo -n "starting $cid..."; docker start $cid if [[ $? != 0 ]];then echo "[ fail ]";exit 3 fi echo "[ OK ]" pid=$(docker inspect -f '{{.State.Pid}}' $cid) fi # set up netns netns_dir=/var/run/netns [ -d $netns_dir ] || mkdir -p /var/run/netns # garbage collect for f in $netns_dir/*;do _pid=$(basename $f) if [ "*" == "$f" ]; then break;fi if [ "$pid" == "$_pid" ]; then echo "network is allready yet";exit 4 fi if ! kill -0 $_pid 2>/dev/null;then ip netns delete $_pid fi done ln -s /proc/$pid/ns/net /var/run/netns/$pid # set up bridge ip link add q$pid type veth peer name r$pid brctl addif $br_name q$pid ip link set q$pid up # set up docker interface ip link set r$pid netns $pid ip netns exec $pid ip link set dev r$pid name $eth ip netns exec $pid ip link set $eth up ip netns exec $pid ip addr add $ip dev $eth ip netns exec $pid ip route add default via $gateway |
注: 最新版本的iproute (至少在iproute-3.10.0-54.el7.x86_64 中)支持 -n 选项,功能为:
1 2 3 4 5 6 7 8 |
-n, -netns <NETNS> switches ip to the specified network namespace NETNS. Actually it just simplifies executing of: ip netns exec NETNS ip [ OPTIONS ] OBJECT { COMMAND | help } to ip -n[etns] NETNS [ OPTIONS ] OBJECT { COMMAND | help } |
至少 iproute-3.10.0-21.el7.x86_64 中还没有该选项, 也就是说
1 |
ip netns exec 7819 ip addr show |
等价于
1 |
ip -n 7819 addr show |
如下br1是我做的网桥:
当docker容器stop时,这里的q7443也就消失了,但是网络名字空间中依然会存在; 可以通过 ip netns delete 来删除
一种情况,由于我的docker宿主机是virtualbox做的虚拟机,virtualbox虚拟机使用的桥接模式,所以,我这么做出来的docker虚拟机是访问不了外网的;因为,尽管docker虚拟机可以往docker宿主机的网卡上写数据,virtualbox的宿主机上的后台进程也不会发送该数据的(至少不会接受响应的数据)。
其实,有一个叫做pipework的工具可以很方便地给容器设置ip; 但是都没有提供卸载容器网卡的功能,有一种比较方便的办法: nsenter 进去 ifdown 就可以了
关于网桥
为什么virtualbox网卡桥接模式下在宿主机上看不到网桥?
virtualbox在后台启动了一个进程,很牛B,能从指定的网卡读取数据和写入数据,如此,就可以做到将某个虚拟机要发送出去的数据写入到指定网卡,而且网卡上收到的传给某个虚拟机(IP)的数据转发给指定虚拟机; 就这么简单。
参考: http://superuser.com/questions/594550/how-does-bridged-networking-work-in-virtualbox
dnsmasq 的一个小配置
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
listen-address=172.16.10.4 # If you want dnsmasq to provide only DNS service on an interface, # configure it as shown above, and then use the following line to # disable DHCP on it. #no-dhcp-interface= # On systems which support it, dnsmasq binds the wildcard address, # even when it is listening on only some interfaces. It then discards # requests that it shouldn't reply to. This has the advantage of # working even when interfaces come and go and change address. If you # want dnsmasq to really bind only the interfaces it is listening on, # uncomment this option. About the only time you may need this is when # running another nameserver on the same machine. bind-interfaces |
一个cobbler错误
在做一个yum源镜像的时候,遇到一个错误: cobbler reposync –only centos-7-aliyun-x86_64-epel
Error: xz compression not available
解决办法:
yum install pyliblzma