phpor

paramiko模块学习

8月 172017

第一步：安装

# pip install paramiko
Collecting paramiko
  Downloading paramiko-2.2.1-py2.py3-none-any.whl (176kB)
    100% |████████████████████████████████| 184kB 10kB/s
Collecting cryptography>=1.1 (from paramiko)
  Downloading cryptography-2.0.3-cp27-cp27mu-manylinux1_x86_64.whl (2.2MB)
    100% |████████████████████████████████| 2.2MB 5.1kB/s
Collecting bcrypt>=3.1.3 (from paramiko)
  Downloading bcrypt-3.1.3-cp27-cp27mu-manylinux1_x86_64.whl (57kB)
    100% |████████████████████████████████| 61kB 3.1kB/s
Collecting pyasn1>=0.1.7 (from paramiko)
^@  Downloading pyasn1-0.3.2-py2.py3-none-any.whl (63kB)
    100% |████████████████████████████████| 71kB 1.3kB/s
Collecting pynacl>=1.0.1 (from paramiko)
  Downloading PyNaCl-1.1.2-cp27-cp27mu-manylinux1_x86_64.whl (539kB)
    100% |████████████████████████████████| 542kB 4.4kB/s
Collecting idna>=2.1 (from cryptography>=1.1->paramiko)
  Downloading idna-2.6-py2.py3-none-any.whl (56kB)
    100% |████████████████████████████████| 61kB 4.5kB/s
Collecting six>=1.4.1 (from cryptography>=1.1->paramiko)
  Downloading six-1.10.0-py2.py3-none-any.whl
Collecting ipaddress (from cryptography>=1.1->paramiko)
  Downloading ipaddress-1.0.18-py2-none-any.whl
Collecting asn1crypto>=0.21.0 (from cryptography>=1.1->paramiko)
  Downloading asn1crypto-0.22.0-py2.py3-none-any.whl (97kB)
    100% |████████████████████████████████| 102kB 4.2kB/s
Collecting enum34 (from cryptography>=1.1->paramiko)
  Downloading enum34-1.1.6-py2-none-any.whl
Collecting cffi>=1.7 (from cryptography>=1.1->paramiko)
  Downloading cffi-1.10.0-cp27-cp27mu-manylinux1_x86_64.whl (392kB)
    100% |████████████████████████████████| 399kB 4.2kB/s
Collecting pycparser (from cffi>=1.7->cryptography>=1.1->paramiko)
  Downloading pycparser-2.18.tar.gz (245kB)
    100% |████████████████████████████████| 256kB 6.0kB/s
Installing collected packages: idna, six, ipaddress, asn1crypto, enum34, pycparser, cffi, cryptography, bcrypt, pyasn1, pynacl, paramiko
  Running setup.py install for pycparser ... done
Successfully installed asn1crypto-0.22.0 bcrypt-3.1.3 cffi-1.10.0 cryptography-2.0.3 enum34-1.1.6 idna-2.6 ipaddress-1.0.18 paramiko-2.2.1 pyasn1-0.3.2 pycparser-2.18 pynacl-1.1.2 six-1.10.0
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

# pip install paramiko

Collecting paramiko

Downloading paramiko-2.2.1-py2.py3-none-any.whl (176kB)

100% |████████████████████████████████| 184kB 10kB/s

Collecting cryptography>=1.1 (from paramiko)

Downloading cryptography-2.0.3-cp27-cp27mu-manylinux1_x86_64.whl (2.2MB)

100% |████████████████████████████████| 2.2MB 5.1kB/s

Collecting bcrypt>=3.1.3 (from paramiko)

Downloading bcrypt-3.1.3-cp27-cp27mu-manylinux1_x86_64.whl (57kB)

100% |████████████████████████████████| 61kB 3.1kB/s

Collecting pyasn1>=0.1.7 (from paramiko)

^@ Downloading pyasn1-0.3.2-py2.py3-none-any.whl (63kB)

100% |████████████████████████████████| 71kB 1.3kB/s

Collecting pynacl>=1.0.1 (from paramiko)

Downloading PyNaCl-1.1.2-cp27-cp27mu-manylinux1_x86_64.whl (539kB)

100% |████████████████████████████████| 542kB 4.4kB/s

Collecting idna>=2.1 (from cryptography>=1.1->paramiko)

Downloading idna-2.6-py2.py3-none-any.whl (56kB)

100% |████████████████████████████████| 61kB 4.5kB/s

Collecting six>=1.4.1 (from cryptography>=1.1->paramiko)

Downloading six-1.10.0-py2.py3-none-any.whl

Collecting ipaddress (from cryptography>=1.1->paramiko)

Downloading ipaddress-1.0.18-py2-none-any.whl

Collecting asn1crypto>=0.21.0 (from cryptography>=1.1->paramiko)

Downloading asn1crypto-0.22.0-py2.py3-none-any.whl (97kB)

100% |████████████████████████████████| 102kB 4.2kB/s

Collecting enum34 (from cryptography>=1.1->paramiko)

Downloading enum34-1.1.6-py2-none-any.whl

Collecting cffi>=1.7 (from cryptography>=1.1->paramiko)

Downloading cffi-1.10.0-cp27-cp27mu-manylinux1_x86_64.whl (392kB)

100% |████████████████████████████████| 399kB 4.2kB/s

Collecting pycparser (from cffi>=1.7->cryptography>=1.1->paramiko)

Downloading pycparser-2.18.tar.gz (245kB)

100% |████████████████████████████████| 256kB 6.0kB/s

Installing collected packages: idna, six, ipaddress, asn1crypto, enum34, pycparser, cffi, cryptography, bcrypt, pyasn1, pynacl, paramiko

Running setup.py install for pycparser ... done

Successfully installed asn1crypto-0.22.0 bcrypt-3.1.3 cffi-1.10.0 cryptography-2.0.3 enum34-1.1.6 idna-2.6 ipaddress-1.0.18 paramiko-2.2.1 pyasn1-0.3.2 pycparser-2.18 pynacl-1.1.2 six-1.10.0

You are using pip version 8.1.2, however version 9.0.1 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

rpmbuild 之 devel

Linux & Unix No Responses »

8月 152017

我们见过比较专业一些的软件会包含多个rpm包，如：

phpor.1.0.0.el7.x86_64

phpor-docs.1.0.0.el7.x86_64

phpor-devel.1.0.0.el7.x86_64

phpor-debuginfo.1.0.0.el7.x86_64

那么，一个spec文件如何build出来多个rpm包呢？

首先，我们以devel为例，参考： https://stackoverflow.com/questions/2913130/building-both-devel-and-normal-version-of-a-rpm-package

Name:      kmymoney
Summary:   The Personal Finances Manager for KDE.
Version:   0.8
Release:   1.%{disttag}%{distver}
License:   GPL
Packager:  %packer
Group:     Productivity/Office/Finance
Source0:   %{name}2-%version.tar.bz2
BuildRoot: %{_tmppath}/%{name}2-%{version}-%{release}-build
BuildRequires: kdebase3-devel
Prereq: /sbin/ldconfig

%description
Description goes here...

%package devel
#Requires:
Summary: KMyMoney development files
Group: Productivity/Office/Finance
Provides: kmymoney-devel

%description devel
This package contains necessary header files for KMyMoney development.

... more to go here ...

%files
... some files ...

%files devel
... the devel files ...

Name: kmymoney

Summary: The Personal Finances Manager for KDE.

Version: 0.8

Release: 1.%{disttag}%{distver}

License: GPL

Packager: %packer

Group: Productivity/Office/Finance

Source0: %{name}2-%version.tar.bz2

BuildRoot: %{_tmppath}/%{name}2-%{version}-%{release}-build

BuildRequires: kdebase3-devel

Prereq: /sbin/ldconfig

%description

Description goes here...

%package devel

#Requires:

Summary: KMyMoney development files

Group: Productivity/Office/Finance

Provides: kmymoney-devel

%description devel

This package contains necessary header files for KMyMoney development.

... more to go here ...

%files

... some files ...

%files devel

... the devel files ...

可见，devel是以子包的形式出现的

示例：

http://kmymoney2.sourceforge.net/phb/rpm-example.html

rpmbuild 安装到BUILDROOT 下面的文件和目录都要打包的rpm包中，否则会报错（rpmbuild会认为你遗忘了），如果不想打包，就提前删掉

rpmbuild 之 /usr/lib/rpm/check-buildroot

Linux & Unix No Responses »

8月 152017

rpm打包之前会执行： /usr/lib/rpm/check-buildroot，其逻辑为，检查将要打包的文件中是否包含buildroot，因为这只是build时候的临时目录，如果出现在打包文件中，就可能会出现问题，起检查方法简单粗暴：

grep -F "$RPM_BUILD_ROOT"

1	grep -F "$RPM_BUILD_ROOT"

如果包含，就会报类似如下的错误信息，然后退出

Found '/data3/rpmbuild/BUILDROOT/gdb-8.0-1.el7.centos.x86_64' in installed files; aborting

1	Found '/data3/rpmbuild/BUILDROOT/gdb-8.0-1.el7.centos.x86_64' in installed files; aborting

关于check-buildroot脚本：

test -z "$QA_SKIP_BUILD_ROOT" || exit 0

if [ -z "$RPM_BUILD_ROOT" -o "$RPM_BUILD_ROOT" = "/" ]; then
        exit 0
fi

tmp=$(mktemp ${TMPDIR:-/tmp}/cbr.XXXXXX)
trap "rm -f $tmp" EXIT

find "$RPM_BUILD_ROOT" \! \( \
    -name '*.pyo' -o -name '*.pyc' -o -name '*.elc' -o -name '.packlist' \
    \) -type f -print0 | \
    LANG=C xargs -0r grep -F "$RPM_BUILD_ROOT" >$tmp

test -s "$tmp" && {
    cat "$tmp"
    echo "Found '$RPM_BUILD_ROOT' in installed files; aborting"
    exit 1
} || :

test -z "$QA_SKIP_BUILD_ROOT" || exit 0

if [ -z "$RPM_BUILD_ROOT" -o "$RPM_BUILD_ROOT" = "/" ]; then

exit 0

tmp=$(mktemp ${TMPDIR:-/tmp}/cbr.XXXXXX)

trap "rm -f $tmp" EXIT

find "$RPM_BUILD_ROOT" \! \( \

-name '*.pyo' -o -name '*.pyc' -o -name '*.elc' -o -name '.packlist' \

\) -type f -print0 | \

LANG=C xargs -0r grep -F "$RPM_BUILD_ROOT" >$tmp

test -s "$tmp" && {

cat "$tmp"

echo "Found '$RPM_BUILD_ROOT' in installed files; aborting"

exit 1

} || :

其一：

通过mktemp来创建临时文件，根本不需要自己制造随机文件名

其二：

trap "rm -f $tmp" EXIT

1	trap "rm -f $tmp" EXIT

保证在脚本退出后删除临时文件

其三：

test -s "$tmp" &amp;&amp; { exit 1; } || :

1	test -s "$tmp" && { exit 1; } \|\| :

注意，这里 ” || : ” 的用法，保证test为假时脚本退出码为零

xfs bug

Linux & Unix No Responses »

8月 152017

[二 8月 15 11:24:30 2017] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[二 8月 15 11:24:30 2017] java            D ffff88000b91f7a0     0 18298   3997 0x00000180
[二 8月 15 11:24:30 2017]  ffff880e4b3dfad8 0000000000000082 ffff88165bd7f300 ffff880e4b3dffd8
[二 8月 15 11:24:30 2017]  ffff880e4b3dffd8 ffff880e4b3dffd8 ffff88165bd7f300 ffff88165bd7f300
[二 8月 15 11:24:30 2017]  ffff88000b91f790 ffff88000b91f798 ffffffff00000000 ffff88000b91f7a0
[二 8月 15 11:24:30 2017] Call Trace:
[二 8月 15 11:24:30 2017]  [<ffffffff8163ae49>] schedule+0x29/0x70
[二 8月 15 11:24:30 2017]  [<ffffffff8163c605>] rwsem_down_write_failed+0x115/0x220
[二 8月 15 11:24:30 2017]  [<ffffffffa03197a8>] ? xlog_grant_head_check+0x58/0x110 [xfs]
[二 8月 15 11:24:30 2017]  [<ffffffffa030942e>] ? xfs_vn_update_time+0x6e/0x190 [xfs]
[二 8月 15 11:24:30 2017]  [<ffffffff81301983>] call_rwsem_down_write_failed+0x13/0x20
[二 8月 15 11:24:30 2017]  [<ffffffff8163a09d>] ? down_write+0x2d/0x30
[二 8月 15 11:24:30 2017]  [<ffffffffa030b9a1>] xfs_ilock+0xc1/0x120 [xfs]
[二 8月 15 11:24:30 2017]  [<ffffffffa030942e>] xfs_vn_update_time+0x6e/0x190 [xfs]
[二 8月 15 11:24:30 2017]  [<ffffffff811f9a85>] update_time+0x25/0xd0
[二 8月 15 11:24:30 2017]  [<ffffffff811f9d30>] file_update_time+0xa0/0xf0
[二 8月 15 11:24:30 2017]  [<ffffffffa030092d>] xfs_file_aio_write_checks+0x11d/0x180 [xfs]
[二 8月 15 11:24:30 2017]  [<ffffffffa0300a23>] xfs_file_buffered_aio_write+0x93/0x260 [xfs]
[二 8月 15 11:24:30 2017]  [<ffffffff811b85a5>] ? mpol_misplaced+0x195/0x1f0
[二 8月 15 11:24:30 2017]  [<ffffffffa0300cc0>] xfs_file_aio_write+0xd0/0x150 [xfs]
[二 8月 15 11:24:30 2017]  [<ffffffff811dde5d>] do_sync_write+0x8d/0xd0
[二 8月 15 11:24:30 2017]  [<ffffffff811de67d>] vfs_write+0xbd/0x1e0
[二 8月 15 11:24:30 2017]  [<ffffffff8110b684>] ? __audit_syscall_entry+0xb4/0x110
[二 8月 15 11:24:30 2017]  [<ffffffff811df11f>] SyS_write+0x7f/0xe0
[二 8月 15 11:24:30 2017]  [<ffffffff816460d2>] tracesys+0xdd/0xe2

[二 8月 15 11:24:30 2017] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

[二 8月 15 11:24:30 2017] java D ffff88000b91f7a0 0 18298 3997 0x00000180

[二 8月 15 11:24:30 2017] ffff880e4b3dfad8 0000000000000082 ffff88165bd7f300 ffff880e4b3dffd8

[二 8月 15 11:24:30 2017] ffff880e4b3dffd8 ffff880e4b3dffd8 ffff88165bd7f300 ffff88165bd7f300

[二 8月 15 11:24:30 2017] ffff88000b91f790 ffff88000b91f798 ffffffff00000000 ffff88000b91f7a0

[二 8月 15 11:24:30 2017] Call Trace:

[二 8月 15 11:24:30 2017] [<ffffffff8163ae49>] schedule+0x29/0x70

[二 8月 15 11:24:30 2017] [<ffffffff8163c605>] rwsem_down_write_failed+0x115/0x220

[二 8月 15 11:24:30 2017] [<ffffffffa03197a8>] ? xlog_grant_head_check+0x58/0x110 [xfs]

[二 8月 15 11:24:30 2017] [<ffffffffa030942e>] ? xfs_vn_update_time+0x6e/0x190 [xfs]

[二 8月 15 11:24:30 2017] [<ffffffff81301983>] call_rwsem_down_write_failed+0x13/0x20

[二 8月 15 11:24:30 2017] [<ffffffff8163a09d>] ? down_write+0x2d/0x30

[二 8月 15 11:24:30 2017] [<ffffffffa030b9a1>] xfs_ilock+0xc1/0x120 [xfs]

[二 8月 15 11:24:30 2017] [<ffffffffa030942e>] xfs_vn_update_time+0x6e/0x190 [xfs]

[二 8月 15 11:24:30 2017] [<ffffffff811f9a85>] update_time+0x25/0xd0

[二 8月 15 11:24:30 2017] [<ffffffff811f9d30>] file_update_time+0xa0/0xf0

[二 8月 15 11:24:30 2017] [<ffffffffa030092d>] xfs_file_aio_write_checks+0x11d/0x180 [xfs]

[二 8月 15 11:24:30 2017] [<ffffffffa0300a23>] xfs_file_buffered_aio_write+0x93/0x260 [xfs]

[二 8月 15 11:24:30 2017] [<ffffffff811b85a5>] ? mpol_misplaced+0x195/0x1f0

[二 8月 15 11:24:30 2017] [<ffffffffa0300cc0>] xfs_file_aio_write+0xd0/0x150 [xfs]

[二 8月 15 11:24:30 2017] [<ffffffff811dde5d>] do_sync_write+0x8d/0xd0

[二 8月 15 11:24:30 2017] [<ffffffff811de67d>] vfs_write+0xbd/0x1e0

[二 8月 15 11:24:30 2017] [<ffffffff8110b684>] ? __audit_syscall_entry+0xb4/0x110

[二 8月 15 11:24:30 2017] [<ffffffff811df11f>] SyS_write+0x7f/0xe0

[二 8月 15 11:24:30 2017] [<ffffffff816460d2>] tracesys+0xdd/0xe2

windows 路由之metric

Windows No Responses »

8月 142017

如下图来看, 虽然172.16.22.29存在一条更精确的路由，但是该精确路由的metric比默认路由的metric更大一些，最终还是选择了默认路由，看来metric的影响力还是很大的

添加路由是指定metric参数吧，如下：

明明指定的metric为9，结果却是19（默认网关的metric+9）

可以通过网络设置来修改接口的跃点数（metric），最小值为10（设置为更小的值也没有用）

4294967295 is just (unsigned long) -1. -1 means that loginuid was not set. This is normal behavior for processes that were not spawned by any login process (e.g. for daemons). loginuid is -1 by default; pam_loginuid module changes it to your user id whenever you login (in a tty/in DM/via ssh), and this value is preserved by child processes.

可惜的是proc.txt并未对此文件有过说明

用途：

追踪进程发起者

Spring Boot Reference Guide

spring, 默认分类 No Responses »

8月 092017

http://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#boot-features-command-line-runner

http://docs.spring.io/spring-session/docs/current/reference/html5/

http://docs.spring.io/spring/docs/4.2.9.RELEASE/spring-framework-reference/htmlsingle/

https://spring.io/docs/reference

http://www.jianshu.com/p/3942cce05f71

http://www.mkyong.com/tutorials/spring-boot-tutorials/

redis 之 appendonly.aof

默认分类 No Responses »

8月 082017

redis.conf 中默认配置的dir为 “./” ,即为当前目录，一些人会想当然认为是redis.conf 所在目录，而appendonly.aof 默认写在该dir下；其实，”./” 为进程的cwd
redis的config set dir命令可以修改redis进程的cwd
redis的appendonly.aof 的位置会参考在config appendonly yes 被执行时的dir的值，在config appendonly yes后修改dir的值不会立即生效

遇到过的问题：

程序一直在tail -f 的方式同步appendonly.aof ，在下午18:16的时候，appendonly.aof 变到了/tmp/appendonly.aof (原本下意识地认为/tmp/appendonly.aof 只是18:16之后的数据，行文至此，才意识到，/tmp/appendonly.aof中不是18:16后的数据，而是全部数据）；目前确认进程没有被重启过，目前能够想到的能够完成这个切换的操作为：

CONFIG SET appendonly no config set dir /tmp CONFIG SET appendonly yes

1
2
3
4
5

CONFIG SET appendonly no

config set dir /tmp

CONFIG SET appendonly yes

但是似乎没有谁会去执行这么复杂的命令

ssh on proxy

代理 No Responses »

8月 052017

缘起：

vpn逐渐不好使了
虽然曾经可以通过ssh来翻墙，现在ssh已经自身难保了，也需要借助其它代理才能出门

借助nc也能搞定

参考： https://blessing.studio/ssh-over-proxy/

docker & cache

docker No Responses »

8月 052017

缘起：

dubbo-monitor容器swap使用率超过 20% ，而且在不断上升，为什么呢？

下面是容器dubbo-monitor在drop-caches前后的对比：

CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
dubbo_monitor 6.25% 1.4 GiB / 2 GiB 69.99% 0 B / 0 B 6.306 GB / 54.5 GB 0

[root@CX-DOCKER-BASE-1-10.139.105.202 ~]# echo 3 &gt; /proc/sys/vm/drop_caches

[root@CX-DOCKER-BASE-1-10.139.105.202 ~]# docker stats dubbo_monitor
CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
dubbo_monitor 3.87% 595.7 MiB / 2 GiB 29.09% 0 B / 0 B 6.373 GB / 54.5 GB 0

CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS

dubbo_monitor 6.25% 1.4 GiB / 2 GiB 69.99% 0 B / 0 B 6.306 GB / 54.5 GB 0

[root@CX-DOCKER-BASE-1-10.139.105.202 ~]# echo 3 > /proc/sys/vm/drop_caches

[root@CX-DOCKER-BASE-1-10.139.105.202 ~]# docker stats dubbo_monitor

CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS

dubbo_monitor 3.87% 595.7 MiB / 2 GiB 29.09% 0 B / 0 B 6.373 GB / 54.5 GB 0

drop-cache后，容器的swap使用率没有变化

结论：

docker stats 显示的内存使用是包含cache的
~~该容器的部分cache信息会被swap出内存~~
1. cache和swap之间存在一些不默契；cache认为内存够多，还可以cache更多信息；swap机制为，经内存充裕，只要是不常被访问的内存数据，尽量swap出来；于是乎，就形成了一个恶性循环，导致swap占用越来越大
2. ~~解决办法：定期drop一下cache~~
系统不会傻到把cache内容换出到swap中的，swap确实为进程使用的（虚拟）内存的一部分
一个允许最大使用2g内存的java进程，swap出来500m或1g不足为奇，如果给他的swap大小为2g的话，占用50%的swap都有可能

Older Entries