目 录CONTENT

文章目录

持续更新 | 常用代码片段

Wissy
2020-08-01 / 0 评论 / 4 点赞 / 216 阅读 / 0 字

一、常用命令

1.1 Shell

文本处理

  • 获取随机字符串

    openssl rand -base64 32
    
  • JSON 转 CSV

    jq -r '(.[0] | keys_unsorted) as $keys | $keys, map([.[ $keys[] ]])[] | @csv'
    
    cat c.txt|jq .data.data | jq -r '(.[0] | keys_unsorted) as $keys | $keys, map([.[ $keys[] ]])[] | @csv' >> 3.csv
    
  • awk 类似 mkString 输出

    # print 默认输出换行,不想换行可以使用pringf
    cat data.json | jq  -r -c '.data|{f1,f2,f3}'|awk 'BEGIN{printf("%s","[")} {printf("%s%s",NR>1?",":"",$0)}END{printf("%s","]")}
    

文件处理

  • 任意文件转 base64

    file_path=xx.xx
    file_mime=`file -I ${file_path}|sed 's/^.*://g;s/;.*$//g;s/[ ][ ]*//g'`
    file_base64=`base64 ${file_path}`
    js_base64="data:${file_mime};base64,${file_base64}"
    echo ${js_base64}
    
  • 批量替换文件后缀

    mv /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db{,.bak}
    

删除乱码文件

我们在工作的时候由于各种原因会产生乱码文件,产生的乱码的文件主要分为两种

  1. 不完全乱码 如:?j?​
    因为我们可以看到一个字符,可以使用 rm -rvf j​,但要注意是否有其他带有关键字的文件

  2. 完全乱码 如:????​
    像这样的数据我们就不能使用正则的方法删除文件

# 1. 查看文件inode节点数
ls -i
# 2. 修改乱码文件为a.txt
find ./ -inum 节点数 -exec mv {} a.txt \; 
# 3. 删除乱码文件
find ./ -inum 节点数 -exec -print -exec rm -rvf {} \;

例如:

wissy@wissy-PC:/tmp$ ls -i
72294967 ?                              69156075 b.txt
wissy@wissy-PC:/tmp$ find ./ -inum 72294967
./?
#删除文件
wissy@wissy-PC:/tmp$ find ./ -inum 72294967 -print -exec rm -rf {} \;
./?
#或者重命名文件
wissy@wissy-PC:/tmp$ find . -inum 69156056 -exec mv {} file.txt \;

网络安全

ACME 更新证书

  1. 下载

    curl  https://get.acme.sh | sh
    
  2. 加载环境变量

    source ~/.acme.sh/acme.sh.env
    或者
    . ~/.acme.sh/acme.sh.env
    
  3. 设置默认签证机构

    acme.sh --set-default-ca --server letsencrypt
    
  4. 使用 Aliyun 的域名认证进行签证

    export Ali_Key='xxx'
    export Ali_Secret='xxx'
    acme.sh  --issue -d '*.wissy.com.cn'  --dns dns_ali
    

    这些认证信息会保存到: ~/.acme.sh/account.conf

  5. 设置自动更新证书 cron

    "/home/xxx/.acme.sh"/acme.sh --cron --home "/home/xxx/.acme.sh"
    或者
    "/root/.acme.sh"/acme.sh --cron --home "/root/.acme.sh"
    
  6. 自动更新 acme

    acme.sh  --upgrade  --auto-upgrade
    

端口映射到本地

  • autossh

    autossh -M 5678 -fCNL 0.0.0.0:80:127.0.0.1:80 -fCNL 0.0.0.0:443:127.0.0.1:443 [email protected]
    

其他片段

GNU/Linux 一键更换国内软件源脚本

curl -sSL https://gitee.com/SuperManito/LinuxMirrors/raw/main/ChangeMirrors.sh|bash -

问题汇总

  • yum/dnf

    Error: Failed to download metadata for repo 'appstream': Cannot prepare internal mirrorlist: No URLs in mirrorlist

    wget 'http://mirror.centos.org/centos/8-stream/BaseOS/x86_64/os/Packages/centos-gpg-keys-8-3.el8.noarch.rpm'
    sudo rpm -i 'centos-gpg-keys-8-3.el8.noarch.rpm'
    
    dnf --disablerepo '*' --enablerepo=extras swap centos-linux-repos centos-stream-repos
    
    sudo dnf distro-sync
    

    FYI: StackoverFlow

  • Npm Permission denied

    Npm 执行 npm install hexo-cli --save 过程中报 Permission denied 并且 使用sudo也不能解决

    npm config set user 0
    npm config set unsafe-perm true
    

1.2 Mac

常用命令

  • 跳过验证

    xattr -d com.apple.quarantine  /Applications/xxxx.app
    
  • 禁用 Chrom 左右双指滑动手势返回上一页

    defaults write com.google.Chrome AppleEnableSwipeNavigateWithScrolls -bool false
    

二、服务配置

2.1 Nginx

Websocket

自动添加 upgrade​ 标头

    map $http_upgrade $connection_upgrade {
        default upgrade;
        ""      close;
    } 

​location​ 配置

http {
    server { 
      location / {
        ...
        proxy_http_version 1.1;
        proxy_set_header   Upgrade $http_upgrade;
        proxy_set_header   Connection "upgrade";
      }
    }
}

Vue​ 单页面配置

http {
    server { 
      #解决Router(mode: 'history')模式下,刷新路由地址不能找到页面的问题
      location / {
        root   /apps/budd/client;
        index  index.html index.htm;
        try_files $uri $uri/ @router; # 配置使用路由
      }
      # 路由配置信息
      location @router {
        rewrite ^.*$ /index.html last;
      }
      ...
    }
}

Gzip​ 配置

  1. ​Gzip_Min_Length​: 字面意思,Gzip 压缩的最低大小限制 e.g 1k​ 1m​

  2. ​Gzip_Comp_Level​: Gzip 压缩等级,1~9 级,其中推荐 1 级即可,等级再高效果就不佳了

  3. ​Gzip_Vary​: 是否发送 vary​ header

http {
    ...
    gzip on;
    gzip_min_length 1k;
    gzip_buffers   4 16k;
    gzip_http_version 1.1;
    gzip_comp_level 4;
    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
    gzip_vary on;
    gzip_proxied any;
    gzip_disable "MSIE [1-6].";
}
http {
    ...
    gzip on;
    gzip_min_length  1k;
    gzip_comp_level 1;
    gzip_types text/plain application/x-javascript text/css application/xml text/javascript
    application/x-httpd-php application/javascript application/json;
    gzip_proxied   expired no-cache no-store private auth;
    gzip_vary on;
}

log_format​

​log_json​

    log_format log_json '{"@timestamp": "$time_local", '
                        '"remote_addr": "$remote_addr", '
                        '"referer": "$http_referer", '
                        '"request": "$request", '
                        '"status": $status, '
                        '"bytes": $body_bytes_sent, '
                        '"agent": "$http_user_agent", '
                        '"upstream_addr": "$upstream_addr",'
                        '"upstream_status": "$upstream_status",'
                        '"up_resp_time": "$upstream_response_time",'
                        '"request_time": "$request_time"'
                        ' }';

2.2 dnsmasq

使用 DNS 屏蔽 AD

## Downloads an extensive list of known ad servers
## saves it at /etc/host.ads, and schedules weekly updates

# URL of ad server list
adserverlist="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext"

# Ensure that there is a downloaded list available, in case the router recently booted
wget -q -O /etc/hosts.ads "$adserverlist"; service dnsmasq restart

# Schedule automatic weekly updates
echo "@weekly wget -q -O /etc/hosts.ads '$adserverlist'; service dnsmasq restart #WeeklyAdServerListUpdate#" >> /var/spool/cron/crontabs/root
addn-hosts=/etc/hosts.ads

FYI:Blocking ads using dnsmasq with an additional hosts file

AdGurd Hosts

curl https://raw.githubusercontent.com/AdguardTeam/AdguardFilters/master/ChineseFilter/sections/adservers.txt|grep '^||'|awk -F '^' '{print $1}'|sed 's/||/address=\//g'|sed 's/$/\/127.0.0.1/g' > /etc/dnsmasq.d/adhosts

crontab

@weekly wget -q -O /etc/hosts.ads 'http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext';curl https://raw.githubusercontent.com/AdguardTeam/AdguardFilters/master/ChineseFilter/sections/adservers.txt|grep '^||'|awk -F '^' '{print $1}'|sed 's/||/address=\//g'|sed 's/$/\/127.0.0.1/g' > /etc/dnsmasq.d/adhosts; service dnsmasq restart #WeeklyAdServerListUpdate#

/etc/dnsmasq.conf


# Include all files in /etc/dnsmasq.d except RPM backup files
conf-dir=/etc/dnsmasq.d,.rpmnew,.rpmsave,.rpmorig

server=/cn/114.114.114.114
server=/taobao.com/114.114.114.114
server=/taobaocdn.com/114.114.114.114
server=/google.com/223.5.5.5

# AD
address=/ad.youku.com/127.0.0.1
address=/ad.iqiyi.com/127.0.0.1
#Bad
address=/freehao123.com/123.123.123.123
addn-hosts=/etc/hosts.ads

address=/wissy.com.cn/103.222.190.158
address=/raw.githubusercontent.com/151.101.56.133

#log all dns queries
log-queries
log-facility=/var/log/dnsmasq/dnsmasq.log
domain-needed
bogus-priv
#no-resolv
#缓存的数量
cache-size=102400
#如果查询的域名没ttl,则使用此设置为缓存ttl时间
neg-ttl=600
#指定允许返回给客户端最大ttl时间
max-ttl=600
#dnsmasq服务器缓存最大时间设定
max-cache-ttl=3600
#dnsmasq服务器缓存最小时间设定

三、其他

国内加速

4

评论区