一、常用命令
1.1 Shell
文本处理
-
获取随机字符串
openssl rand -base64 32 -
JSON 转 CSV
jq -r '(.[0] | keys_unsorted) as $keys | $keys, map([.[ $keys[] ]])[] | @csv' cat c.txt|jq .data.data | jq -r '(.[0] | keys_unsorted) as $keys | $keys, map([.[ $keys[] ]])[] | @csv' >> 3.csv -
awk 类似 mkString 输出
# print 默认输出换行，不想换行可以使用pringf cat data.json | jq -r -c '.data|{f1,f2,f3}'|awk 'BEGIN{printf("%s","[")} {printf("%s%s",NR>1?",":"",$0)}END{printf("%s","]")}
文件处理
-
任意文件转 base64
file_path=xx.xx file_mime=`file -I ${file_path}|sed 's/^.*://g;s/;.*$//g;s/[ ][ ]*//g'` file_base64=`base64 ${file_path}` js_base64="data:${file_mime};base64,${file_base64}" echo ${js_base64} -
批量替换文件后缀
mv /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db{,.bak} -
删除乱码文件
我们在工作的时候由于各种原因会产生乱码文件，产生的乱码的文件主要分为两种
- 不完全乱码 如：
?j?
因为我们可以看到一个字符，可以使用rm -rvf *j*，但要注意是否有其他带有关键字的文件 - 完全乱码 如：
????
像这样的数据我们就不能使用正则的方法删除文件
# 1. 查看文件inode节点数
ls -i
# 2. 修改乱码文件为a.txt
find ./ -inum 节点数 -exec mv {} a.txt \;
# 3. 删除乱码文件
find ./ -inum 节点数 -exec -print -exec rm -rvf {} \;
例如：
wissy@wissy-PC:/tmp$ ls -i
72294967 ? 69156075 b.txt
wissy@wissy-PC:/tmp$ find ./ -inum 72294967
./?
#删除文件
wissy@wissy-PC:/tmp$ find ./ -inum 72294967 -print -exec rm -rf {} \;
./?
#或者重命名文件
wissy@wissy-PC:/tmp$ find . -inum 69156056 -exec mv {} file.txt \;
网络安全
ACME 更新证书
-
下载
curl https://get.acme.sh | sh -
加载环境变量
source ~/.acme.sh/acme.sh.env 或者 . ~/.acme.sh/acme.sh.env -
设置默认签证机构
acme.sh --set-default-ca --server letsencrypt -
使用 Aliyun 的域名认证进行签证
export Ali_Key='xxx' export Ali_Secret='xxx' acme.sh --issue -d '*.wissy.com.cn' --dns dns_ali这些认证信息会保存到: ~/.acme.sh/account.conf 中
-
设置自动更新证书 cron
"/home/xxx/.acme.sh"/acme.sh --cron --home "/home/xxx/.acme.sh" 或者 "/root/.acme.sh"/acme.sh --cron --home "/root/.acme.sh" -
自动更新 acme
acme.sh --upgrade --auto-upgrade
端口映射到本地
-
autossh
autossh -M 5678 -fCNL 0.0.0.0:80:127.0.0.1:80 -fCNL 0.0.0.0:443:127.0.0.1:443 [email protected]
其他片段
GNU/Linux 一键更换国内软件源脚本
curl -sSL https://gitee.com/SuperManito/LinuxMirrors/raw/main/ChangeMirrors.sh|bash -
问题汇总
-
yum/dnf
Error: Failed to download metadata for repo 'appstream': Cannot prepare internal mirrorlist: No URLs in mirrorlist
wget 'http://mirror.centos.org/centos/8-stream/BaseOS/x86_64/os/Packages/centos-gpg-keys-8-3.el8.noarch.rpm' sudo rpm -i 'centos-gpg-keys-8-3.el8.noarch.rpm' dnf --disablerepo '*' --enablerepo=extras swap centos-linux-repos centos-stream-repos sudo dnf distro-syncFYI: StackoverFlow
-
Npm Permission denied
Npm 执行 npm install hexo-cli --save 过程中报 Permission denied 并且 使用sudo也不能解决
npm config set user 0 npm config set unsafe-perm true
1.2 Mac
系统配置命令
-
跳过验证
xattr -d com.apple.quarantine /Applications/xxxx.app -
禁用 Chrom 左右双指滑动手势返回上一页
defaults write com.google.Chrome AppleEnableSwipeNavigateWithScrolls -bool false -
禁止生成 .DS_Store 文件（需要重启）
defaults write com.apple.desktopservices DSDontWriteNetworkStores true defaults read com.apple.desktopservices DSDontWriteNetworkStores查看是否设置成功
音视频软件
-
使用 FFmpeg 合并音频（顺序合并）
ffmpeg \ -i "Input 01.mp3" \ -i "Input 02.mp3" \ -i "Input 03.mp3" \ -i "Input 04.mp3" \ -i "Input 05.mp3" \ -filter_complex "[0:0][1:0][2:0][3:0][4:0]concat=n=5:v=0:a=1[out]" -map "[out]" \ -f mp3 Output.mp3 -
使用 FFmpeg 合并音频（音轨叠加）
ffmpeg \ -i "Input 01.mp3" \ -i "Input 02.mp3" \ -filter_complex amix=inputs=2:duration=first:dropout_transition=2 -f mp3 Output.mp3
1.3 Linux
替换软件源
-
Debian
sed -i 's#https\?://deb.debian.org/debian#https://mirrors.tuna.tsinghua.edu.cn/debian#g' /etc/apt/sources.list.d/debian.sources
二、服务配置
2.1 Nginx
Websocket
自动添加 upgrade 标头
map $http_upgrade $connection_upgrade {
default upgrade;
"" close;
}
location 配置
http {
server {
location / {
...
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
}
Vue 单页面配置
http {
server {
#解决Router(mode: 'history')模式下，刷新路由地址不能找到页面的问题
location / {
root /apps/budd/client;
index index.html index.htm;
try_files $uri $uri/ @router; # 配置使用路由
}
# 路由配置信息
location @router {
rewrite ^.*$ /index.html last;
}
...
}
}
Gzip 配置
-
Gzip_Min_Length: 字面意思，Gzip 压缩的最低大小限制 e.g1k1m -
Gzip_Comp_Level: Gzip 压缩等级，1~9 级，其中推荐 1 级即可，等级再高效果就不佳了 -
Gzip_Vary: 是否发送vary header
http {
...
gzip on;
gzip_min_length 1k;
gzip_buffers 4 16k;
gzip_http_version 1.1;
gzip_comp_level 4;
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
gzip_vary on;
gzip_proxied any;
gzip_disable "MSIE [1-6].";
}
http {
...
gzip on;
gzip_min_length 1k;
gzip_comp_level 1;
gzip_types text/plain application/x-javascript text/css application/xml text/javascript
application/x-httpd-php application/javascript application/json;
gzip_proxied expired no-cache no-store private auth;
gzip_vary on;
}
log_format
log_json
log_format log_json '{"@timestamp": "$time_local", '
'"remote_addr": "$remote_addr", '
'"referer": "$http_referer", '
'"request": "$request", '
'"status": $status, '
'"bytes": $body_bytes_sent, '
'"agent": "$http_user_agent", '
'"upstream_addr": "$upstream_addr",'
'"upstream_status": "$upstream_status",'
'"up_resp_time": "$upstream_response_time",'
'"request_time": "$request_time"'
' }';
2.2 dnsmasq
使用 DNS 屏蔽 AD
## Downloads an extensive list of known ad servers
## saves it at /etc/host.ads, and schedules weekly updates
# URL of ad server list
adserverlist="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext"
# Ensure that there is a downloaded list available, in case the router recently booted
wget -q -O /etc/hosts.ads "$adserverlist"; service dnsmasq restart
# Schedule automatic weekly updates
echo "@weekly wget -q -O /etc/hosts.ads '$adserverlist'; service dnsmasq restart #WeeklyAdServerListUpdate#" >> /var/spool/cron/crontabs/root
addn-hosts=/etc/hosts.ads
FYI:Blocking ads using dnsmasq with an additional hosts file
AdGurd Hosts
curl https://raw.githubusercontent.com/AdguardTeam/AdguardFilters/master/ChineseFilter/sections/adservers.txt|grep '^||'|awk -F '^' '{print $1}'|sed 's/||/address=\//g'|sed 's/$/\/127.0.0.1/g' > /etc/dnsmasq.d/adhosts
crontab
@weekly wget -q -O /etc/hosts.ads 'http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext';curl https://raw.githubusercontent.com/AdguardTeam/AdguardFilters/master/ChineseFilter/sections/adservers.txt|grep '^||'|awk -F '^' '{print $1}'|sed 's/||/address=\//g'|sed 's/$/\/127.0.0.1/g' > /etc/dnsmasq.d/adhosts; service dnsmasq restart #WeeklyAdServerListUpdate#
/etc/dnsmasq.conf
# Include all files in /etc/dnsmasq.d except RPM backup files
conf-dir=/etc/dnsmasq.d,.rpmnew,.rpmsave,.rpmorig
server=/cn/114.114.114.114
server=/taobao.com/114.114.114.114
server=/taobaocdn.com/114.114.114.114
server=/google.com/223.5.5.5
# AD
address=/ad.youku.com/127.0.0.1
address=/ad.iqiyi.com/127.0.0.1
#Bad
address=/freehao123.com/123.123.123.123
addn-hosts=/etc/hosts.ads
address=/wissy.com.cn/103.222.190.158
address=/raw.githubusercontent.com/151.101.56.133
#log all dns queries
log-queries
log-facility=/var/log/dnsmasq/dnsmasq.log
domain-needed
bogus-priv
#no-resolv
#缓存的数量
cache-size=102400
#如果查询的域名没ttl，则使用此设置为缓存ttl时间
neg-ttl=600
#指定允许返回给客户端最大ttl时间
max-ttl=600
#dnsmasq服务器缓存最大时间设定
max-cache-ttl=3600
#dnsmasq服务器缓存最小时间设定
2.3 Kubernetes
K3S 镜像加速
修改 /etc/rancher/k3s/registries.yaml
mirrors:
"docker.io":
endpoint:
#- "https://6curw25x.mirror.aliyuncs.com" # Aliyun 不全且不更新呢实时
- "https://docker.proxy.xxxxxx" #自定义
"k8s.gcr.io":
endpoint:
- "https://lank8s.cn"
- "https://k8s.lank8s.cn"
"gcr.io":
endpoint:
- "https://ghcr.nju.edu.cn"
"ghcr.io":
endpoint:
- "https://ghcr.nju.edu.cn"
"registry.k8s.io":
endpoint:
- "https://registry.lank8s.cn"
configs:
"docker.io":
auth:
username: username
password: password
"registry.cn-hangzhou.aliyuncs.com":
auth:
username: username
password: password
"goodrain.me":
tls:
insecure_skip_verify: true
三、其他
国内加速
-
Google Fonts 加速代理
- https://ajax.googleapis.com =>https://ajax.googleapis.cnpmjs.org
- https://fonts.googleapis.com =>https://fonts.googleapis.cnpmjs.org
- https://fonts.gstatic.com =>https://fonts.gstatic.cnpmjs.org
- https://themes.googleusercontent.com =>https://themes.googleusercontent.cnpmjs.org
如 https://fonts.googleapis.com/css?family=Lato:400,700|Source+Code+Pro:400,500
=>
https://fonts.googleapis.cnpmjs.org/css?family=Lato:400,700|Source+Code+Pro:400,500
-
Github 加速
-
NodeJS 加速
.yarnrc 配置文件registry "https://registry.npmmirror.com" sass_binary_site "https://npmmirror.com/mirrors/node-sass/" phantomjs_cdnurl "http://cnpmjs.org/downloads" electron_mirror "https://npmmirror.com/mirrors/electron/" sqlite3_binary_host_mirror "https://foxgis.oss-cn-shanghai.aliyuncs.com/" profiler_binary_host_mirror "https://npmmirror.com/mirrors/node-inspector/" chromedriver_cdnurl "https://npmmirror.com/mirrors/chromedriver"
评论区