需求说明
- 提供路由服务，并透明代理
- 提供 VPN 服务
- 提供影视库功能
- 提供照片备份功能
- 迅雷下载电影
心路历程
最开始纠结是 PVE 还是直接跑 Linux ，但是想了一下都 all in one 了干脆直接 Linux 省的套娃。
最后整体思路：
- 创建 gateway 的 netns 作为软路由，里面跑 dnmasq 提供 dhcp 。配合 iptables 和 ipset 实现全局流量走 clash 透明代理。源码
- 使用自己实现的 wg 工具来配置 wireguard 提供 VPN 服务。源码
- 影音库 Jellyfin 运行在 lxc 容器中，透传核显提供 GPU 解码。
- immich 提供照片备份功能，通过 nerdctl 跑 immich 的 docker compose 服务。
- 按照迅雷网盘版提供磁力下载功能。源码
- nginx 提供 https 反向代理。
- 服务管理，基本通过 systemd
硬件说明
小黄鱼淘了一台 4 网口的 AMD 主板,16G 内存，CPU 型号忘了，看了一下是 es 的，无所谓了
root@shawn-aio:~# grep name /proc/cpuinfo | cut -f2 -d: | uniq -c 4 AMD Eng Sample root@shawn-aio:~# free -h total used free shared buff/cache available Mem: 14Gi 1.9Gi 9.8Gi 200Mi 3.4Gi 12Gi Swap: 4.0Gi 0B 4.0Gi 网络结构
eno1 做 wan 口，通过 iptables nat 表做 masquerade 给内部提供外网访问能力。wan 口只开放 vpn 端口。
iptables -t nat -A POSTROUTING -o eno1 -j MASQUERADE root@shawn-aio:~# iptables -S INPUT -P INPUT DROP -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -i br-mgmt -j ACCEPT -A INPUT -i eno1 -p udp -m udp --dport 51820 -j ACCEPT -A INPUT -i wg0 -j ACCEPT -A INPUT -i lxcbr0 -j ACCEPT -A INPUT -i br-0ad574e582c7 -j ACCEPT -A INPUT -i nerdctl0 -j ACCEPT 创建 br-mgmt ，接入 enp3s0 ，enp4s0 ，enp5s0 做 lan 口，同时通过 veth 将 netns gateway 接到 br-mgmt ，并在 netns 中提供 dhcp ，下发默认路由和 dns 地址。
root@shawn-aio:~# ip a show br-mgmt 6: br-mgmt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether ea:a4:3d:6c:ce:d0 brd ff:ff:ff:ff:ff:ff inet 172.20.0.254/24 brd 172.20.0.255 scope global br-mgmt valid_lft forever preferred_lft forever inet6 fe80::e8a4:3dff:fe6c:ced0/64 scope link proto kernel_ll valid_lft forever preferred_lft forever root@shawn-aio:~# ip netns exec gateway ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host proto kernel_lo valid_lft forever preferred_lft forever 12: gTb@if13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 52:ef:a4:d0:0c:e9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 172.20.0.1/24 scope global gTb valid_lft forever preferred_lft forever inet6 fe80::50ef:a4ff:fed0:ce9/64 scope link proto kernel_ll valid_lft forever preferred_lft forever root@shawn-aio:~# ip l show master br-mgmt 3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br-mgmt state UP mode DEFAULT group default qlen 1000 link/ether 20:76:93:5b:14:4f brd ff:ff:ff:ff:ff:ff 4: enp4s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel master br-mgmt state DOWN mode DEFAULT group default qlen 1000 link/ether 20:76:93:5b:14:50 brd ff:ff:ff:ff:ff:ff 5: enp5s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel master br-mgmt state DOWN mode DEFAULT group default qlen 1000 link/ether 20:76:93:5b:14:51 brd ff:ff:ff:ff:ff:ff 13: bTg@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-mgmt state UP mode DEFAULT group default qlen 1000 link/ether aa:36:50:e4:db:81 brd ff:ff:ff:ff:ff:ff link-netns gateway 15: veth18XBnR@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-mgmt state UP mode DEFAULT group default qlen1000 link/ether fe:e5:47:3f:90:3b brd ff:ff:ff:ff:ff:ff link-netnsid 1 16: vethdxnWc4@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-mgmt state UP mode DEFAULT group default qlen 1000 link/ether fe:c1:4c:94:06:ba brd ff:ff:ff:ff:ff:ff link-netnsid 2 服务详情
nginx 反向代理
root@shawn-aio:~# systemctl status nginx ● nginx.service - A high performance web server and a reverse proxy server Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; preset: enabled) Active: active (running) since Fri 2025-05-09 08:38:57 CST; 1 week 0 days ago Invocation: a6039cd62e984eb0bd649511a52f5060 Docs: man:nginx(8) Main PID: 6945 (nginx) Tasks: 5 (limit: 17733) Memory: 8.6M (peak: 13.6M) CPU: 1.111s CGroup: /system.slice/nginx.service ├─6945 "nginx: master process /usr/sbin/nginx -g daemon on; master_process on;" ├─6946 "nginx: worker process" ├─6947 "nginx: worker process" ├─6948 "nginx: worker process" └─6949 "nginx: worker process" May 09 08:38:57 shawn-aio systemd[1]: Starting nginx.service - A high performance web server and a reverse proxy server... May 09 08:38:57 shawn-aio systemd[1]: Started nginx.service - A high performance web server and a reverse proxy server. 软路由和透明代理
自己写了个脚本，创建了一个 netns 后，在 netns 中跑 dnsmasq 提供 dhcp 服务。里面跑 clash ，并设置 iptables 规则来实现透明代理。
root@shawn-aio:~# systemctl status gateway ● gateway.service - Gateway service enable tproxy Loaded: loaded (/etc/systemd/system/gateway.service; enabled; preset: enabled) Active: active (running) since Fri 2025-05-09 08:38:57 CST; 1 week 0 days ago Invocation: 76718511daa5473bad8931fedd4d54c9 Main PID: 6873 (gateway.sh) Tasks: 3 (limit: 17733) Memory: 5.1M (peak: 6.8M) CPU: 1d 22h 47min 56.956s CGroup: /system.slice/gateway.service ├─6873 /usr/bin/bash /opt/gateway/gateway.sh start ├─6973 /usr/sbin/dnsmasq -C /opt/gateway/dnsmasq.conf └─6974 bash /opt/gateway/tproxy.sh start May 15 08:25:45 shawn-aio dnsmasq-dhcp[6973]: DHCPREQUEST(gTb) 172.20.0.165 c8:a3:62:57:df:69 May 15 08:25:45 shawn-aio dnsmasq-dhcp[6973]: DHCPACK(gTb) 172.20.0.165 c8:a3:62:57:df:69 luchengdeMBP May 15 13:53:13 shawn-aio dnsmasq-dhcp[6973]: DHCPREQUEST(gTb) 172.20.0.165 c8:a3:62:57:df:69 May 15 13:53:13 shawn-aio dnsmasq-dhcp[6973]: DHCPACK(gTb) 172.20.0.165 c8:a3:62:57:df:69 luchengdeMBP May 15 19:10:29 shawn-aio dnsmasq-dhcp[6973]: DHCPREQUEST(gTb) 172.20.0.165 c8:a3:62:57:df:69 May 15 19:10:29 shawn-aio dnsmasq-dhcp[6973]: DHCPACK(gTb) 172.20.0.165 c8:a3:62:57:df:69 luchengdeMBP May 16 00:33:54 shawn-aio dnsmasq-dhcp[6973]: DHCPREQUEST(gTb) 172.20.0.165 c8:a3:62:57:df:69 May 16 00:33:54 shawn-aio dnsmasq-dhcp[6973]: DHCPACK(gTb) 172.20.0.165 c8:a3:62:57:df:69 luchengdeMBP May 16 06:08:01 shawn-aio dnsmasq-dhcp[6973]: DHCPREQUEST(gTb) 172.20.0.165 c8:a3:62:57:df:69 May 16 06:08:01 shawn-aio dnsmasq-dhcp[6973]: DHCPACK(gTb) 172.20.0.165 c8:a3:62:57:df:69 luchengdeMBP root@shawn-aio:~# ip netns gateway (id: 0) VPN 服务
使用场景很简单，感觉没必要用 tailscale 或者 netmaker 。同时自己不太喜欢界面，所以自己模仿 wg-easy 写了个 wg 服务来提供 VPN 服务。客户端直接鉴权是模仿 k8s apiserver 和 kubectl 通过 tls 证书鉴权。
root@shawn-aio:~# wgctl subnet list UUID Name Address Public Key e4212eda-233f-11f0-8903-2076935b144e default 10.67.0.1/24 <pub key> root@shawn-aio:~# wgctl peer list -s e4212eda-233f-11f0-8903-2076935b144e UUID User Address Public Key Enable f9615f20-233f-11f0-8903-2076935b144e lucheng 10.67.0.2/24 <pub key> true root@shawn-aio:~# wg interface: wg0 public key: <server pubkey> private key: (hidden) listening port: 51820 peer: <peer pubkey> endpoint: 10.28.66.18:36303 allowed ips: 10.67.0.2/32 latest handshake: 29 minutes, 23 seconds ago transfer: 5.14 MiB received, 35.12 MiB sent 影音功能
lxc 跑 ubuntu 在里面装 jellyfin
root@shawn-aio:~# lxc-ls -f NAME STATE AUTOSTART GROUPS IPV4 IPV6 UNPRIVILEGED jellyfin RUNNING 1 - 172.20.0.2 - false ubuntu RUNNING 1 - 172.20.0.3 - false 照片备份
root@shawn-aio:~# nerdctl ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 05a100ce77d8 quay.io/shawnlu0127/immich/immich-server:v1.131.1 "tini -- /bin/bash s…" 6 days ago Up 0.0.0.0:2283->2283/tcp immich_server 414297ba5e5f quay.io/shawnlu0127/immich/postgres:pg14-v0.2.0 "docker-entrypoint.s…" 6 days ago Up immich_postgres 6fa52ce3d9e1 quay.io/shawnlu0127/immich/redis:6.2-alpine "docker-entrypoint.s…" 6 days ago Up immich_redis 
2
Select Language