πŸ€– I Gave an AI My Whole Proxmox Box. It Still Cannot Delete Anything.

It reads every container, every listener, every route. pct destroy stops dead and asks. 14 phases, Proxmox + OPNsense.

An AI agent with root on a hypervisor can delete a container, wipe storage, rewrite a NAT rule or reboot the host. It will do any of it confidently, in one command, and tell you it went well.

Every guide out there says point the agent at your server. None of them ship the leash.

      WITHOUT A LEASH                WITH ONE
      ──────────────                 ────────
      pct destroy 220                πŸ”΄ blocked, asks first
      rewrite br0                    πŸ”΄ blocked, asks first
      systemctl stop nginx (LIVE)    πŸ”΄ blocked, asks first

      pct list · ss -lntp · curl     🟒 runs alone, instantly
      restart a DEV service          🟑 runs, reversible

That is the whole idea. A junior sysadmin who can see everything and break nothing.

:bullseye: What you end up with

  • An agent that runs every read-only diagnostic on its own β€” no approval, no waiting
  • A written permission matrix: the exact commands it may run alone, and the exact ones where it must stop and ask
  • An 11-rule contract file the agent reads before it acts
  • A fault ladder so you stop rebuilding proven bridges because one app returned a 502
  • Production still recoverable, auditable and yours

:stopwatch: Before and after

I got tired of being a copy-paste machine between my terminal and a chat window.

   BEFORE                          AFTER
   ──────                          ─────
   I run the command               agent inspects on its own
   I copy the output          ➜    agent does reversible DEV work
   I paste it to the AI            I approve anything that
   I get the next command          touches production

So I turned an existing Ubuntu workstation that already reaches my Proxmox host into a proper management host β€” in 14 phases β€” and wrote down what the agent was allowed to do before it was allowed to do anything.

The objective was never β€œgive an AI root.”

:vertical_traffic_light: Every phase below carries a colour

🟒 GREEN   read-only. Look, don't touch.
           The agent runs these on its own.

🟑 YELLOW  reversible. Management workstation or DEV only.

πŸ”΄ RED     production routing, firewall, storage, deletion.
           The agent stops. A human says yes, or it does not happen.

The build is deliberately conservative, in this order:

  • preserve existing SSH access
  • establish known-good Proxmox connectivity
  • install standard management tools
  • document the environment
  • inspect current LXC state
  • verify network and services before changing anything
  • install agent tooling only after the Node/npm environment is understood
  • never expose OPNsense, Proxmox, SSH or API credentials unnecessarily

:backpack: Not running Proxmox? Take these two anyway

Phase 9 β€” the operating rules. Eleven lines in one text file. Stack-agnostic.
Part 4 β€” the permission matrix. GREEN/YELLOW/RED per command. Works on anything you can SSH into.

πŸ—ΊοΈ Stack glossary β€” Proxmox Β· LXC Β· OPNsense

Proxmox VE β€” Debian-based hypervisor managing KVM virtual machines and LXC containers on one host. Web UI on :8006, CLI via pct (containers) and qm (VMs).
LXC β€” OS-level container sharing the host kernel. No guest kernel, no hardware emulation, so it boots in under a second and costs a fraction of a VM.
OPNsense β€” FreeBSD-based firewall/router distribution. Owns WAN rules, NAT port-forwards and the public edge. Everything reaching your LXCs from the internet passes through it.

Every address, hostname, container ID and network range below is a documentation example, not my real infrastructure.


:world_map: Phase index β€” what each one actually runs

 β”Œ PART 1 Β· πŸ’» UBUNTU MANAGEMENT WORKSTATION ────────────────────┐
 β”‚  1  🟒  hostname Β· whoami Β· ip -br addr                       β”‚
 β”‚  2  🟑  apt install β€” git curl jq tmux mtr nc dig rsync       β”‚
 β”‚  3  🟒  ssh root@<pve-ip>  β€” direct IP, no alias yet          β”‚
 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
 β”Œ PART 2 Β· πŸ—„οΈ PROXMOX HOST SHELL ───────────────────────────────┐
 β”‚  4  🟒  pveversion Β· pct list                                 β”‚
 β”‚  5  🟒  pct config <id> β€” net0, bridge, gw, mounts            β”‚
 β”‚  6  🟒  pct exec <id> -- ss -lntp   (listeners :22 :80 :3001) β”‚
 β”‚  7  🟒  ping + nc -vz <lxc-ip> 22                             β”‚
 β”‚  8  🟒  pct exec <id> -- curl -I http://127.0.0.1:3001/       β”‚
 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
 β”Œ PART 3 Β· πŸ’» BACK ON UBUNTU ───────────────────────────────────┐
 β”‚  9  🟑  /opt/AI-Agent tree + README-FIRST operating rules ⭐   β”‚
 β”‚ 10  🟑  ~/.ssh/config β€” Host alias, appended not overwritten   β”‚
 β”‚ 11  🟒  node/npm/uname baseline  β†’  STOP POINT before install β”‚
 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
 β”Œ PART 4 Β· 🧠 DESIGN β€” NO COMMANDS ─────────────────────────────┐
 β”‚ 12  🚦  GREEN/YELLOW/RED agent permission matrix              β”‚
 β”‚ 13  🧠  docs/ scripts/ + agent feedback layer                 β”‚
 β”‚ 14  πŸ”΄  Proxmox API + OPNsense API, scoped read-only identity β”‚
 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Phases 1–8 are read-only. No writes to Proxmox, OPNsense, LXC config, routing or firewall. Run all eight against a live host and its state is unchanged.

First write is Phase 9, and it targets /opt on your own workstation.

Portable to any stack: Phase 9’s operating rules and Part 4’s permission matrix. Neither depends on Proxmox.


:laptop: PART 1 β€” on your Ubuntu box

:green_circle: :laptop: Phase 1 β€” Confirm Which Machine You Are Using

🟒 GREEN β€” Identify the Ubuntu management workstation
hostname
whoami
ip -br addr
  • Purpose: Confirms which machine, user, and network interfaces are active.
  • Why: Avoids accidentally running workstation commands on the Proxmox host or vice versa.
  • Risk: GREEN β€” read-only.
  • Changes made: None.

:yellow_circle: :laptop: Phase 2 β€” Install Basic Management Utilities

🟒🟑 GREEN/YELLOW β€” Refresh Ubuntu package information
sudo apt update
  • Purpose: Downloads the latest package indexes from configured Ubuntu repositories.
  • Why: Package information should be current before installing management utilities.
  • Risk: GREEN/YELLOW β€” updates package metadata only.
  • Does not modify: Proxmox, OPNsense, LXCs, routing, or firewall configuration.
🟑 YELLOW β€” Install the management toolbox
sudo apt install -y git curl jq openssh-client python3 python3-pip tmux \
  dnsutils netcat-openbsd traceroute mtr-tiny rsync ca-certificates
  • Purpose: Installs common administration, troubleshooting, API, and automation tools.
  • Included tools:
    • git β€” source/configuration version control
    • curl β€” HTTP and API testing
    • jq β€” JSON processing
    • openssh-client β€” SSH connectivity
    • python3 β€” scripting/automation
    • tmux β€” persistent terminal sessions
    • dnsutils β€” dig, nslookup, and DNS testing
    • netcat-openbsd β€” TCP port testing with nc
    • traceroute β€” route diagnostics
    • mtr-tiny β€” continuous route/latency troubleshooting
    • rsync β€” file synchronization and backups
    • ca-certificates β€” trusted HTTPS certificate authorities
  • Risk: YELLOW β€” installs software only on the Ubuntu management workstation.
  • Does not modify: Proxmox or OPNsense.

:green_circle: :laptop: Phase 3 β€” Prove Direct SSH Access to Proxmox

Do not create aliases or automation until the direct IP connection is proven.

🟒 GREEN β€” Connect directly to the Proxmox management address
  • Purpose: Opens an SSH session from the Ubuntu workstation to the Proxmox server.
  • Why: Establishes the most basic management path before adding aliases or agent tooling.
  • Expected: A Proxmox/Debian login banner followed by a prompt similar to:
root@pve-node-1:~#
  • Risk: GREEN β€” connecting alone does not modify the server.
  • Important: Never copy the shell prompt itself as part of a command.

For example:

root@pve-node-1:~#

is a prompt, not a command.

203.0.113.10 is an example documentation address. Replace it with your real Proxmox management address on your private copy.


:file_cabinet: PART 2 β€” you are now on the Proxmox box

:green_circle: :file_cabinet: Phase 4 β€” Establish the Current Proxmox Baseline

Once connected to the Proxmox shell, verify the node and LXC inventory.

🟒 GREEN β€” Show Proxmox version and running containers
hostname
pveversion
pct list
  • Purpose: Identifies the Proxmox node, installed version, running kernel, and LXC inventory.
  • Why: Live system state should take priority over old notes when documentation disagrees.
  • Risk: GREEN β€” read-only.
  • Changes made: None.

Example sanitized output:

pve-node-1
pve-manager/9.x/...

VMID  Status   Name
201   running  git-service
210   running  web-live
220   running  web-dev
222   running  llm-gateway
230   running  web-alt

This type of check is extremely useful because documentation can become stale while the live host reflects the current truth.


:green_circle: :file_cabinet: Phase 5 β€” Inspect Relevant LXC Configuration

Before changing NAT, SSH, bridges, or routing, verify the containers themselves.

🟒 GREEN β€” Inspect DEV, gateway, and ALT container configuration
pct config 220
pct config 222
pct config 230
  • Purpose: Displays the configuration for the DEV web container, AI/API gateway, and ALT container.
  • Why: Confirms each container’s network interface, bridge, IP configuration, gateway, mount points, and other settings.
  • Risk: GREEN β€” read-only.
  • Changes made: None.

What to inspect carefully:

  • net0
  • bridge assignment
  • IPv4 address
  • gateway
  • hostname
  • startup settings
  • mount points
  • unusual firewall flags

Do not change any values during this phase.


:green_circle: :file_cabinet: Phase 6 β€” Verify Services Actually Listening Inside the LXCs

A public SSH or web failure does not automatically mean the firewall or bridge is broken.

First prove whether the destination service exists inside the container.

🟒 GREEN β€” Show listening TCP services inside the relevant containers
pct exec 220 -- ss -lntp
pct exec 222 -- ss -lntp
pct exec 230 -- ss -lntp
  • Purpose: Lists TCP services currently listening inside each container.
  • Why: Separates an internal service problem from an OPNsense NAT/firewall problem.
  • Risk: GREEN β€” read-only.
  • Changes made: None.

Important example ports:

  • TCP 22 β€” SSH
  • TCP 80 β€” HTTP
  • TCP 443 β€” HTTPS
  • TCP 3001 β€” application/API listener

If port 22 is not listening inside a container, adding an OPNsense NAT rule will not solve SSH.

If the application port is not listening inside the gateway container, troubleshooting the public reverse proxy first wastes time.


:green_circle: :file_cabinet: Phase 7 β€” Test Internal Network Reachability

Only after configuration and listeners are understood should connectivity be tested.

🟒 GREEN β€” Test private LXC reachability from Proxmox
ping -c 3 10.20.30.220
ping -c 3 10.20.30.222
ping -c 3 10.20.30.230
  • Purpose: Verifies basic IP reachability to the relevant containers.
  • Risk: GREEN β€” diagnostic traffic only.
  • Changes made: None.

The 10.20.30.0/24 network shown here is a sanitized example.

🟒 GREEN β€” Test SSH TCP ports on the private network
nc -vz 10.20.30.220 22
nc -vz 10.20.30.222 22
nc -vz 10.20.30.230 22
  • Purpose: Tests whether TCP port 22 is reachable on each container.
  • Why: Distinguishes an SSH service problem from an external firewall/NAT problem.
  • Risk: GREEN β€” connection test only.
  • Changes made: None.

Typical results:

succeeded

means the TCP service is reachable.

connection refused

normally means the host is reachable but nothing is listening on that port.

timed out

can indicate firewall, routing, or reachability problems.


:green_circle: :file_cabinet: Phase 8 β€” Verify the Application Locally Before Testing Public Routing

Application health should be proven locally before reverse-proxy troubleshooting.

🟒 GREEN β€” Test the application inside its LXC
pct exec 222 -- curl -I http://127.0.0.1:3001/
  • Purpose: Tests the application HTTP listener from inside its own container.
  • Expected: A successful HTTP response such as HTTP/1.1 200 OK.
  • Why: Proves the application itself is serving before introducing nginx, NAT, DNS, TLS, or OPNsense into the troubleshooting path.
  • Risk: GREEN β€” read-only HTTP request.
  • Changes made: None.

A useful troubleshooting rule:

APPLICATION
    ↓
PRIVATE NETWORK
    ↓
REVERSE PROXY
    ↓
OPNSENSE NAT/FIREWALL
    ↓
PUBLIC DNS
    ↓
PUBLIC HTTPS

Prove each layer in that order.


:laptop: PART 3 β€” back on Ubuntu

:yellow_circle: :laptop: Phase 9 β€” Create the Agent Workspace on Ubuntu

Return to the Ubuntu management workstation before running this phase.

🟑 YELLOW β€” Create organized management directories
sudo mkdir -p /opt/AI-Agent/{docs,scripts,logs,work,backups}
sudo chown -R "$USER":"$USER" /opt/AI-Agent
chmod 700 /opt/AI-Agent
  • Purpose: Creates one predictable workspace for agent documentation, scripts, logs, work files, and backups.
  • Risk: YELLOW β€” creates directories on the Ubuntu workstation only.
  • Changes made to Proxmox: None.
  • Changes made to OPNsense: None.

Directory layout:

/opt/AI-Agent/
β”œβ”€β”€ docs/
β”œβ”€β”€ scripts/
β”œβ”€β”€ logs/
β”œβ”€β”€ work/
└── backups/
🟑 YELLOW β€” Create the initial agent operating rules
cat > /opt/AI-Agent/README-FIRST.txt <<'EOF'
AI MANAGEMENT AGENT HOST

OPERATING RULES

1. Preserve production.
2. Inspect before changing.
3. Do not alter Proxmox bridges without explicit approval.
4. Do not alter production routing without explicit approval.
5. Do not alter OPNsense NAT/firewall without explicit approval.
6. Do not modify LIVE services without explicit approval.
7. Do not delete containers, VMs, storage, or backups without explicit approval.
8. DEV changes must be reversible and verified.
9. Never store passwords, API secrets, private SSH keys, or provider keys in documentation or Git.
10. Established infrastructure facts should not be repeatedly re-derived unless new evidence contradicts them.
11. Document important discoveries so the next agent can continue from the current state.
EOF
  • Purpose: Gives humans and future agents a permanent safety contract.
  • Risk: YELLOW β€” creates a local documentation file only.
  • Changes made to infrastructure: None.

:yellow_circle: :laptop: Phase 10 β€” Add a Convenient SSH Alias Only After Direct SSH Works

Do not overwrite an existing SSH configuration.

🟑 YELLOW β€” Prepare the SSH client configuration
mkdir -p ~/.ssh
chmod 700 ~/.ssh

touch ~/.ssh/config
chmod 600 ~/.ssh/config

Inspect the existing file first:

cat ~/.ssh/config
  • Purpose: Ensures the SSH configuration directory/file exists with safe permissions.
  • Risk: YELLOW β€” changes local workstation files only.
  • Important: Never overwrite a working SSH config blindly.
🟑 YELLOW β€” Add a Proxmox SSH alias

If there is no existing alias for the Proxmox host, append one:

cat >> ~/.ssh/config <<'EOF'

Host pve-main
    HostName 203.0.113.10
    User root
    ServerAliveInterval 30
    ServerAliveCountMax 3
EOF
  • Purpose: Allows:
ssh pve-main

instead of repeatedly typing the full IP address.

  • Risk: YELLOW β€” modifies only the user’s SSH client configuration.
  • Important: Replace the example documentation IP with the real management IP in your private configuration.
🟒 GREEN β€” Verify the new SSH alias
ssh pve-main 'hostname; pveversion; pct list'
  • Purpose: Confirms the alias works and executes a read-only Proxmox inventory command.
  • Expected: Hostname, Proxmox version, and LXC list.
  • Risk: GREEN β€” read-only against Proxmox.

:green_circle: :laptop: Phase 11 β€” Inspect Node.js and npm Before Installing an AI Agent

Do not blindly reinstall Node.js.

Existing Node/npm installations may use different package paths, repositories, or version managers.

🟒 GREEN β€” Inspect Ubuntu runtime before agent installation
node --version 2>/dev/null || echo "NO NODE"
npm --version 2>/dev/null || echo "NO NPM"
which node 2>/dev/null || true
which npm 2>/dev/null || true
uname -m
lsb_release -a 2>/dev/null || cat /etc/os-release
  • Purpose: Determines Ubuntu version, architecture, and current Node/npm installation.
  • Why: Prevents overwriting or conflicting with an existing runtime.
  • Risk: GREEN β€” read-only.
  • Changes made: None.

STOP POINT

Review this output before installing Codex or another Node-based management agent.

Do not combine:

runtime repair
+
agent installation
+
infrastructure troubleshooting

into one large change.

Establish the baseline first.


:brain: PART 4 β€” no commands from here, just the design

:vertical_traffic_light: :brain: Phase 12 β€” Agent Permission Model

The management agent should not begin with unrestricted production authority.

Recommended agent permission levels

GREEN β€” autonomous inspection

Agent may perform:

pct list
pct config
qm list
ip addr
ip route
bridge inspection
ss
curl
ping
nc
dig
systemctl status
journalctl reads
DNS checks
HTTP health checks
application health checks
Proxmox API inventory
OPNsense read-only API queries

YELLOW β€” controlled/reversible DEV work

May eventually include:

restart DEV service
edit DEV reverse-proxy configuration
create systemd service
install package in designated DEV LXC
update DEV application
create backups before change
restore known-good DEV configuration

RED β€” explicit human approval

Agent must stop and request approval before:

changing OPNsense NAT
changing OPNsense firewall rules
changing public routing
changing Proxmox bridges
changing production network bridges
changing LIVE services
modifying storage architecture
deleting LXC/VM
destroying backups
rebooting the Proxmox host
exposing new public management services

:brain: :brain: Phase 13 β€” Add Agent Feedback / Institutional Memory

A recurring infrastructure problem is that each new AI agent begins by rediscovering information that previous agents already established.

A feedback/memory layer can reduce this waste.

Recommended documentation structure
/opt/AI-Agent/

docs/
β”œβ”€β”€ START-HERE.md
β”œβ”€β”€ NETWORK-MAP.md
β”œβ”€β”€ LXC-MAP.md
└── OPERATING-RULES.md

scripts/
β”œβ”€β”€ host-health.sh
β”œβ”€β”€ lxc-health.sh
β”œβ”€β”€ app-health.sh
└── opnsense-health.sh

logs/
work/
backups/

A repository-specific agent feedback system can additionally record lessons such as:

Do not assume an SSH alias already exists.

Do not paste terminal prompts as commands.

Live pct output overrides stale container status documentation.

Check Node/npm path and versions before reinstalling them.

Do not troubleshoot application failures by redesigning proven bridge mappings.

Prove local application health before testing public NAT and reverse proxy.

Do not repeatedly re-prove infrastructure facts that are already established.

The permanent documentation remains the authoritative source.

Agent feedback records the friction that should be prevented next time.


:red_circle: :brain: Phase 14 β€” OPNsense Automation Comes After the Management Path Is Proven

Do not begin by giving an agent the OPNsense administrator password.

OPNsense automation design

Preferred design:

Ubuntu Management Host
        |
        +------ SSH ------> Proxmox
        |
        +------ HTTPS ----> Proxmox API
        |
        +------ HTTPS ----> OPNsense API

Use a dedicated OPNsense automation identity.

Do not place credentials in:

Git
public documentation
HTML
JavaScript
Discourse posts
agent feedback records
shell-history examples

Start with read-only/status privileges.

Only after the complete diagnostic path is proven should controlled firewall or NAT modification be considered.


Key Troubleshooting Principle

Bridge crossed β€” do not repeatedly redesign proven infrastructure

When the following have already been verified:

physical NIC mapping
Proxmox bridge mapping
OPNsense WAN mapping
private service bridge
public IP blocks
gateway addressing

do not repeatedly rebuild those layers because one application or SSH connection fails.

Work from the application outward:

SERVICE LISTENING?
      ↓
PRIVATE IP REACHABLE?
      ↓
LOCAL FIREWALL?
      ↓
REVERSE PROXY?
      ↓
OPNSENSE NAT?
      ↓
OPNSENSE WAN FIREWALL?
      ↓
PUBLIC IP?
      ↓
DNS?
      ↓
TLS?

This single habit can save hours of unnecessary reconfiguration.


Public-Safety Note

All addresses, hostnames, container IDs, container names, and network ranges in this post are examples.

For public technical documentation, sanitize:

public management IPs
private subnets
hostnames
usernames
container names
container IDs where appropriate
exact software fingerprints
login-source addresses
API endpoints unique to your environment

Never publish:

passwords
API secrets
tokens
SSH private keys
provider credentials
session cookies
private certificates
recovery codes

The examples in this post intentionally use documentation/example values rather than production infrastructure identifiers.


What Success Looks Like

At the end of this initial build:

Ubuntu management workstation
        ↓
working SSH path
        ↓
Proxmox inventory available
        ↓
LXC configuration inspectable
        ↓
LXC service health testable
        ↓
application locally verifiable
        ↓
organized management workspace
        ↓
agent safety contract
        ↓
Node/npm baseline understood
        ↓
ready for controlled agent installation
        ↓
later: scoped Proxmox + OPNsense API automation

The objective is not:

β€œgive an AI root.”

The objective is:

give a controlled agent enough visibility and narrowly scoped authority to remove repetitive infrastructure work while keeping production recoverable, auditable, and understandable.

1 Like