🔑 34 Free LLM Providers Behind One Key — On Your Own Box

[BCBC]

One self-hosted gateway pools every free AI provider into a single endpoint you control — built to survive reboots and roll back the moment anything breaks.

What you are actually building

FreeLLMAPI collapses 34 free providers and 635 free model endpoints — roughly 7.4 billion tokens a month — into one OpenAI-compatible /v1 endpoint. One base URL, one bearer token.

    BEFORE                          AFTER
    ──────                          ─────
    34 signups                      http://your-box:3001/v1
    34 SDKs                    ➜    one key
    34 rate limits                  router picks whoever
    34 places it breaks             still has quota left

Point Claude Code, Cursor, Cline or any OpenAI client at it and stop thinking about which provider is rate-limited today.

Running it yourself is the part that matters. Your provider keys sit AES-encrypted in a SQLite file on hardware you own. Nothing is held by a third party.

Why this guide and not the README

The upstream README tells you how to start it. It does not tell you how to survive it.

• A backup you actually verified — vzdump, then zstd -t, then a full tar traversal, then md5 and sha256, checked. A file existing is not proof it restores.
• A proof chain, not a health page — every layer tested with curl before the next one is added. Local, then private, then proxy, then a real authenticated completion.
• A clone manifest — 12 things that must differ per instance. Clone blindly and two gateways silently share one encryption key and one set of provider credentials. It will work. It is ruined.
• Reboot persistence proven, not assumed.

The 13 sections

 ┌ PLAN ────────────────────────────────────────────────┐
 │  1  worksheet — CTID, IP, bridge, storage            │
 │  2  🛡️ verified cold backup + snapshot   ← do first   │
 └──────────────────────────────────────────────────────┘
 ┌ BUILD ───────────────────────────────────────────────┐
 │  3  pct create — unprivileged LXC                    │
 │  4  clone + npm build, pinned commit                 │
 │  5  🔑 ENCRYPTION_KEY outside the repo, 0600         │
 │  6  systemd unit + hardening flags                   │
 └──────────────────────────────────────────────────────┘
 ┌ PROVE ───────────────────────────────────────────────┐
 │  7  private path :3001 before any proxy              │
 │  8  Nginx reverse proxy + TLS                        │
 │  9  real completion through the whole chain          │
 │ 10  restart + LXC reboot persistence                 │
 └──────────────────────────────────────────────────────┘
 ┌ SCALE ───────────────────────────────────────────────┐
 │ 11  clone vs fresh build — the identity manifest     │
 │ 12  fleet cold-backup order, one at a time           │
 │ 13  ✅ final proof checklist, 21 lines               │
 └──────────────────────────────────────────────────────┘

Sections 1 and 2 change nothing. They only look and back up.

Already routing across free providers and just want the fallback order? That is the sibling thread: Free Token Limits.

Every CTID, IP, hostname and path below is a placeholder. Fill in the worksheet in section 1 first.

---

Upstream project:

https://github.com/tashfeenahmed/freellmapi

1. Architecture worksheet

Proxmox host
  |
  |-- private bridge <vmbr-lan>
  |      |-- FreeLLMAPI LXC <CTID>, <APP_IP>:3001
  |      `-- Nginx LXC <PROXY_CTID>, <PROXY_IP>:80/443
  |
  `-- backup storage <BACKUP_STORAGE>

Fill this in first:

FreeLLMAPI CTID:    <CTID>
Hostname:           <freellmapi-dev-01>
Private address:    <10.20.30.122/24>
Gateway:            <10.20.30.1>
Bridge:             <vmbr-lan>
LXC storage:        <LXC_STORAGE>
Backup storage:     <BACKUP_STORAGE>
Nginx proxy:        <PROXY_PRIVATE_IP>
Public hostname:    <llm.example.net>

Verify that both the CTID and IP are unused:

pct list
qm list
pvesm status
ip neigh show
ping -c 2 <PROPOSED_PRIVATE_IP>

A failed ping does not prove the address is free.

Also inspect:

DHCP leases
DNS
firewall aliases
static-address documentation
existing Proxmox guests
intended subnet plan

2. Back up before changing an existing environment

Inventory:

hostname
pveversion
pct list
qm list
pvesm status
cat /etc/pve/storage.cfg
df -h
lsblk

Inspect related containers:

for id in <RELATED_CTIDS>; do
  pct config "$id"
  pct listsnapshot "$id"
done

Verified cold backup:

vzdump <SOURCE_CTID> \
  --mode stop \
  --storage <BACKUP_STORAGE> \
  --compress zstd

Verify:

test -s <BACKUP_ARCHIVE>
zstd -t <BACKUP_ARCHIVE>
zstdcat <BACKUP_ARCHIVE> | tar -tf - >/dev/null

md5sum <BACKUP_ARCHIVE> > <BACKUP_ARCHIVE>.md5
sha256sum <BACKUP_ARCHIVE> > <BACKUP_ARCHIVE>.sha256

md5sum -c <BACKUP_ARCHIVE>.md5
sha256sum -c <BACKUP_ARCHIVE>.sha256

If any verification fails, stop.

A file existing is not proof that the backup is usable.

Where supported, create a pre-change snapshot after the full verified backup:

pct snapshot <SOURCE_CTID> pre-freellmapi-change-<YYYYMMDD> \
  --description 'After verified backup; before FreeLLMAPI change'

pct listsnapshot <SOURCE_CTID>

Snapshots do not replace backups.

3. Create a small Debian/Ubuntu LXC

Select a reviewed template:

pveam update
pveam available | grep -E 'debian|ubuntu'
pveam download <TEMPLATE_STORAGE> <TEMPLATE_FILE>

Example creation:

pct create <CTID> <TEMPLATE_VOLUME> \
  --hostname <HOSTNAME> \
  --cores 2 \
  --memory 2048 \
  --swap 512 \
  --rootfs <LXC_STORAGE>:8 \
  --net0 name=eth0,bridge=<BRIDGE>,ip=<PRIVATE_IP/CIDR>,gw=<GATEWAY>,type=veth \
  --unprivileged 1 \
  --features nesting=0 \
  --onboot 1 \
  --start 1

Prefer an unprivileged container. FreeLLMAPI should not require a privileged LXC merely to run a Node application.

Verify:

pct status <CTID>
pct config <CTID>
pct exec <CTID> -- ip -br address
pct exec <CTID> -- ip route
pct exec <CTID> -- getent hosts github.com

4. Install prerequisites and build the application

Inside the LXC:

apt update
apt install -y git build-essential python3 curl ca-certificates openssl

Verify the actual runtime:

node --version
npm --version

Install a Node.js version that satisfies the exact FreeLLMAPI release you selected.

Create the runtime account:

useradd \
  --system \
  --home-dir /opt/freellmapi \
  --shell /usr/sbin/nologin \
  freellmapi

Clone and build:

git clone https://github.com/tashfeenahmed/freellmapi.git /opt/freellmapi
chown -R freellmapi:freellmapi /opt/freellmapi

cd /opt/freellmapi

If sudo exists:

sudo -u freellmapi npm install
sudo -u freellmapi npm run build

If it does not:

runuser -u freellmapi -- npm install
runuser -u freellmapi -- npm run build

Record the exact commit:

git rev-parse HEAD

Prefer a reviewed tag or commit for reproducibility.

5. Store secrets outside the repository

install -d -o root -g root -m 0700 /etc/freellmapi

Generate the encryption key:

sh -c '
  umask 077
  printf "ENCRYPTION_KEY=%s\n" "$(openssl rand -hex 32)" \
    > /etc/freellmapi/freellmapi.env
'

Protect it:

chown root:root /etc/freellmapi/freellmapi.env
chmod 0600 /etc/freellmapi/freellmapi.env
stat -c '%U:%G %a %n' /etc/freellmapi/freellmapi.env

Provider credentials do not belong in:

Git
Forgejo
source-tree .env files
Nginx configuration
Proxmox notes
screenshots
forum posts
shared/public backups

Back up the application database and encryption key together to protected storage.

6. Create and enable the systemd service

Verify the actual application entry point first.

Example service:

[Unit]
Description=FreeLLMAPI Gateway
Documentation=https://github.com/tashfeenahmed/freellmapi
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=freellmapi
Group=freellmapi
WorkingDirectory=/opt/freellmapi
Environment=NODE_ENV=production
Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
EnvironmentFile=/etc/freellmapi/freellmapi.env

# VERIFY AGAINST YOUR BUILD:
ExecStart=/usr/bin/node /opt/freellmapi/server/dist/index.js

Restart=on-failure
RestartSec=5s
TimeoutStopSec=30s
KillSignal=SIGTERM

NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
ProtectHome=true
ReadWritePaths=/opt/freellmapi/server/data

[Install]
WantedBy=multi-user.target

Prepare data directory:

install \
  -d \
  -o freellmapi \
  -g freellmapi \
  -m 0700 \
  /opt/freellmapi/server/data

Enable:

systemctl daemon-reload
systemctl enable --now freellmapi

Verify:

systemctl --no-pager --full status freellmapi
journalctl -u freellmapi -n 100 --no-pager
ss -lntp | grep ':3001'
curl -sS -o /dev/null -w 'local UI: %{http_code}\n' http://127.0.0.1:3001/

7. Verify the private path before Nginx

From the Proxmox host:

nc -vz -w 5 <APP_PRIVATE_IP> 3001

UI:

curl -sS \
  -o /dev/null \
  -w 'private UI: %{http_code}\n' \
  http://<APP_PRIVATE_IP>:3001/

Test the expected unauthenticated behavior:

curl -sS \
  -o /dev/null \
  -w 'unauthenticated API: %{http_code}\n' \
  -H 'Content-Type: application/json' \
  -d '{"model":"auto","messages":[{"role":"user","content":"ping"}]}' \
  http://<APP_PRIVATE_IP>:3001/v1/chat/completions

Verify the actual expected result against your selected release.

Do not configure the public proxy until the private path works reliably.

8. Reverse proxy through Nginx

Dedicated hostname example:

server {
    listen 443 ssl http2;
    server_name <llm.example.net>;

    ssl_certificate     <FULLCHAIN_PATH>;
    ssl_certificate_key <PRIVATE_KEY_PATH>;

    location / {
        proxy_pass http://<APP_PRIVATE_IP>:3001;
        proxy_http_version 1.1;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

        proxy_read_timeout 300s;
        proxy_buffering off;
    }
}

Validate first:

nginx -t

Then reload:

systemctl reload nginx

Subpath deployments

A route such as:

https://example.net/freellmapi-01/

can be useful when hosting several instances behind one domain, but it adds complexity.

Test carefully for:

absolute asset URLs
WebSocket paths
redirects
API prefixes
frontend base-path assumptions
cookie paths
trailing-slash behavior

A dedicated hostname is usually simpler if you control DNS.

9. Prove provider and model routing

A healthy dashboard is not enough.

Use a protected API key and perform an actual completion request.

Example pattern:

read -rsp 'Unified API key: ' FREEAPI_KEY
echo

curl -sS \
  https://<PUBLIC_HOST>/v1/chat/completions \
  -H "Authorization: Bearer ${FREEAPI_KEY}" \
  -H 'Content-Type: application/json' \
  -d '{"model":"auto","messages":[{"role":"user","content":"Reply with exactly: gateway works"}]}'

unset FREEAPI_KEY

Prove that:

client
 -> Nginx
 -> FreeLLMAPI
 -> configured provider
 -> selected model
 -> response
 -> client

10. Prove restart and reboot persistence

Service restart:

systemctl restart freellmapi
systemctl is-active freellmapi
ss -lntp | grep ':3001'

Repeat the API test.

Then, during an approved maintenance window, restart the LXC:

pct reboot <CTID>

After it returns:

pct status <CTID>
pct exec <CTID> -- systemctl is-enabled freellmapi
pct exec <CTID> -- systemctl is-active freellmapi
pct exec <CTID> -- ss -lntp

Repeat:

• private UI test;
• private API test;
• reverse-proxy test;
• authenticated provider completion.

11. Clone versus fresh build

Once one reference LXC is fully proven, you can either build additional gateways from scratch or clone the known-good reference.

Do not clone blindly.

Before cloning, decide how the following must differ per instance:

CTID
hostname
private IP
application identity
database/state
admin account
API keys
encryption key
provider credentials
Nginx route
backup identity
monitoring identity

A clone that accidentally shares identity or secret material with its source may work technically while being operationally wrong.

Use a written clone manifest and verify every destination independently.

12. Fleet backup pattern

For multiple gateway LXCs, a safe cold-backup pattern is:

remember previous state
  ->
graceful shutdown
  ->
vzdump
  ->
zstd integrity test
  ->
full tar traversal
  ->
MD5 generation + verification
  ->
SHA-256 generation + verification
  ->
restart if previously running
  ->
verify service
  ->
next LXC

Do one container at a time.

Do not leave an entire service fleet down while backups run.

A later replication stage can copy only verified archives to a second machine and verify SHA-256 again after transfer.

13. Final proof checklist

[ ] CTID/IP collision checks complete
[ ] LXC unprivileged unless a proven need says otherwise
[ ] Exact FreeLLMAPI commit recorded
[ ] Build succeeds
[ ] Secrets outside repository
[ ] systemd enabled
[ ] systemd active
[ ] Listener present
[ ] Local UI responds
[ ] Private UI responds
[ ] Authentication behavior tested
[ ] Provider configured
[ ] Authenticated completion succeeds
[ ] Nginx config passes nginx -t
[ ] Public proxy responds
[ ] Restart persistence proven
[ ] LXC reboot persistence proven
[ ] Verified backup exists
[ ] Snapshot exists where supported
[ ] Rollback path documented
[ ] Clone-specific identity proven
[ ] Remaining unknowns documented

BCBC field rule: evidence beats memory

The safest operator is not the one who remembers the most.

The safest operator is the one who can prove:

what was true before
what changed
why it changed
what proves the new state
where the recovery artifact lives
how to roll it back
what is still unknown

Erase the conversation. Preserve the evidence. Continue safely.