FreeLLMAPI: one socket, sixteen free model tiers with auto-fallback

The Stack

Steps

1. 1

   Write the client config and validate it

   After installing the backend (curl install or docker compose, port 3001) and pasting your provider keys in the dashboard, capture the unified key and your fallback order. CI checks the client config shape; the actual chat call needs your key and is fenced.

   cat > freellmapi.json <<'JSON'
   {
     "endpoint": "http://localhost:3001/v1",
     "unified_key": "freellmapi-REPLACE_WITH_YOUR_KEY",
     "fallback_chain": ["google", "groq", "cerebras", "mistral", "cohere", "nvidia"],
     "request": { "model": "auto", "messages": [{ "role": "user", "content": "hi" }] }
   }
   JSON
   node -e '
   const fs = require("fs");
   const c = JSON.parse(fs.readFileSync("freellmapi.json", "utf8"));
   function bad(m) { console.error("BAD: " + m); process.exit(1); }
   if (!c.endpoint || c.endpoint.indexOf(":3001/v1") === -1) bad("endpoint must target the FreeLLMAPI port :3001/v1");
   if (!c.unified_key || c.unified_key.indexOf("freellmapi-") !== 0) bad("unified_key must start with freellmapi-");
   const chain = c.fallback_chain;
   if (!Array.isArray(chain) || chain.length < 2) bad("fallback_chain must list at least 2 providers");
   if (!chain.every((p) => typeof p === "string")) bad("fallback_chain entries must be strings");
   if (new Set(chain).size !== chain.length) bad("fallback_chain has duplicate providers");
   const req = c.request || {};
   if (!req.model) bad("request.model is required (use auto or a specific id)");
   if (!Array.isArray(req.messages) || req.messages.length === 0) bad("request.messages must be a non-empty array");
   if (!req.messages.every((m) => m.role && typeof m.content === "string")) bad("each message needs a role and string content");
   console.log("config OK: FreeLLMAPI at :3001/v1, unified key prefix ok, fallback chain of " + chain.length + " provider(s), request well-formed");
   '

2. 2

   Call it (the model step, not checked by CI)

   Send the request with your real unified key (the Step 3 curl in the guide). A reply means the backend is live and routing. Expect sharper answers in the morning and weaker late-day fallbacks. The call and its quality are fenced.