Turn a Hand-Drawn UI into Working Code with tldraw

The Stack

Tested Against

Prerequisites

• A React app (Vite or Next.js)
• Node 20.x + npm
• `npm install tldraw` (plus `import 'tldraw/tldraw.css'`)
• A vision-model API key for the make-real step, e.g. OpenAI or Anthropic
• Commercial use: a paid tldraw license key, check the terms before you ship

Steps

1. 1

   Check your toolchain

   Make sure Node and npm are installed, that's all the deterministic setup needs. (The make-real step later also needs a vision-model API key, but nothing here does.)

   for bin in node npm grep sed; do
     command -v "$bin" >/dev/null 2>&1 || { echo "MISSING required: $bin"; exit 1; }
   done
   echo "preflight OK, node $(node -v)"

2. 2

   Drop the canvas into your React app

   After `npm install tldraw`, a five-line component gives you a full infinite canvas. This writes that App.tsx and confirms the import and stylesheet are wired correctly.

   mkdir -p src
   cat > src/App.tsx <<'EOF'
   import { Tldraw } from 'tldraw'
   import 'tldraw/tldraw.css'
   
   export default function App() {
     return (
       <div style={{ position: 'fixed', inset: 0 }}>
         <Tldraw />
       </div>
     )
   }
   EOF
   if grep -q "import { Tldraw } from 'tldraw'" src/App.tsx && grep -q "tldraw/tldraw.css" src/App.tsx; then
     echo "tldraw canvas wired: src/App.tsx"
   else
     echo "WIRING FAILED"; exit 1
   fi

3. 3

   Turn a sketch into a model prompt

   This is the deterministic heart of 'make real': your canvas is just a tree of shapes, and before any model sees it those shapes get serialized into a prompt. Here we take a sample three-shape sketch, Email, Password, Sign in, and build the exact prompt the vision model would receive. No API key needed.

   cat > canvas.json <<'EOF'
   { "shapes": [
     { "type": "geo", "props": { "geo": "rectangle", "text": "Email" } },
     { "type": "geo", "props": { "geo": "rectangle", "text": "Password" } },
     { "type": "geo", "props": { "geo": "button", "text": "Sign in" } }
   ] }
   EOF
   cat > make-prompt.js <<'EOF'
   const fs = require('fs');
   const doc = JSON.parse(fs.readFileSync('canvas.json', 'utf8'));
   const lines = doc.shapes.map(function (s) { return '- ' + s.props.text + ' (' + s.props.geo + ')'; });
   console.log('Build a working HTML UI containing:\n' + lines.join('\n'));
   EOF
   node make-prompt.js

4. 4

   Make it real (the vision-model step, not checked by CI)

   Send the canvas image plus that prompt to a vision model and render what comes back as live code. This is the magic, but it's non-deterministic and needs an API key, so CI doesn't run it, the verified badge covers only the toolchain check, the canvas wiring, and the prompt build above. This step's proof is the saved transcript.