Arcmira + n8n: a bounded YouTube transcript source shortlist

The Stack

Steps

1. 1

   Build and offline-test the hardened n8n import

   Run this from a fresh directory. It downloads only the exact upstream workflow and Apache-2.0 license at a fixed commit, checks byte sizes and digests before parsing, leaves the request and wiring unchanged, and replaces only the final result-building Code node. That node preserves missing coverage as unknown, validates the result count, video IDs and timestamps, and reconstructs links only for exact Arcmira watch URLs. The script refuses to overwrite an existing output directory. It runs synthetic and malformed-response tests without any Arcmira key or API request. Keep the downloaded LICENSE with the modified import.

   #!/bin/sh
   # Rebuild the FlowStacks-hardened n8n import from Arcmira's immutable example.
   # No Arcmira credential or API request is used here.
   set -eu
   
   SOURCE_COMMIT=1619f09f9c8f8e6798800a726b830731256248a8
   OUTPUT_DIR=arcmira-shortlist-flowstacks
   if [ -e "$OUTPUT_DIR" ]; then
     echo "Refusing to overwrite $OUTPUT_DIR; run from a fresh directory." >&2
     exit 1
   fi
   mkdir "$OUTPUT_DIR"
   cd "$OUTPUT_DIR"
   
   SOURCE_URL="https://raw.githubusercontent.com/arcmira/integrations/$SOURCE_COMMIT/examples/n8n-source-shortlist"
   curl --proto '=https' --proto-redir '=https' --tlsv1.2 --max-filesize 16384 --max-redirs 2 -fsSL \
     "$SOURCE_URL/arcmira-source-shortlist.n8n.json" -o upstream.n8n.json
   curl --proto '=https' --proto-redir '=https' --tlsv1.2 --max-filesize 16384 --max-redirs 2 -fsSL \
     "$SOURCE_URL/LICENSE" -o LICENSE
   
   node <<'NODE'
   const assert = require('node:assert/strict');
   const crypto = require('node:crypto');
   const fs = require('node:fs');
   const vm = require('node:vm');
   
   const source = fs.readFileSync('upstream.n8n.json');
   const license = fs.readFileSync('LICENSE');
   const hash = (bytes) => crypto.createHash('sha256').update(bytes).digest('hex');
   assert.equal(source.length, 5305, 'upstream workflow size changed');
   assert.equal(hash(source), 'c776e6a8b642972dfc38da574486b3b0dabfb06c6d048447255e16f3b81d83d9', 'upstream workflow digest changed');
   assert.equal(license.length, 11358, 'license size changed');
   assert.equal(hash(license), 'cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30', 'license digest changed');
   assert.match(license.toString(), /Apache License[\s\S]*Version 2\.0/);
   
   const workflow = JSON.parse(source);
   assert.equal(workflow.active, false);
   assert.deepEqual(Object.keys(workflow).sort(), ['active', 'connections', 'name', 'nodes', 'settings']);
   assert.deepEqual(workflow.nodes.map((node) => [node.name, node.type]), [
     ['Manual trigger', 'n8n-nodes-base.manualTrigger'],
     ['Search input', 'n8n-nodes-base.code'],
     ['Search Arcmira', 'n8n-nodes-base.httpRequest'],
     ['Build source shortlist', 'n8n-nodes-base.code'],
   ]);
   assert.deepEqual(Object.keys(workflow.connections).sort(), ['Manual trigger', 'Search Arcmira', 'Search input']);
   for (const [from, to] of [
     ['Manual trigger', 'Search input'],
     ['Search input', 'Search Arcmira'],
     ['Search Arcmira', 'Build source shortlist'],
   ]) {
     assert.deepEqual(workflow.connections[from].main, [[{ node: to, type: 'main', index: 0 }]]);
   }
   const http = workflow.nodes[2];
   assert.equal(http.parameters.method, 'GET');
   assert.equal(http.parameters.url, 'https://api.arcmira.com/v1/search');
   assert.equal(http.parameters.authentication, 'genericCredentialType');
   assert.equal(http.parameters.genericAuthType, 'httpBearerAuth');
   assert.deepEqual(http.credentials, {});
   assert.equal(http.retryOnFail, false);
   assert.equal(http.parameters.options.redirect.redirect.followRedirects, false);
   assert.equal(http.parameters.options.timeout, 30000);
   assert.deepEqual(http.parameters.queryParameters.parameters, [
     { name: 'q', value: '={{ $json.topic }}' },
     { name: 'after', value: '={{ $json.after }}' },
     { name: 'before', value: '={{ $json.before }}' },
     { name: 'limit', value: '3' },
   ]);
   assert.ok(!/(arc_sk_[A-Za-z0-9]+|arc_tk_[A-Za-z0-9]+|executionData|pinData|credentialId)/.test(source.toString()));
   console.log('Arcmira source pinned OK: one inactive, credential-free, fixed-origin GET at 1619f09');
   
   // This replaces only the response-to-candidate Code node. The other three
   // nodes and their connections remain byte-for-byte equivalent in the JSON data.
   const hardenedCode = String.raw`const response = $input.first().json;
   const requested = $('Search input').first().json;
   if (!response || !Array.isArray(response.chunks)) throw new Error('Expected a search response with chunks.');
   if (response.chunks.length > 3) throw new Error('More than three passages returned.');
   if (!Number.isInteger(response.returned) || response.returned !== response.chunks.length) throw new Error('Result count does not match chunks.');
   if (!response.window || !response.search_index || typeof response.note !== 'string') throw new Error('Missing coverage metadata.');
   const partial = typeof response.partial === 'boolean' ? response.partial : null;
   const failedBatches = Number.isInteger(response.failed_batches) && response.failed_batches >= 0 ? response.failed_batches : null;
   if (partial === false && failedBatches !== null && failedBatches !== 0) throw new Error('Conflicting partial and failed_batches fields.');
   const candidates = response.chunks.map((c, index) => {
     if (!c || typeof c.video_id !== 'string' || !/^[A-Za-z0-9_-]{11}$/.test(c.video_id)) throw new Error('Invalid YouTube video ID.');
     if (c.start_seconds !== null && (!Number.isInteger(c.start_seconds) || c.start_seconds < 0)) throw new Error('Invalid passage timestamp.');
     if (typeof c.watch_url !== 'string') throw new Error('Missing Arcmira watch link.');
     const match = /^(?:https:\/\/arcmira\.com)?\/watch\?v=([A-Za-z0-9_-]{11})(?:&t=(0|[1-9][0-9]*))?$/.exec(c.watch_url);
     if (!match || match[1] !== c.video_id || (match[2] !== undefined && Number(match[2]) !== c.start_seconds)) throw new Error('Unsafe or mismatched Arcmira watch link.');
     const timestamp = c.start_seconds === null ? '' : '&t=' + c.start_seconds;
     const arcmiraUrl = 'https://arcmira.com/watch?v=' + c.video_id + timestamp;
     const youtubeUrl = 'https://www.youtube.com/watch?v=' + c.video_id + timestamp;
     return {
       ordinal: index + 1, video_id: c.video_id, title: c.video_title ?? null,
       channel: c.channel_name ?? null, published_at: c.published_at,
       passage: c.text, passage_start_seconds: c.start_seconds,
       source_watch_url: arcmiraUrl, arcmira_url: arcmiraUrl,
       original_source_url: youtubeUrl, transcript_source: c.source,
       transcript_source_label: c.source_label ?? null,
       speaker_names: c.speakers ?? [], speaker_entities: c.speakers_by ?? [],
       cut_start_seconds: null, cut_end_seconds: null, playback_verified: false,
       speaker_identity_verified: false, reuse_permission: 'not established',
       selection: 'unselected candidate'
     };
   });
   return [{ json: {
     query: response.query, requested_window: { after: requested.after, before: requested.before },
     applied_window: response.window, returned: response.returned,
     partial, failed_batches: failedBatches, as_of: response.as_of,
     search_index: response.search_index, note: response.note,
     access: response.access ?? null, unlock: response.unlock ?? null, candidates
   } }];`;
   workflow.nodes[3].parameters.jsCode = hardenedCode;
   workflow.nodes[3].notes = 'Copyright 2026 Arcmira. Apache-2.0. FlowStacks modified this Code node on 2026-10-11 to fail closed on malformed source links, preserve unknown coverage fields, and validate result counts. See the accompanying LICENSE.';
   workflow.name = 'Arcmira - YouTube transcript source shortlist (FlowStacks-hardened)';
   const output = 'arcmira-source-shortlist-flowstacks.n8n.json';
   fs.writeFileSync(output, JSON.stringify(workflow, null, 2) + '\n');
   assert.equal(workflow.active, false);
   assert.deepEqual(workflow.nodes.slice(0, 3), JSON.parse(source).nodes.slice(0, 3));
   assert.deepEqual(workflow.connections, JSON.parse(source).connections);
   console.log('Hardened import built OK: original request and wiring retained; Apache-2.0 notice included');
   
   function evaluate(response) {
     const context = {
       $input: { first: () => ({ json: response }) },
       $: () => ({ first: () => ({ json: { after: '2026-08-01', before: '2026-09-01' } }) }),
     };
     return vm.runInNewContext('(function(){' + hardenedCode + '})()', context, { timeout: 1000 })[0].json;
   }
   const chunk = {
     video_id: 'aHhB3sjGjkI', video_title: 'Example', channel_name: 'Channel',
     published_at: '2026-08-02T00:00:00Z', text: 'Example spoken passage',
     start_seconds: 42, watch_url: '/watch?v=aHhB3sjGjkI&t=42', source: 'creator_captions',
   };
   const base = {
     query: 'creativity', returned: 1, chunks: [chunk],
     window: { after: '2026-08-01T00:00:00Z', before: '2026-09-01T00:00:00Z' },
     search_index: { state: 'catching_up', missing_before: '2015-06-23' },
     note: 'Only indexed material is covered.',
   };
   const normal = evaluate({ ...base, partial: false, failed_batches: 0 });
   assert.equal(normal.partial, false);
   assert.equal(normal.candidates[0].arcmira_url, 'https://arcmira.com/watch?v=aHhB3sjGjkI&t=42');
   assert.equal(normal.candidates[0].original_source_url, 'https://www.youtube.com/watch?v=aHhB3sjGjkI&t=42');
   assert.equal(normal.candidates[0].reuse_permission, 'not established');
   const unknown = evaluate(base);
   assert.equal(unknown.partial, null);
   assert.equal(unknown.failed_batches, null);
   assert.equal(evaluate({ ...base, partial: true, failed_batches: 2 }).partial, true);
   assert.equal(evaluate({ ...base, chunks: [{ ...chunk, watch_url: 'https://arcmira.com/watch?v=aHhB3sjGjkI&t=42' }] }).candidates.length, 1);
   for (const bad of [
     { ...base, chunks: [{ ...chunk, watch_url: 'https://attacker.example/watch?v=aHhB3sjGjkI&t=42' }] },
     { ...base, chunks: [{ ...chunk, watch_url: '//attacker.example/watch?v=aHhB3sjGjkI&t=42' }] },
     { ...base, chunks: [{ ...chunk, watch_url: 'javascript:alert(1)' }] },
     { ...base, chunks: [{ ...chunk, watch_url: null }] },
     { ...base, chunks: [{ ...chunk, video_id: 'bad' }] },
     { ...base, chunks: [{ ...chunk, start_seconds: '42&x=1' }] },
     { ...base, returned: 2 },
     { ...base, chunks: [chunk, chunk, chunk, chunk], returned: 4 },
     { ...base, search_index: null },
     { ...base, partial: false, failed_batches: 2 },
   ]) {
     assert.throws(() => evaluate(bad));
   }
   console.log('Offline shortlist tests OK: normal/unknown/partial cases and 10 malformed or unsafe cases');
   console.log('No Arcmira API request, key, model, media download, or transcript-rights claim was used');
   NODE
   
   printf 'Import %s/%s into n8n and attach your own HTTP Bearer Auth credential.\n' \
     "$OUTPUT_DIR" arcmira-source-shortlist-flowstacks.n8n.json
   

2. 2

   Import it inactive and attach your own credential

   Import arcmira-shortlist-flowstacks/arcmira-source-shortlist-flowstacks.n8n.json into n8n. Confirm it remains inactive, the only HTTP node is one GET to https://api.arcmira.com/v1/search, redirects and retries are off, and the credential binding is empty. Create an HTTP Bearer Auth credential in n8n from your own Arcmira key and attach it to Search Arcmira; never paste the key into a Code node, field, exported JSON, or public issue. Edit the Search input topic and inclusive-after/exclusive-before UTC dates to a window your plan serves.

3. 3

   Run once, then inspect the source and its limits

   Execute manually. Expect at most three candidates. Treat partial=true or partial=null as incomplete or unknown; read failed_batches, search_index, applied_window, access, unlock, and note even when partial=false. Open each original YouTube recording and verify the words, speaker, timestamp, and context before citing it. Establish reuse rights and exact cut boundaries separately. An empty shortlist is not evidence that nobody discussed the topic; n8n may retain execution data according to your settings. The first five search passages per request are currently free, but access, freshness, pricing, and coverage can change.