Skip to main content

The JavaScript engine

Kog runs your compiled app on MQuickJS, a single embedded JavaScript engine tuned for the ESP32. It is Fabrice Bellard's MicroQuickJS, vendored and forked in native/vendor/mquickjs. It is Kog's only engine: your app compiles to MQuickJS bytecode, which runs in place from read-only flash on the device.

Everything above the __kog host interface is the same regardless of the engine: the same compiler, signals runtime, @kog/ui and reconciler, and bytecode-only device model. MQuickJS defines only how JavaScript is executed and where the bytecode lives in memory. There is nothing to configure: kog dev and kog deploy read the connected board's bytecode version over the dev-link handshake and confirm it matches the MQuickJS bytecode the toolchain produces.

Migrated from a dual-engine build

Earlier Kog builds shipped a second engine, quickjs-ng, as the default. Kog has since unified on MQuickJS and retired quickjs-ng. A board still running the old firmware reports the retired bytecode version; kog dev / kog deploy and the browser Run on your board page detect it and ask for a one-time reflash (kog flash) to the MQuickJS runtime. KOG_JS_ENGINE survives only as a build guard: it accepts mquickjs and errors on anything else.

Why MQuickJS​

The CYD (a bare ESP32 with no PSRAM, ~220 KB of DRAM) is severely short of RAM, and it drove the engine choice. quickjs-ng, the engine Kog previously used, copies the compiled bytecode bundle into the JS heap to run it, so once an app is loaded, that board has only ~3.6 KB of free heap. Add the ~29 KB zero-import networking prelude (spec/13) and the JS heap overflows. On that class of board, quickjs-ng could render a UI but had no headroom left for networking or the hot-reload dev-link.

MQuickJS targets exactly those costs:

  • Bytecode executes in place from flash. It is never copied into the JS heap.
  • The standard library lives in ROM. It is baked into const C tables at build time, not built in RAM at boot.
  • The engine uses ~10 KB of RAM and has a compacting GC. No libc malloc, no reference counting, no fragmentation; it allocates only inside a caller-provided buffer.

The engine's own code is also far smaller: on a host -Os build the core is ~144 KB vs quickjs-ng's ~634 KB (~4.4× smaller). In the real firmware image, that showed up at the migration milestone as a CYD build of 1.38 MB, about 394 KB (~22%) smaller than the equivalent quickjs-ng build. (Current images are larger because they include the networking stack.)

Run in place from flash​

Running in place is where the RAM savings come from, and it is the key difference from the engine Kog replaced. quickjs-ng loaded bytecode by copying it into the JS heap; MQuickJS's JS_LoadBytecode references a flash-resident buffer directly, so the bundle never enters the heap.

The catch is that MQuickJS bytecode is base-specific: relocation bakes absolute pointers for the exact address the buffer will be mapped at, and the loader asserts the buffer sits at that base (see spec/16). Kog resolves this with a per-boot relocation cache:

  • On boards with a kogpre region (the CYD), each A/B app slot stores the canonical signed base-0 image, which is never rewritten. Every boot, after verifying it, the firmware relocates a RAM copy for the virtual address that slot maps at (discovered per boot via the ESP32 flash MMU, not assumed) and persists it in the same slot as a derived KOGR trailer. The app then runs in place from that trailer, with no load-time pointer fix-up. The trailer is recomputed from the verified canonical bytes on any mismatch, so either slot is bootable and a firmware rebuild doesn't invalidate the installed app. The deploy arc (stage → trial boot → confirm → promote) is validated on-glass.
  • The boot preludes (the runtime prelude, which is a builtins shim plus the __kog and clock assembly, and the networking prelude) are firmware-baked. On boards with a read-only kogpre flash region (the CYD), they are relocated for that region's VA (Vp, also discovered per boot) and loaded in place; kogpre is re-provisioned when its VA or the stdlib generation changes. Boards without a kogpre partition (the ESP32-S3 layout) copy the baked prelude to RAM. Either way, the preludes are never source-eval'd.
  • Relocation happens on the device, against [ROM stdlib + prelude@Vp]. Doing it on the device keeps the host bundle engine-agnostic: one signed .kogb, with no engine-specific atom addresses baked in by the host.
  • Boot is a single load-all-then-run-all pass: map the prelude and app, JS_LoadBytecode both, then run prelude → app.

The payoff, measured on the working ROM-stdlib harness: the minimum JS heap needed to mount an app drops from 128 KB (source-eval) to 48 KB (run-from-bytecode), because the bytecode body stays in the flash buffer instead of the heap.

An important, counterintuitive finding: source-eval on MQuickJS needs more heap than quickjs-ng's source-eval did. All of the RAM savings come from the bytecode-in-flash path, so on device that path is required, not an optimization. The networking prelude in particular must be flash-resident bytecode. Source-eval'ing it (the way the old engine ran everything) is a big part of why the CYD runs out of memory.

The ES5-lowering pipeline​

MQuickJS is a strict-mode ES5 subset: no let/const, arrow functions, template literals, class, destructuring, spread, optional chaining, or nullish coalescing at the syntax level. Upstream also lacked most of the ES2015+ library; Kog's fork adds Promise, Map, Set, the iteration protocol and the builtins an RN/TS developer relies on as engine-native code (fork divergences 6, 7, 8 and 10 below). WeakMap, WeakSet, Proxy, Reflect, BigInt and Intl are still absent. Kog's compiler emits modern ES2020, so the build adds a lowering stage after bundling:

esbuild (iife) → tsc target ES5 → Babel elision-fill

  1. esbuild emits an IIFE, not an ES module. MQuickJS has no module system, so the bundle must be a self-contained script. It boots itself via a top-level if (globalThis.__kog) mount(); there is no export to strip.
  2. tsc with target: ES5, downlevelIteration: false lowers all syntax in one tool, including async/await, which becomes TypeScript's Symbol-free __awaiter/__generator state machines. Kog avoids Babel's regenerator on purpose: regenerator-runtime pulls in Symbol and per-app RAM the CYD can't spare. downlevelIteration: false avoids tsc's per-step iterator helpers; the compiler protects every for…of around this pass so it reaches the engine as a native loop, which handles arrays, strings, Map, Set and custom iterables.
  3. A ~5-line Babel plugin fills array elisions. tsc's __generator emits array holes like _.trys.push([0, 3, , 4]), and MQuickJS's stricter parser rejects array elisions, so the pass replaces each hole with undefined.

The same pipeline runs everywhere Kog compiles an app: the CLI's kog dev/kog deploy, and the in-browser docs playground (which lowers with the browser twin of the CLI pass). The syntax gap therefore closes almost entirely on the compiler side, at zero runtime cost. A small baked JS prelude runs before your app. It assembles the __kog host tree and fills any library gaps the engine itself does not cover; most of the ES2015+ library is now engine-native (divergence 10). The prelude is plain ES5 with no C. These pieces are not part of the ROM stdlib, which is the C __kog/console host surface plus the engine's own builtins. Promise, Map and Set are engine-native ROM classes, not prelude code (fork divergences 6 and 7).

The ROM standard library​

A conventional embedding builds the __kog host surface at runtime with JS_NewCFunction. MQuickJS has no such call. The entire host surface (~30 __kog methods, ~70 functions including hw/flatlist/net) must be declared at build time as JSPropDef tables and baked into ROM by mquickjs_build's codegen; only the C bodies are linked in. This has two consequences:

  • Nested host objects can't be baked inside another baked object, so __kog.hw / .flatlist / .net are declared as top-level ROM objects and the boot prelude assembles the tree (__kog.hw = __kogHw; …).
  • The 32-bit device word size matters: the ROM JSWord tables are word-size-specific, so the host codegen runs with -m32 to emit the tables the xtensa build links against.

Engine fork divergences​

Per the fork charter (optimize for the ESP32; diverge freely, record everything), the vendored engine carries ten intentional in-tree changes to mquickjs.c itself. Each is diffable against the pinned upstream baseline and documented in full in native/vendor/mquickjs/PROVENANCE.md and spec/16. (PROVENANCE numbers them 1–9: it counts Map/Set and the iteration protocol as one divergence.)

  1. Catch-scope relaxation. MQuickJS has no block scope and rejects a duplicate catch binding name in one function. Minified bundles reuse short catch names (e/t/n) across sibling catch clauses, so the parser reuses the existing local slot instead of erroring. (The binding still leaks, but our minified input never relies on a catch value persisting across sibling clauses.)
  2. ArrayBuffer C API. MQuickJS exposes no ArrayBuffer accessors, which the networking byte-marshaling needs, so JS_GetArrayBuffer / JS_NewArrayBufferCopy were added (moving-GC caveat: copy the pointer out before the next allocation).
  3. ROM atom-table sizing. Each JS_LoadBytecode pushes one ROM atom table; upstream sized the array for stdlib + one module. Kog loads several in place from flash (runtime prelude + net prelude + app), so N_ROM_ATOM_TABLES_MAX was raised 2 → 6. The cost is a few extra pointers in the context; atom lookups walk the actual table count, so there's no per-lookup penalty.
  4. Remote-VA relocation deref fix. Device provisioning relocates a RAM copy of the combined prelude for the flash VA it will be mapped at later, which is an address that isn't mapped yet. Upstream's relocation dereferenced the not-yet-mapped new base and hung the watchdog on-glass; the fix dereferences the referenced block at its current address while storing the new-base pointer. For the in-place case the two addresses coincide, so behavior is byte-identical there.
  5. Copy-on-promote string interning. Interning a not-yet-unique string upstream mutates the input string in place. For bytecode run in place from read-only flash, an app that uses a constant-pool string as a property key would write into flash, which causes an ESP32 LoadStoreError on-glass. The fix canonicalizes into a fresh RAM copy and returns that. It is the flash-resident analogue of the "don't mutate ROM" rule.
  6. Native Promise + an engine microtask queue. Promise (with then/catch/finally and Promise.resolve/reject/all/allSettled/race) is an engine class with a C microtask queue and unhandled-rejection reporting. It replaced an earlier JS-shim Promise in the boot prelude when quickjs-ng was retired. C callers create promises with JS_NewPromiseCapability.
  7. Native Map/Set. ROM-stdlib classes with SameValueZero keys and a hybrid lookup index (linear below 16 entries, hashed above). get/set/has/delete are O(1) amortized (deletes leave a tombstone and the table compacts, with live cursors surviving), size is O(1), and iteration is in insertion order and live. They replaced the earlier array-backed JS shims in the prelude.
  8. The ES iteration protocol. A ROM Symbol (string-keyed: Symbol.iterator is a string), [Symbol.iterator] on arrays, strings, Map and Set, Array.from, new Map(iterable)/new Set(iterable), and a native for…of over arrays, strings, typed arrays, Map, Set and custom iterables. for…of over an array or a Set allocates nothing; over a Map it allocates one [key, value] pair per step, as V8 does. keys()/values()/entries() are live iterators, as in V8, and a native __read makes const [a, b] = set work. An early break does not call iterator.return(). App code goes through TypeScript's ES5 lowering, so the compiler protects each for…of before that lowering and restores it after, which keeps the loop native.
  9. JS_NewShortInt shifts unsigned. Upstream left-shifts a signed int32_t, which is undefined behavior for every negative number entering JS (including every -kog_err_t the HAL and net bridges return). The fork shifts an unsigned value instead, with bit-identical results.
  10. The ES builtins and semantics an RN/TS developer relies on. A full ES5 Date, with no time zones: local time is UTC, and Date.now reads the same clock as the clock global. JSON gets the replacer, space, toJSON and the reviver. Own keys list integer keys first. Arrays get real holes, which cost no extra RAM, and a non-index key on an array throws instead of silently misbehaving. sort, apply and bound functions follow ES. The additions include encodeURIComponent and its siblings, queueMicrotask, Promise.any with a global AggregateError, Error {cause}, JSON.parse's exact grammar, the common ES2015–2022 Object/Array/String/Number/Math methods, typed-array methods and iterators, and RN's global alias. Everything is ROM tables, so there is zero RAM per context; the cost is about 29 KB of device flash. The remaining language differences are listed in Coming from React Native.

Adapter & prelude conventions (not engine changes). Three behaviors shape the boot but live in the C adapter or the JS prelude, not the vendored engine:

  • Load-all-then-run-all boot ordering. Running a loaded prelude interns atoms into RAM, after which JS_LoadBytecode refuses to load more ("no atom must be defined in RAM"). So the boot maps and loads every module in place first, then runs them. The adapter enforces this invariant (paired with the atom-table sizing above).
  • The eval_bundle base dispatch. The adapter's kog_js_eval_bundle branches on the bytecode header's base_addr:
    • Relocated for the address where it sits: load in place (don't rewrite read-only flash).
    • base_addr == 0: a base-0 image in writable RAM (the simulator and the kog dev hot-reload push). Self-relocate for the current address.
    • Any other base: a deploy base mismatch. Fail loudly rather than fault on read-only flash.
  • The runtime-prelude Symbol shim. The net prelude installs Symbol.iterator/asyncIterator on its streams; a small shim in the JS prelude gives Symbol.* string keys when the engine has no Symbol, so those installs succeed as plain string-keyed properties. The engine now has a ROM Symbol (divergence 8), so the shim is only a fallback.

Status​

MQuickJS ships in every prebuilt image now (it is the only engine):

  • Desktop simulator: the six original example apps (counter, gallery, tic-tac-toe, flatlist-stress, hardware-demo, net-dashboard) mount and render real LVGL widget trees on MQuickJS.
  • Live CYD hardware: gallery and net-dashboard render, running in place from read-only flash. They are the first Kog apps to do so on a bare ESP32. Post-init free heap is 96,796 B (~95 KB) with net-dashboard running stably, versus 3,636 B under the retired quickjs-ng runtime. Running in place is what lets networking and the ~16 KB hot-reload dev-link fit on the CYD at all.
  • Live ESP32-S3 hardware: the Waveshare 1.46B prebuilt is on-glass validated (display, touch, and kog dev hot reload over native USB; see native/firmware/HARDWARE-VALIDATION.md). The Waveshare AMOLED-2.06 prebuilt runs on its real board too: display, the touch controller, kog dev hot reload and kog deploy are confirmed, with ~6.49 MB free heap after init (PSRAM-backed).
  • Pending: final RAM slimming and the broader on-device convergence.

For the full derivation, the RAM benchmarks, the deploy/relocation layout, and the complete gap analysis, see native/vendor/mquickjs/PROVENANCE.md and spec/16-rom-bytecode-inplace.md in the repository.