HILOS / take-home Demo Open the editor
Staff engineer take-home · Rocco

Reshape the whole shoe, on any device.

One tool, Grab, on the full 700k-face mesh. Built twice: once on the HILOS stack, once in Rust for the web and a native iPad app.

700kfaces, no proxy mesh
~0 msCPU per pen move
92 / 96 MBworst case, 100 undo steps
16 / 16shape hashes match across builds

Three ways to try it

Desktop · iPad
The TypeScript editor with the toe of the shoe raised by a Grab stroke
TypeScript · React · three.js · WebGL2

TypeScript editor

  • Drops into Ghost. The same stack HILOS ships: React Three Fiber and drei on WebGL2.
  • A pen move costs one vector. The GPU draws the stroke; the CPU work lands when the pen lifts, in slices that keep every frame under 10 ms on desktop.
  • Nothing is lost. Every stroke is saved on pen-up, synced to a server, and survives a reload, a lost GPU, or going offline.
  • Undo is exact, to the byte, and the sole never moves.
Open the editorTry the demo shoe
The Rust and WebAssembly editor in Chrome, running on WebGPU
Rust · wgpu · WebGPU, WebGL2 fallback

WebAssembly editor

  • 0.7 ms of CPU a frame on WebGPU in desktop Chrome, 2.1 ms on WebGL2.
  • The same code as the iPad app. One codebase for web and native, so fewer platform-specific bugs.
  • Proven to match. Its Rust core replays the same recorded strokes to a shoe bit-identical to the TypeScript build's, at every step (checked natively; the wasm run is next).
  • Saves on the device. No server sync in this build.
Open wasm.simmer.io
The native iPad app running on Metal
Rust · Metal · iPadOS 16+

iPad app

  • Native Metal, outside Safari's per-tab memory limit, the one that kills heavy web pages on iPad.
  • Works offline. The shoe ships inside the app, and edits survive closing it.
  • 1.0 ms of CPU a frame in the iPad simulator.
Waiting for Apple's beta review
  1. On an iPad, install TestFlight from the App Store.
  2. Open the TestFlight link from Rocco (sent once Apple approves the build).
  3. Tap Accept, then Install "HILOS Shoe Editor". No sign-in.
iPad only. Screenshot from the iPad simulator.
DesignDecisionsEvery choice in the TypeScript build, with the numbers behind it. How it was builtProcessThe agents, how their work was checked, and what went wrong.

Two builds

I built two versions of the app. One matches the stack HILOS uses: TypeScript, React and Vite, with three.js through React Three Fiber and drei, drawing with WebGL2. The other is an experimental build in Rust, compiled to WebAssembly for the web and to a native iOS app from the same code. Android, native Mac and Windows wouldn't be a stretch either: the Rust core and the libraries under it (wgpu for drawing, egui for the UI) already support those platforms, though I haven't built for them.

The benefits of the Rust build: one codebase for web and native, so less opportunity for platform-specific bugs, and native just runs faster. On the same CPU work, the Rust build measured 11.7 ms a frame where TypeScript took 17 to 25 ms. And the native app doesn't crash the way a Safari tab does on mobile.

Why Safari tabs crash: iOS kills a web page that uses more memory than it allows a tab. That limit is far below the device's RAM, it isn't documented, and the page pays for the browser's own overhead on top of its own data. That's why the brief has a 96 MB budget. A native app talks to the GPU through Metal directly and gets a much larger memory allowance, so by design the native build steps out from under the tab limit. The web version would effectively become a marketing tool: get people up and running quickly, with strong nudges towards the native versions, and especially strong ones if a crash was detected.

What gives me an edge in the age of AI

Elias asked me what gives me a competitive advantage in the age of AI.

  • Information gathering. Two things. In the first interview I asked Elias for access to the actual product so I could play with it. I did, and eventually figured out how to get a 3D scene to render. I had Claude inspect the code and help me understand how to build a replica, but better. I also messaged Beichen and got him to tell me exactly how the interview is weighted.
  • Technical skill. Mine is fine. I'm a jack of all trades, master of none.
  • Engineering instincts. These are top notch. I've run teams.
  • No longer bounded by output. In previous lives I was bounded by my ability to produce quality code fast enough, and I'd often anchor on how hard something was to engineer. That's no longer an issue.
  • Rewrites. Before AI, I would NEVER EVER have suggested that rewriting a codebase was the right move. My feelings have changed in the age of unlimited intelligence.

I've done this before

This setup is exactly what we did at Meta for VR avatars. There was a C++ core, and a wrapper for Unity developers that called the C++ APIs and added Unity rendering, shading, tests and perf checks. There was also a pure C++ version that Oculus Home used, and a WebAssembly tool that let us browse avatars directly on internal websites. I mentored the junior developer who built the web version.

The C++ core only compiled for Windows and Android (the Oculus VR OS). With some clever #ifdefs and header replacements, I got it compiling for Mac and iOS. That unlocked two things: games like PokerStars could have cross-platform Meta avatars, and developers inside Meta were no longer locked to Windows. It opened up the entire company to innovating on avatars.

Defending the code

The TypeScript version is easier for me to defend, because I have experience in that language. The Rust one is completely experimental, and I'd have a hard time defending the technical decisions made there: I don't have a strong background in Rust. For reference, here's the document I had generated that will help me defend the TypeScript version.

Of note: the agent told me not to build the Rust version, because I wouldn't be able to defend the code as well. That's true. But it went against my engineering instinct to give the best possible experience on every platform. We're no longer bounded by engineers' skill level, but by ambition.

If I joined HILOS

If I were hired as a staff engineer at HILOS, my first move would likely be to point a long-running agent process at building a cross-platform native prototype of Ghost. But a prototype is one thing, and an entire platform shift is another. The steps I'd take:

  • First, analyse all the tests to make sure they cover as many use cases as possible.
  • Build an analytics tool that captures all user input and can play sessions back for us to inspect. Every non-deterministic step (for example, AI libraries generating new processes) would be recorded deterministically, as the LLM's output, so sessions replay exactly.

Playback pays off twice:

  1. We see how real users actually interact with the product.
  2. While building a faster native app, we can automate playback and record video of each session on every platform, to make sure they produce the same results.

Security

I think all code launched these days should be highly secure. It only takes a single rogue agent to bring down a whole system. For this project:

  • The repos are private, and the shoe is only served from this site, which tells search engines not to index it.
  • The deploy token can only edit Workers. It lives in 1Password and reaches the deploy tool through op run, never in chat, a file, or an agent's screenshot.
  • The server checks every saved edit byte for byte before storing it, and caps request sizes (including a compressed-size bomb), because one bad record would break that shoe for everyone.
  • The agents ran under permission rules, and when a rule blocked something, they handed it back to me instead of working around it.
  • What's missing: there are no accounts. An unguessable shoe URL is the only key, which is fine for a prototype and not for a product.

The agent setup

I generally use Claude. I started my agents on an Arch Linux machine, because it removes all the distractions of my usual Mac. I set up a single desktop with herdr (a tmux derivative) and closed all my previous sessions. Then I realized the main platform was the iPad. So I used /remote-control, fired up my Mac, and had the agents hand off everything they'd been working on.

What I'd do differently

  • I normally have an adversarial review agent review my code: a reviewer specifically told NOT to let the code pass. In my push to move fast I skipped it, and it would have produced a better product overall.
  • I'd normally spend more time getting a local development environment going, rather than deploying at every step.
  • I absolutely think the app is light on features.

What I did right

I made an easy-to-type website where you can get every build of the project. Deployment and ease of use are absolutely key to getting someone to adopt your project. A 37-step iOS upload? No thanks. Deploying to TestFlight? Great.

Stuff I wish you'd told me

I didn't know this would be iPad-centric and built around iPad memory limits. Yes, you can simulate it in a local Safari, but that isn't the same as the real hardware. I drove over to my ex-wife's house to grab the iPad. I had an old Apple Pencil lying around to test pressure-based input, but I could never get it going. It was dead. I mean dead dead.

How I wrote this

I scribbled down all my thoughts in a text editor, with poor grammar and organization. Then I told an agent to review and fix up the document, but absolutely not take away from my voice. I cherish it.