Cleaning Up AI Slop with the Help of AI, Piece by Piece

Since August, Fran Reyes and I have been working on our first AI slop cleanup project.

The original prototype was written in vanilla JavaScript embedded in plain PHP scripts, alongside the CSS, HTML, and SQL.

We are using AI to transform the embedded frontend code^[1] into a SPA written in TypeScript, using React and the reffects framework.

We are transforming the application screen by screen. We’ve transformed two screens so far. Our client prioritized the screens from most to least valuable, and we’re following that order.

We transform each screen in several steps, which we’ve previously explored in independent spikes:

1. Transform the vanilla JS of the screen into a React SPA without changing the backend.
2. Transform the global CSS that styles the screen into CSS Modules.
3. Change the React SPA so that it uses the reffects framework.

We perform all these steps in a separate branch so we can preserve the original prototype and avoid altering its behaviour.

After those three transformations, we have a JavaScript SPA using React and the reffects framework that includes one screen.

The next step is to transform the JavaScript code into TypeScript, but that’s where things get more complicated.

Since reffects packages are written in JavaScript, their APIs don’t have TypeScript types. To use reffects from TypeScript, we first had to provide local TypeScript declarations for all its packages: reffects, reffects-store, and reffects-batteries.

Those type declarations describe the framework’s events, handlers, effects and coeffects, store operations, React subscriptions, and built-in coeffects and effects. These declarations let TypeScript understand the library APIs and give the app typed integration points for its own state and components.

Later, we added typed wrappers around the store and its state effects and coeffects. These wrappers let TypeScript detect invalid state paths at compile time, rather than letting those errors surface at runtime. This shortens the feedback cycle considerably, in fact, the IDE can flag an invalid path as we write it. We’ll talk about how this works in a future post.

We generated the type declarations incrementally using AI agents while building the seed of a walking skeleton and adding the build configuration with Vite and the test configuration with Vitest. That seed consisted of porting one of the reffects examples, with-react-todos, to TypeScript.

We also had to write TypeScript interfaces and implementations for clients injected into built-in reffects effects and coeffects whose implementation reffects leaves open, such as the http effect.

Once we had the reffects example and its tests running in TypeScript, we started wiring in the first screen, one JavaScript component at a time. We pasted each component into the project and used AI agents to convert it to TypeScript.

When a component’s event handlers require a new kind of side effect, we add the necessary custom effects and coeffects.

This piecemeal growth has allowed us to integrate the AI-generated code in small batches.

But what about validation? How can we know that the behaviour of the prototype has survived these AI-generated transformations?

To address this concern, we used the sampling + golden master (GM) technique.

We recorded golden master data for key user journeys and important computations in the prototype, and used it to write broad integration tests for those scenarios in the TypeScript app. These tests allow us to verify that the prototype’s behaviour has been preserved so far. As the migration progresses, they will also help us catch regressions as we bring more components into the TypeScript app.

Because these tests are broad, they give us room to refactor some interfaces generated by the AI agents that aren’t as good as we’d like. Since the tests aren’t coupled to those interfaces, we can change them at a lower cost. We are incrementally complementing the broad tests with unit tests at the seams provided by reffects’ functional architecture.

This approach to the AI-assisted rewrite has allowed us to maintain a good pace without producing slop. We only demo code that has already been validated, while components that are still works in progress remain hidden.

It’s not all smooth sailing, though. The AI isn’t doing as well at transforming* the global CSS of the prototype into CSS modules. It tends to take a shortcut by using :global inside the modules, which effectively gives us global CSS again, only this time distributed across modules (like the “distributed monolith” version of CSS). This concerns me, and it’s technical debt that we’ll have to address. We also need to steer the agents to avoid this shortcut as we transform the next screens.

So, this is how we’re approaching a rewrite of an AI-generated prototype on a tight schedule: using AI agents to our advantage while trying to avoid generating more slop.

All in all, we’re enjoying this project and learning a lot. We hope to find time to write more soon about some of the other interesting things we’re learning.

Acknowledgements.

I’d like to thank Fernando Aparicio and Fran Reyes for giving me feedback about this post.

Finally, I’d also like to thank cottonbro studio for the photo.

Notes.

[1] We’re taking a more traditional rewrite approach with the backend: incrementally rebuilding instead of incrementally transforming. One important reason for this difference is that, on the frontend, we wanted to preserve users’ familiarity with the existing GUI as much as possible.