Scratching an Itch
So there is a lot going on right now at Restless HQ, much of which will require separate posts. RestlessIDE (the soon-to-be best web-based development environment in the world!) is still going strong and recently got some new updates, including a shiny new base image for the code editor. If it seems like I haven’t been posting much recently, it’s because client work has been kicking my butt, which is a great problem to have.
What I want to talk about today is actually something different.
Interlude
A few months back I finally splurged on my first brand-new computer in probably a dozen years, a Framework Desktop. I know what you’re thinking. “Mike, that’s ridiculous. You have 5 computers on your desk right now, and have a 10 machine KVM switch sitting as the top item in your Amazon Saved for Later queue as we speak because you’ve outgrown the current 4 machine model and it really bothers you having to connect via Splashtop or VNC like a cretin.” And that is all true.
What you should know is that my love of collecting computers is only bested by my stinginess. I buy everything used, and don’t mind waiting for a deal, even if it takes years.
The Framework Desktop was different.
I saw all the Youtube videos you did. AMD Ryzen AI Max 395+ (Plus!!!) with 128GB of memory that could be shared between the CPU and built-in GPU. For running AI models locally. Big ones that can do Real Work. It’s literally the only piece of technology I’ve bought that is worth more today than when I bought it, seeing as how I got it 2 months before the Memory Apocalypse.
In the last few months I’ve been putting it through its paces, trying all of the latest Qwen and Gemma models and pumping out everything from Godot code for experimental games to Urdu translations for future app ideas.
A Bump in the Road
One of the tasks I wanted ol’ Framey (I need to name him soon. I’m listening to recommendations if you have any.) to take a look at was customizing themes for Erstwhile Framework, the single-page application framework I made for RestlessIDE. Now, the thing about single-page applications is that they require browsers with Javascript enabled to work properly. I have been using KiloCode (a VS Code plugin that can connect to local LLMs for development tasks) to interact with Framey, but when I gave it a URL to check some rendering issue, it was at a loss. It would do a simple fetch, see the skeleton index.html of the app, and tell me it has no idea what I’m talking about.
I don’t need it to see the page as it looks when it first loads. I need it to see the page after all of the Javascript has run to populate my application.
That’s the tricky part. The world of programmatic browsers (that is, browsers that can be scripted to do things via code rather than by user input) is pretty large, but it it’s also complicated. Many of the tools that let you do it have evolved to become full-fledged unit testing frameworks that have whole scripting languages of their own for walking through your apps automatically. I need something super simple, almost like a wget or curl but with the ability to a) run Javacript and b) wait a little bit before returning so the page has a chance to render.
So I made something: Simple SPA Request.
As a node module, it can be installed via NPM:
npm install -g simple-spa-request
Once installed, you can point it to a URL and it will return the DOM of the page after a certain timeout, the default being 3000ms.
simple-spa-request -t 5000 -f html https://erstwhilefw.org
It has some other tricks up its sleeve, though. It can also produce Markdown or JSON output, which lets you include additional information about your request:
- A list of all of the network requests made by the page.
- The output to the console log.
- A list of all of the stylesheets and the rules they define.
All of this is written straight to stdout, and you can exclude any of those that you don’t need to make the output shorter.
There are also some more experimental features that I’ve added but not really tested yet.
Javascript Execution
Simple SPA Request lets you pass in some inline Javascript via the command line if you need to, say, click a button on your app after it renders to see what the output is. Or if you need to inspect the application and log something out to the console.
simple-spa-request -j "document.getElementById('button').click()" -w 1000 https://restlessdev.com
If you use this feature you can also set a second timeout with -w to let your script have a chance to finish before printing out the output.
HTTP Headers
Simple SPA Request lets you set headers when making your request to let you pass things like authentication tokens.
simple-spa-request -h "Authorization: Bearer token123" https://restlesside.com
Different Types of Requests
While this is probably less useful for SPAs, you can also use Simple SPA Request to send POST requests with a body to endpoints and see the output from that.
simple-spa-request -m POST -b '{"key": "value"}' https://example.com/api
The goal is to create a scriptable browser that you can use as part of your chain, whether it’s for a bash script or for input into an LLM.
Conclusion
So this was a fun little thing to make, and hopefully other people find it useful. 99% of the work is done by the Playwright library; my task was just to try to parameterize my own problem, and then call the library in a way that solved it.
Speaking of which, the Playwright CLI looks pretty neat, and is definitely worth a look if you want to be able to just script a browser from the command line. It seemed a bit like overkill for my needs at the moment, but needs have a way of changing.
Til next time.