In this post I document a little experiment I have been playing around with for a while that explores an absolutely minimal language model. I’m specifically interested in how small a language model can be made while still doing something. I got the idea while I was reading a post about GPT-2. A pre-COVID model, GPT-2 now seems minuscule in comparison to current frontier models. Its largest incarnation was trained on about 40 GB of text, had 1.5B parameters and used an embedding dimension of 1600. Nowadays, a GPT-2-like model can easily be trained in a consumer GPU with ~16-24 GB of VRAM.
If the GPT-2 family of models pales in comparison to current model dimensions, the model I present here today pales in comparison to GPT-2. Since it is extremely minimal, it can be trained on a single CPU core in a few seconds and needs very little RAM. It can have single-digit embedding dimensions, does not follow the transformer architecture, has no attention mechanism, and may not even use any hidden layers! It is a purely academic model, in the sense that it is absolutely simple but does nothing useful.
Improving the Milky Way visuals in Gaia Sky: particles to volume
By Toni Sagrista Selles
25 minute read
The Milky Way is an essential component in any 3D Universe software. Rendering a realistic (i.e. science-based) and visually appealing galaxy is paramount to the user experience, and to the cohesion of the simulated universe as a whole. Galaxies are made of gas and dust clouds, so they are volumetric in nature. In Gaia Sky we have been using particle-based representations for a long time, even though we have had the volumetric rendering infrastructure in place also for a while. This post describes the rendering technique we use in Gaia Sky 3.9.0 to render the volumetric Milky Way, and how we have constructed the dataset that enables it. Interested? Keep reading.
I opened it promptly and, indeed, it was not a prank. Finally, after years and years of bull**** from the Chrome team, they are shipping it. They are shipping a JPEG XL decoder.
JPEG XL in a nutshell
For those of you who don’t know, JPEG XL is a new image codec that includes most of, if not all, the features we’d want in the image codec of the future:
File size reduction by \(~20-60\%\) w.r.t JPEG.
Lossless JPEG transcoding.
Progressive decoding.
Wide gamut, HDR and 32-bit support.
Animations and transparency (alpha channel).
Shines with high-fidelity photographic images.
Fast-ish encoding and decoding.
Royalty-free and FOSS.
Support for super high-resolution images, of up to 1 terapixels ( \(2^{30}-1\) pixels per side).
What is the probability of getting an all-numbers Git hash prefix?
By Toni Sagrista Selles
3 minute read
Today, I was playing around with a new feature in Gaia Sky, and I created a new release from master. Immediately, I noticed something unusual in the generated package name:
packages-3.8.0.066962508
The 3.8.0 part is the release version, while the last 9 characters are the abbreviated Git commit hash.
Usually, that last part looks something like:
packages-3.8.0.a31f7c92e
This time, however, every single one of the hash characters is a number, 066962508. My first stupid reaction was thinking that something had gone wrong. However, git log confirmed the hash was fine.
feat: Handle gaiasky:// URLs forwarded from second instances and macOS open events. Part of #935.
- Scope dataset URL dedup to the startup path only, so re-sent URLs are always processed by the running instance.
- Register install4j StartupNotification listener for macOS open-URL events, with pending-URL queue for pre-UI arrival.
- Add --enable-native-access=ALL-UNNAMED to launcher VM parameters - Add compileOnly install4j-runtime dependency.
Obviously, I asked myself, “how likely is that”? So I did some digging.
Version 152 of the popular browser includes an option to enable JXL in Labs
By Toni Sagrista Selles
2 minute read
Yesterday, Firefox 152 was released. In the release notes, under the Firefox Labs section, it states:
The Firefox Labs section of the release notes of Firefox 152. If you see JPEG-XL works!, it means your browser is JPEG-XL capable. If you see JPG :(, it means your browser does not support it.
New open models provide amazing value for local inference
By Toni Sagrista Selles
4 minute read
Building on the previous post about how good current open TTS models are, this post focuses on local LLMs, and their explosion in quality in the recent years months. I have been using OpenRouter with, for the most part, DeepSeek v4 Flash. It is a very capable model, dirt-cheap (seems to be heavily subsidized, at $0.14/M tokens input and $0.28/M tokens output), and it works very well when it is free to browse the internet. I have also been tinkering with local models like the Qwen3.5 family, Gemma 3/4, Mistral, and more. They are good, I haven’t been able to use the versions that can run reasonably on consumer hardware for anything too productive. That is until now. The brand new Qwen3.6 models, particularly the 27B dense, and the 35B A3B MoE, have really impressed the community. They are trivial to run on common setups (e.g. 16GB RTX card, ~32-64GB RAM), and when boosted with agentic capabilities, they perform very well.
