RC RANDOM CHAOS

machine-learning

2 posts

Rewriting weights live on 8GB of VRAM
Article

Rewriting weights live on 8GB of VRAM

mini-AGI trains a byte-level MoE model from scratch on an 8GB GPU, keeping weights on disk and paging experts in, learning as it reads.

The model kept no receipts
Article

The model kept no receipts

A mathematician presents evidence that OpenAI may have trained its Astra model on his private work on a conjecture it was then credited with solving.