Training an LLM has become such a common practice now that people are doing it on pre-1930s text just for a laugh. There are absolute benefits to having a massive compute cluster, a whole team of talented staff and a big pile of money - but there are limits to
