CISOOnline

Rolling the cyber dice with open-source and open-weight AI models

But if we look at what the research already does. In Winter Soldier, a team poisoned under 0.005% of pre-training tokens, 64 documents and made a model learn a hidden prompt-and-response pair that never appears in the training data at all. Auditing the corpus would not find it, because it was never written down. While this is not yet a production-scale threat, it demonstrates that the technique works, waiting for someone to make it stealthy.

That is the part worth planning around. The detection asymmetry runs against us: these behaviors survive safety training, and searching the weights for them costs more compute than most of us will spend. We are choosing suppliers now for something we would not be able to see if it arrived. If the U.S. does not have enough strong open models, we are going to rely on Chinese ones, and that is a bet placed under exactly that uncertainty.

Meta recently launched Muse Glimmer, a 30-billion-parameter model under an Apache 2.0 license, and plans to open the weights of its flagship, Muse Spark 1.2. Note that most of the coverage called Glimmer open source. It is open-weight: Meta released the parameters, not the training data or training code. That slippage in the trade press is the same one that shows up in our architecture reviews, and it is worth catching in both places. The move looks aimed at OpenAI and Anthropic, and at bolstering U.S. models against the influx of Chinese releases.



Source link