Base Labs partners with Hugging Face on AI safety

Baseten's research arm, Base Labs, announced a collaborative effort on Wednesday to build robust safety evaluation and monitoring infrastructure for open-weight models. The partnership addresses a critical vulnerability in the open-source ecosystem: a technique called abliteration, which allows bad actors to strip away safety guardrails from existing models. Hugging Face currently hosts more than 6,000 of these abliterated models, highlighting the scale of the security challenge.

To counter this threat, the alliance plans to develop and publish transparent methods for training and monitoring open models. Instead of applying safety measures as an afterthought, the partners aim to integrate these protections directly into the training and deployment pipelines. Baseten argues that the inherent visibility of open-source software actually makes it easier to implement precise, actionable safety controls compared to proprietary, closed-source alternatives.

While the technical details of the integration remain undisclosed, the partners bring significant resources and specialized expertise to the project. Goodfire AI, which focuses on model interpretability to explain how AI systems make decisions, will likely spearhead the internal safety mechanisms. Goodfire recently secured a $150 million Series B funding round led by B Capital. Meanwhile, Baseten itself is highly capitalized, having raised a $1.5 billion Series F round in June that valued the inference provider at $13 billion.

For AI practitioners, this initiative promises a more secure foundation for deploying open-weight models in production. By establishing a standardized framework for built-in safety, developers can leverage the flexibility of open-source AI without the looming risk of easy exploitation or guardrail removal. Base Labs has issued an open call to the broader developer community to contribute to the evolving framework, aiming to foster a safer, collaborative ecosystem.