Open vs. Closed AI: Why Governance, Not Model Type, Is the Real Question for CIOs

Recently, there was extensive news coverage of Jensen Huang’s letter on open-weight AI models. The coverage focused on who signed on to the document (and who didn’t). And while it was interesting to see the list of companies that pledged their support to the ideas he outlined, I stopped for a moment and thought about whether I should really care deeply about this discussion. And the answer is yes, and I think you should care deeply too, as I believe the future of your enterprise depends on it.

The open weight vs. closed weight isn’t a big topic on its own. After all, when an AI agent with access to your finance systems takes an action nobody authorized, "was the underlying model open-weight" is not a question your CFO, CEO or Board will ask. They will want to know who approved the agent, what it was allowed to touch, and who's accountable for what it did. These are all governance issues. Here are some thoughts on why you should be interested in this conversation from a governance perspective, and some of the advantages open weight models offer:

What happens to your governance when you switch AI vendors?

If you’re writing critical code for your company, building hundreds of agents on top of Anthropic’s Opus 5, you are pretty much locked in to continuing to use Opus 5. It’s not like you can take all your code, markdown files and agents and easily port them over to OpenAI’s GPT-5.6 Sol. These are closed models that have similarities, but also have important behavioral differences that make portability difficult. When you move, your governance doesn’t necessarily transfer with you – at least not easily. With open-weights, you can move your workloads from one vendor to another, leveraging your favorite version of AI in a similar environment. You should want more flexibility. This is a basic governance concern.

Data Sovereignty and residency: Who controls where your data and models run?

Where your data sits is becoming more important than ever. There are still more guidelines than rules, but we’re not far from regulatory bodies putting rules in place that are so difficult and onerous to achieve that it might make you want to question why you’re doing business in a particular country. So for many companies, the path forward is to self-host your own models, giving greater flexibility with governance options. You can host your own with models like Llama, DeepSeek and Kimi on your own GPUs in whatever data center and country you need to run your business. But if you’re a Microsoft CoPilot customer, you’re locked into the geographic regions where they’re offering the services. Companies like Microsoft are starting to offer data sovereignty and residency options, but managing that data within those boundaries can be extremely difficult, especially when you have older applications that may not be location-aware.

Should AI spending be Opex or Capex?

For a long time, CFOs wanted to turn as much as possible into Opex. That trend seems to be reversing. I recently spoke with a CEO of a mid-sized automobile technology company who did not want to be at the mercy of companies like Anthropic, OpenAI or Microsoft, spending gobs of variable Opex as they chase their dreams. The better play is to start hosting their own GPUs with Open weight models and leverage GAAP and state regulations to capitalize on this over the long term. Open weights and on-prem hosting are really the only way to make this shift to Capex and give you optionality on the financial governance side of the house.

Will model versioning help most companies?

Another upside of open-weight models and hosting your own version is that you know exactly how it works and when changes are introduced. If your auditor or regulator comes calling and asks you how decisions were made, you can prove behavioral stability with a fixed version of a model in lower environments all the way to production. You can tune your prompts and agents and take advantage of a “fixed in time” version of these models. 

What are the risks of open-weight models?

Of course, a downside of the open-weight models at this point is that most are Chinese. Open-weight models don’t guarantee complete transparency, and there are risks in using models that come from an untrusted part of the world. Your auditor or regulator may have extra questions, making the governance structure very important.  There are a lot of CIOs who’d like to be more aggressive in trying out and using these models, but they don’t have the staff or prioritization to test and secure them as they’d like at this point.  And it’s an expensive endeavor, with an IDC study finding that enterprises are allocating more than 15% of their budgets to AI security and governance.

It’s also important to know that, generally speaking, all open-weight models are at least one model generation behind the closed frontier class of models, such as Claude, GPT and Gemini. History tells us that those open-weight models will catch up, but when? 

Who’s accountable when an AI agent acts?

In closing, it’s great to see the technology leaders we know, love and depend on taking a stance in favor of open weight models. And when it comes to governance, you have a role to play too.  Get involved and give feedback to efforts like the Shared AI Findings Exchange (SAFE).  The Non-Human Identity Management Group says AI is scaling faster than security and governance teams can keep up, so we all must do our part to help drive security, costs and growth. A CIO must ask themselves, “How much capability am I willing to trade for these benefits, and is hosting my own model worth the price I’ll have to pay?” The choice isn’t only what capability you’ll trade, but also whether you’ll let a vendor’s architecture decide what you’re able to govern and account for.

‍
Todd Mazza is the former CTO of Factory Mutual Insurance Company. He’s also held leadership roles at Rockwell Automation, Workday, AECOM, Levi Strauss & Co, MGM Mirage and NBC Universal. Contact Todd: LinkedIn | @ToddMazzaCIO on X