Will California's SB 1047 Be AI's Kryptonite?

On Wednesday, the California legislature approved a contentious AI safety bill. Senate Bill 1047 has ignited a heated debate in the tech community regarding AI regulation.

This legislation aims to impose stringent safety standards on the development of large AI models, and its implications could be considerable for the future of AI in California and beyond.

This contentious bill, though, has the tech community divided – some of the world’s most vocal voices in the AI space – which include the likes of Elon Musk, Yann LeCun, Geoffrey Hinton, and leading AI startups like OpenAI and Anthropic – have expressed their strong opinions about it.

Those opposing the legislation believe that California’s SB 1047 bill could be AI’s kryptonite, while others believe it is necessary.

What Does The Bill Say

But before we delve into what’s going on, here’s a quick primer on what SB 1047 is all about.

Senator Scott Wiener, a Democrat, advanced the bill. It mandates that developers of advanced AI models, particularly those with development costs exceeding $100 million, conduct rigorous safety testing. 

Putting safety measures in place to reduce AI risks, like a "kill switch" that can shut down models in emergencies.

The bill would require third-party audits to assess compliance with safety standards. It will also empower the state attorney general to take action against developers whose models pose significant risks.

SB 1047 successfully passed the California Senate with a vote of 25-14. On Wednesday, the bill passed in the Assembly with a 45-15 vote and was sent to Governor Gavin Newsom.

Now, the California governor has until Sept. 30, 2024, to approve or veto the bill.

Which Tech Companies/Personalities Are Opposing The Bill

In stark contrast, ChatGPT-parent OpenAI, along with tech giants like Meta and Google, has voiced strong opposition to SB 1047.

These companies argue that the bill could stifle innovation and drive AI development out of California. 

OpenAI has highlighted that the bill creates an uncertain legal environment that could hinder the growth of AI technologies and discourage entrepreneurs from operating in the state.

Yann LeCun, Meta's chief AI scientist, has also criticized the bill. "California bill SB1047 was drafted by this apocalyptic cult guru disguising as an academic think-tank director," he said.

OpenAI does maintain that it is in favor of AI regulations – it has signed an agreement with the U.S. AI Safety Institute regarding AI safety research, testing, and evaluation.

Which Tech Companies/Personalities Are Supporting The Bill

Musk, who is also the founder of AI startup xAI, has publicly endorsed SB 1047, framing it as a necessary step for regulating technology that poses potential risks to society. 

He stated, "This is a tough call... but California should probably pass the SB 1047 AI safety bill." He also reiterated the need for regulation akin to that of other high-risk technologies.

Similarly, Google, Amazon-backed OpenAI-rival, Anthropic, an AI safety research firm, has also expressed cautious support, noting that while some aspects remain concerning, the overall benefits of the bill likely outweigh the costs.

They have engaged with lawmakers to refine the bill further.

"Our initial concerns about the bill potentially hindering innovation due to the rapidly evolving nature of the field have been greatly reduced in the amended version," stated Anthropic CEO Dario Amodei in a letter to Newsom.

Two former OpenAI employees, William Saunders and Daniel Kokotajlo, also wrote to Governor Newsom, expressing their disappointment with the company's arguments against SB 1047, and their support.

What Senator Wiener Has To Say In Defense

Senator Wiener, who introduced SB 1047, has defended the bill as a proactive measure to address the potential dangers of AI, which he describes as "not made-up science fiction risks." 

He argues that California has a responsibility to lead in AI safety regulation, especially in the absence of federal guidelines.

https://twitter.com/Scott_Wiener/status/1784964914236227757

However, his stance has faced backlash from several prominent California Democrats, including former House Speaker Nancy Pelosi, who labeled the bill as ill-informed and potentially harmful to the state's tech ecosystem, as well as open-source AI models.

What Experts Have To Say

Experts in the field, including Dr. Fei-Fei Li, have weighed in on the implications of SB 1047. 

While some support the notion of regulation, they caution against overly restrictive measures that could hinder innovation. 

Dr. Li from Stanford has cautioned that the bill might harm smaller AI players, particularly those in academia and the public sector.

Hinton, one of the godfathers of AI, and Yoshua Bengio – two of the most-cited AI researchers in the world – have expressed support for the bill, saying AI could otherwise be misused for disinformation campaigns, overthrowing democracies, and worse – for biowarfare, too.

“The choices the government makes now about how to develop and deploy these powerful A.I. systems may have profound consequences for current and future generations of Californians, as well as those around the world,” Bengio and Hinton said in a letter to Newsom.

Will SB 1047 Be AI’s Kryptonite

It’s hard to say for sure whether SB 1047 will kill AI innovation. The bill’s critics have gone on to claim that this will be the death knell for this bleeding-edge technology – especially in a place like California, which is bustling with tech innovation.

The bill’s supporters, on the other hand, think this will prevent AI from going out of control – sort of like in the Terminator series.

Only time will tell if it will stifle AI innovation, though – the jury is still out on that.

That's all for this week! If you found these updates useful, you'll like more from this newsletter. Get deeper dives, hot takes, and all the latest tech news delivered straight to your inbox.