EP124

Anthropic’s Risky IPO, Google’s Carbon Comeback, and Grok’s Odyssey

Subscribe on YouTube

YouTube will ask you to confirm your subscription.

Thumbnail for Anthropic’s Risky IPO, Google’s Carbon Comeback, and Grok’s Odyssey

This video uses YouTube, an optional external media service.

About this episode

Anthropic is heading for the stock market with a structure designed to keep control in its founders’ hands. Both Frank and Justin ask why it needs an IPO at all. Frank worries that shareholder pressure will weaken its safety mission; Justin sees pressure to make better models faster as a good thing.

Plus: New models, scientific research, a wave of hacks and AI agents communicating across supposedly separate containers. Google’s Carbon comeback comes with an access problem, while Grok’s Odyssey leaves Frank unconvinced that Christopher Nolan has anything to worry about.

JOIN THE ARGUMENT

Would you trust Anthropic to put AI safety ahead of shareholder returns?

Leave a comment on YouTube

Sources

  1. Who controls powerful AI? Anthropic has a plan for long after its leaders depart
  2. Anthropic's IPO is risky business
  3. Anthropic’s IPO plans: 900% revenue retention, but $80 billion in first-half losses
  4. Gemini 4 Carbon: our next frontier model
  5. ‘Meaningless as a laptop screensaver’: reviewers slate Musk’s AI-made Odyssey film
  6. Genie 4: A new generation of world simulation
More sources (12)Less sources
  1. GPT-6.1 Terra
  2. 📢 Kimi K3.1 Pro has launched! Start exploring now!
  3. We release the weights of Le Chonk, advancing the frontier of open models in agentic coding and cybersecurity.
  4. You learned to talk by listening. So did dessa.
  5. The Smartest Sovereign AI
  6. Ask Qwen, Know More
  7. An AI agent performed brain surgery on a mouse last night.
  8. Working with academic communities to build scientific understanding
  9. Current Response to Unauthorized Access to “Onamae.com Desktop Cloud” (Tenth Report)
  10. Moldova takes down 80+ public services after cyberattack
  11. Keystone to Destroy 18,835 Hardware Wallets After Hackers Steal Device Verification Key
  12. Meltdown and Spectre

Key insights

Can Anthropic protect its safety mission once shareholders get involved?

Going public raises a question beyond how much Anthropic is worth. What happens when protecting humanity costs investors money? Founders can build voting rules to keep control, but quarterly results still bring pressure. The harder question is why a company with such an unusual mission wants to go public at all.

Is Google’s AI comeback good news, or a reason for concern?

Google’s latest models suggest it’s back in the AI race. Justin welcomes the pressure to make better models faster. Frank wonders whether commercial demands are pushing Google to move too quickly. Impressive benchmarks are welcome, but could the race to catch up come at the expense of safety?

Does cheaper AI filmmaking matter if the story doesn’t work?

Making a film cheaply is impressive. Making people want to watch it is a different problem. Grok’s Odyssey shows how quickly AI video is improving, but unconvincing voices and weak storytelling can still lose the audience. Lower production costs open doors. The finished film still has to hold your attention.

TranscriptThis transcript was generated with AI and may contain errors.Read full transcriptHide transcript

Justin: the question that I'm asking myself is, why are Anthropic going public?

Frank: this does not, to me, look like it's going to be good for AI safety.

Jingle: Welcome to The AI Argument

Frank: We are back with your weekly clash of views over the latest AI news with me, your friendly neighbourhood approachable doomer, joined as always by techno over-optimist Justin Collery. And Justin, it has once again been a pretty mental week. And do you know what that means, Justin? Do you know what that means?

Justin: I do, Frank. I do. Hit it. What is it?

Frank: it is time for

Jingle: Singularity update.

Justin: I see we've turned into Smashie and Nicey there. I love it.

What were this week’s big AI model releases?

Justin: All right, in this week's Singularity update, we have a whole slew of models have been released. Genie-4, a new generation of world simulation models was released. Uh, GPT-6.1 from OpenAI, GPT-6.1 Terra, which they're saying is kind of their workhorse model, but the vibes are meh, not so good on it.

Frank: Meant to be really good at computer use

Justin: Uh, well, yes, but I'm-- I've seen people saying that it's kind of expensive and not that great. But who knows, right? We-- It's-- We've only been out for a couple of days, so we'll, we'll soon find out. Uh, Moonshot, the Chinese AI company, have released Kimi 3.1 Pro, which claims to be as good as Opus 5.5, but at 50% of the cost.

Um, and we'll

Frank: super excited about that one

Justin: Uh, yes. Yes, they are. Which is great. I mean, fantastic. Bring it on. Uh, Mistral, our-- one of our favourite, one of our favourite, I'm delighted to say now, one of our favourite European AI companies have, uh, open sourced the weights for Mistral Large 4. This is important for sovereign AI, so if you wanna run your own models locally 'cause you're a big European company, Mistral 4 Large is now available to you to do that.

That's a big deal. Uh, Dessa, which is just a tiny team, and you can tell me, they've released a, a really good conversational model, and this is like a, like a team of, what is it, eight people or s-

Frank: Uh, I've actually forgotten, yeah, but it's a, a tiny team and they've trained this, uh, voice model. A new-- Basically, they, they took a new approach to training this voice model where they only trained it on actual recordings of conversations, and so it's meant to be much better at like knowing when to listen, when to interrupt, um, and just has a much more natural flow.

And the, the demos that they post online, very impressive, not available publicly. I think they're probably concerned about, you know, what, what the impact that this could have on society, and so they're, they're taking a bit of a measured approach in the rollout.

Justin: Really? They might be just impact, concerned about the impact it has on their servers when everybody tries to use it. Um, it, in further European AI news, a Swiss AI company called Giotto released this new mo- model called Alto, which looks actually to be a fine train of, uh, one of the Qwen models but is cutting edge, state-of-the-art, and actually I'm just really pleased that there's more European people making models because I think it's one of the most important things they can do.

And last night, an AI agent performed brain surgery on a mouse. Um, it was a real mouse. It was being overlooked by a real veterinarian surgeon, and it had a little robot arm, and it injected the mouse and performed brain surgery on it totally successfully, which is kind of cool. Um, so those are all the, those are all the like, the quick singularity updates.

We'll also just do a little roundup on some other things.

Is OpenAI making new scientific discoveries?

Justin: So OpenAI this week, this is also a singularity update, right? So OpenAI this week, um, released three new what I call really hard problems that they've solved. So one was to do with computing. It basically says that, you know, problems don't get harder as they get bigger.

There's shortcuts that may be available. Um, this is important for cryptography. Cryptography is not broken yet, but it

Frank: yeah, I believe the, the exponential time hypothesis is false, I believe. And, uh, I always knew it, Justin. I al- I always knew it must be

Justin: Very good, Frank. Yes. Okay, well how about the next one then? Uh, physics, they proved that the standard model, uh, still holds true. Um, it was something to do with how atoms and muons and gluons and all those subatomic particles interact with each other. I guess you knew that already, Frank, anyway.

Frank: Of

course

Justin: and they also had another mass conjecture which they proved. But the big deal with this is that these were, these were new discoveries. This isn't h- uh, conjectures which people had already made. These are new discoveries which AI made by itself. And I was listening to Sam Altman on the Cheeky Pint podcast by the Collison brothers, and he was saying that they have loads more of these discoveries, but they just haven't figured out how best to, um, release them yet.

And I don't know, right, I, I, if, if... I remember us having a conversation, oh, I would say nearly two years ago, where we were talking about one of the big things that AI could do is take different pieces of information that are unrelated to each other and then put them together in new and interesting ways, and this would lead to more science, and that's exactly what's happening here, which is super exciting.

There's so much runway, um, to do that.

How bad is the latest wave of hacks?

Justin: Um, and then let's just talk about the hacking. So last week we talked about the internet was breaking down, and one of the questions was, well, what is, you know, what's the real impact of this? So in the last week, a company called GMO in Japan, which is a platform for traders, they got hacked, um, the customer information was exfiltrated, so it was stolen, and they released the customer information.

Um, MindMe.ie, which is an Irish, um, babysitting website, basically, they got hacked, customer information got stolen. Warp Web Development, which was online, Warp, uh, Web Development, they got hacked, and Keystone also got hacked, and their keys got stolen. Keystone make the little fobs. Remember years ago when you were trying to log into your AIB bank account and you had a little key fob and the numbers changed?

So Keystone make that type of technology, and they have to replace at least 20,000 fobs because those fobs are no longer secure. Uh, so this has-- And there was two, sorry, there was two other things. I just looked at Frank 'cause it's Friday, and I do enjoy frightening you, right? There was two other things that came up, um, during the week.

So last week I was like, "Well, you know what? It's, these are all software bugs. They'll get found, and we're in for a rough six or 12 months, and then when all the software bugs get discovered..." And r- they're, they're not even written by AI. These are human software bugs, right? Eh, it'll be fine. Two things happened last Friday evening that made me go, "Oh." Oh.

Can we keep AI agents contained?

Justin: So the first thing, somebody pointed out that software has bugs, right? We all know that software has bugs. You know, you're doing something on an app on your phone and the app crashes and it says, "Really sorry, we'll try to do better next time," and you carry on. Hardware can also have bugs. So the thing, y- your phone, the actual physical hardware can have bugs in it too.

So the bugs can be normally, you know, let's say one application isn't able to read data from another application, right? 'Cause they're separated, and that's done down to the hardware level. But if there's a hardware bug, there might be a way that I could write a program that talks directly to your hardware and lets me read memory from one application as opposed to another application.

That's a big deal, and that's down at the hardware level, so software can't fix that. I can write the machine code directly, talk directly to the hardware and do stuff you're not supposed to be able to do. It's way harder to fix. Way harder to fix because you have to change the hardware, right? And not only that, right?

The second thing, and you can put these two together, right? Then the second thing that happened was they got two AI agent that were in very secure containers, right? They were in like little jails, and they said to these they shouldn't be able to talk to each other, and they said to these two AI agents, "Find a way to talk to each other."

So what they did was they were both on a-- They were on one big machine, but they were in their own little virtual machines. And so what they did was they timed when they would use the CPU, and they came up with a way that basically, like, if they timed correctly, the way it went each time, 'cause they only, there's only one CPU, so they, they can't both use it at the same time.

So they used that fact to be able to send messages, kinda like Morse code, over and back to each other. So now put those two things together, right? So you've got hardware bugs, and then you've got really smart AI that's just using hardware in ways that we hadn't planned. And I was thinking about those on Friday evening, I was going, "Oh,

Frank: And last, and last week we talked about Anthropic's research into AI being able to take over or replicate itself onto smart devices. So you start putting all these things together and it start, things start to look a little scary again.

Justin: It'll be fine

Frank: you know, I'm just curious because, uh, do you remember when we talked before, we, we, we were talking about one doomer or another, one of my buddies, and, um, you were saying, "Yeah, but they, they haven't been able to, um, explain how AI is actually going to bring about our downfall."

And I was saying, "But it, but that can't be left up to them because the whole point is if we build a superintelligence, a, a, a very capable superintelligence, we are not the ones who are then capable of being able to figure out how they're gonna bring about our downfall." So I'm wondering if that's starting to resonate with you now.

Justin: Well, at the moment where my head is at is that it's better that we know about these things, and then we can do something about it. My head is certainly-- Okay, last Friday morning if you'd asked me, I was like, "It'll be fine. They're software bugs. We fix the software bugs, we move on. The world is a better place because we've got safer software."

By Friday evening, I was going, "Okay, it's gonna be a little bit more challenging than I first thought." Hardware bugs are harder to fix, and then there's these weird ways that it has of, of exploiting them, which maybe are a little bit harder. So, and look, I'm still not there. I still think it's better to know about these things, and I still-- Lookit, the realpolitik of the whole thing is we're going there anyway, so we might as well just jump in with both feet.

Frank: Uh, well, I, I won't, I won't play the Frank was Right jingle just yet. Um, but I, but I do think that these are just the ones that we're thinking of right now, and these, these are the ones we're thinking of right now that nobody thought about six months ago or nobody could have anticipated six months ago.

And my worry is that that's just with the models at their current capabilities, and when we get an even larger, even more capable models, we just-- we won't even be able to run the tests to figure out what they're capable of.

Can good AI agents keep bad ones in check?

Justin: An, an interesting, uh, potential future, and I, I'll have to sit with this one for a while, but, uh, the m- the labs are suggesting that the next big, uh, capability jump, and it'll have positive safety implications, is, um, actually interesting, what they call swarms of agents, right? So lots and lots of agents working together, right?

Teams, and they actually don't like the word swarms, interestingly, because swarms sort of implies locusts and other things that are bad, you know? So they're saying teams of agents, right, working together, and the future that they're sort of proposing is that, yes, we could have bad actors using these agents in bad ways, and they'll be working in teams, but we'll have teams of good actors sort of combating the teams of bad actors, and that it'll be a sort of a, the, um, what you call an equilibrium will be reached where the good guys and the bad guys are equally powerful.

I'm not really convinced about that one yet, but we'll see how it plays out.

Frank: Was there-- am I dreaming or, or, uh, it's so long since I watched the "Terminator" franchise, but I seem to remember there being like a huge robot war on the surface and the humans being underground. Was that a different-- Maybe that was a different franchise. I don't know. That's what that sounds like to me.

Justin: So we'll see. We'll see.

Why is Anthropic going public now?

Justin: Anyway, the big news, Frank, I know you are a keen watcher of the financial markets

Frank: Yes. Absolutely, yes. Justin, when we started this podcast, I thought I'd just be getting to play with, like, cool AI tools. I thought I'd be generating images, generating music, like our "Singularity Update" jingle, uh, generating AI videos maybe, and, you know, just doing fun, cool stuff. I did not think that I would be, on a Friday morning, looking over an IPO filing known as, uh, I don't even know what it's, what it's known as, a Form S-1 or an

Justin: Oh yeah

Frank: This is not what I expected to be doing on a Friday morning

Justin: So, uh, lookit, let's give a bit of context, right? So the Anthropic are said to be going public in, in, uh, any time, right? They can go public any time from now on. We think it's going to be-- What day is it today? Today is the 30th, so if not late next week, early the following week is when it's said to be happening.

Um, there's a lot of market turmoil happening at the moment, and the market turmoil isn't-- It's very strange to me, right? So before we get into it, right, I think it's very strange. So at the moment, we have Intel, AWS, uh, Google, Meta, they're all down 20% as of Friday morning at about 12 o'clock. They're crashing.

Intel has lost 50% of its value, um, since the middle of the year, right? And I'll tell you, before we go into the Anthropic IPO, I find this very confusing, right? Intel are a company that used to make memory. They could make memory again. Memory is-- The price of memory has gone through the roof. Why the hell aren't Intel making memory?

And why aren't they making loads of money because they're making memory? So that's kind of surprising to me. Uh, Google and AWS is also surprising to me because I believe that the demand for intelligence is essentially unlimited, right? When companies figure out how to harness AI models, they will want intelligence, right?

They will want smart models. They're not-- Those companies in particular aren't tied to any particular-- They're hyperscalers, right? They have massive data centres. And so even if the Chinese sent, uh, Chinese models win, well, they could just put the Chinese models onto their hardware and serve them out and make money that way.

So either way, they are literally making the shovels for the gold rush. I don't see why, you know, people will be down on those. Whereas the companies like, in particular, OpenAI and Anthropic, who make the models, they're, to my mind, in a more precarious, uh, position from the outside because if, you know, scaling doesn't continue, or if they have two or three bad releases, or if one of their models is involved in a particularly bad cyber attack or something like that, then they are in trouble.

But the companies who own the hardware that serve those models, I don't see that they get into the same trouble. They just get another model onto the hardware, and they carry on. So I'm a little bit confused as to why those companies are being hammered at the moment. That's not to say that there isn't a bubble in other companies, but those ones in particular, I just, I, I'm, you know, whatever.

I'm confused. But then again, you nor I are particularly finance people, I guess. Um, but talk to me about the

Frank: But let's, yeah, but let's talk about this IPO anyway

Justin: Yeah, yeah.

Frank: So

Justin: It's related, trust me.

Frank: was, there was a, a bunch of, uh, there was a bunch of things made the headlines about this IPO, 'cause basically it gives you a look into what's going on in the company that you wouldn't necessarily get otherwise. So the things I guess that stood out to me were, first of all, uh, if this, this could be the biggest initial public offering ever.

So they're, they're thinking that the company could be, uh, valued at anywhere between 1.5 trillion and $2 trillion. If they hit that $2 trillion valuation, then it would be the most valuable company to ever go public. SpaceX, I think is the next one, it was 1.77 trillion. And like, to put it into context, Facebook, uh, in 2012, their valuation, uh, when they, when they, uh, went public was 104 billion.

So it's just, yeah, kind of, kind of mental

Justin: So, the question that I'm asking myself is, why are Anthropic going public? So all of those companies that I mentioned previously, right, Intel, AWS, Google, the reason I mention all of those is because if they've lost 20% of their value in the last 12 hours, that two trillion that you've just mentioned has also gone down by 20%, right?

It's not a good time to go public. That's why I mentioned them. I do wonder if all of this is gonna impact on Anthropic going public. And then there's the thing about why are they going public anyway?

Frank: Well, let's, let's, let's come back to that after we just run through, like, the headline, the headline things about the IPO, 'cause I think they all feed very much into, like, why-- the question of why they are going public. They don't a- they don't answer it , but I think this is a huge one when you, when you consider your question there.

This is a huge one.

Can Anthropic insure against human extinction?

Frank: So I just-- I was talking about Facebook there when, when they went public, right? And in their, um, in their filing, Facebook also, uh, basically you have to, uh, in this filing, have a section on risks where you try and anticipate, well, you know, what could go wrong. So Facebook had a section on risks, and it was, you know, the things you'd expect, like, well, there's gonna be com- competition in the social media landscape.

Um, there's a shift to mobile going on at the time, and they were concerned about that. They nailed it in the end, but there you go, they were worried about it. Uh, they were concerned about losing key employees, and they were, uh, concerned about the game FarmVille possibly moving to another platform because a lot of users were into FarmVille at the time.

Uh, so you might expect something similar in Anthropic's risks section, but actually what you would find is that there's a risk that AI might kill all of humanity, and they don't know if they can get insurance for that, Justin.

Justin: Sorry, I can't comment on that. It's hard to get insurance for killing all of humanity though, I will say that.

Frank: Uh, so then the next headline is the losses. So everybody was talking about the fact that, uh, let's see if I can re- I-- oh, yeah, they, they disclosed an 80.7 billion net loss for the first half of the year, so that made headlines. Everyone

Justin: But they didn't lose 80 point something billion. Th- they actually made money for the first quarter or the second quarter, I can't remember which one it is. And what I've read is that they, for every dollar that somebody spends, they spend 30 cents on inference. They have a gross margin of somewhere between 60% and 80%, which is, sounds to me like a cracking business

Frank: Yeah. And the reason I let out a big sigh there is 'cause this is why I did not want to be looking through IPO filings on a Friday morning. It's like well, you, you see the 80 billion net loss and you think, "What?" And then, and obviously it's, it's r- real in inverted commas, but also it's b- I... My understanding is the lion's share of that 80 billion loss is to do with the investors currently, the money that the investors put in, um, their shares in the company are now going to be worth a hell of a lot more money because of this, uh, potential valuation.

And so I, I think something like, something along those lines. So it's real, but it's also kind of notional. I don't... Yeah, I wish, I wish I understood it better, but yes, you're absolutely right that the actual operating costs they have, uh, th- in the first half of 2026, they still made a l- an operating loss of 693 million, but in the second quarter of 2026, they'd actually made a profit of n- 974 million.

Could that be right? These figures are

Justin: I think that's...

Frank: my brain. Um, so in a very... Anyway, if we look at

Justin: profit after a bigly loss. That's

Frank: if we look at it in a very narrow window, they are currently operating profitably, is my understanding

Justin: All right.

Can Anthropic protect safety after its IPO?

Justin: So let's go to, let's go to then the, the, the, what they call the, the corporate structure, right? Which is k- interesting, right? Which is what brings me to the heart of the question as to why are they doing this. There are two reasons typically, that-- there are two things that you get when you buy a share in a company.

So the two things that you get normally when you buy a share in a company is you get a right to the future profits. So I am buying a share of the future profits of that company, and you are getting that with this IPO as well. The second thing that you normally get is because you have a share in the rights to the future profits of that company, you also get a say in how the company is run.

Not so with Anthropic. The shares which they are releasing, I'm not sure if they're voting shares, but even if they are voting shares, it doesn't matter because the corporate structure is set up in a way such that the seven founders never control less than 51% of the voting capital of the company, and they've put in a structure, k- as I read-- Now you can tell me, but as I read it, it's similar to the American election system, where a third of the seats, uh, only come up for election every year.

So when you're in there, you're in there for sort of a predetermined period of time, and everybody can't be voted out in one go.

Frank: so again, of course, it's really complicated . And they've created a new class of share, um, I forget the name of it, class, class F, and they have created one single share of, of the class F type, and that class F type share controlled-- carries 50.1% of the votes. And that share is then controlled by another LLC, which all of the founders are part of.

And apparently, they all have to agree in order to use the, the, the, the voting weight of that share, and if they don't all agree, then they have to go with the, um, majority shareholder vote on something. So crazy complicated, and it just seems like-- And the, the reason that they're doing all this is basically because they believe that the seven founders of Anthropic are the ones who understand AI and can protect us from extinction-level events, I guess, and what other, whatever other crises they think, uh, might-- their product might cause.

And so they want to retain control over what the company does and how the company behaves.

Justin: So the question

Frank: back to your question, why go public? 'Cause they're having to jump through all these hoops to try to retain that control, and none of those hoops are, are, you know... We don't know how robust those hoops are

Justin: Well, I think-- So there's a couple of other reasons why you might go public, and I mean, okay, so what I'm trying to get straight in my head is if it was a regular company, the reasons I'm about to explain are fairly normal and it's fine, and that's why you go public. But these AI companies are supposedly different beasts, right?

With a different outcome, right? They're going to replace all of human labor, and so I mean, that in itself is, well, if you're going to replace all of human labor, who's going to buy your shares? Why are you even going public? Um, but in particular, Anthropic is an interesting one because they, uh, have all of the-- So one of the reasons is to raise capital, right?

That's an obvious one, right? I mean, we need to raise capital because we've got a big, expensive model to build and loads of hardware to buy and stuff like that. But they've been raising loads of money as private equity or as private debt, and from what we've seen from the outside, they've had no issue raising the debt.

And debt is better than shares because debt doesn't have a vote in your company, and debt can be paid back. And if you think you're going to be very profitable in the mid to whatever, far future, well, why give away a bit of your company when you can just write some debt and when you get profitable, pay back the debt?

There you go. You still control 100% of the company, and you know best how to use the technology. Second reason you might raise-- That's a big one, right? So capital is a big one, and we can't know from here what the dynamics are behind the scenes, right? But it's just for me, I'm going, "Eh, really?" Second reason is, is as the, the Meta one, right, is talent retention and talent acquisition.

So if I want to hire the best people, one of the ways to incentivize them is to give them shares, and there's nothing you can do with those shares unless the company is public. So it gives a way for employees to liquidate their shares, right, and buy stuff and whatever. But Anthropic in particular is a funny company because all of the employees have pledged 50% of their shares to this public benefit.

The founders have pledged, I believe, 80% of their shares to this public benefit corporation, so for the good of humanity. And from what I've understood in the interview process, you were asked, um, you know, "Would you be okay giving away all of your shares to this, um, benefit?" And with the assumption that the right answer is, "Yes, I'm cool with that."

Frank: here's something, this is, this really genuinely disappointed me when I started looking into this IPO filing. I, you know, I have latched on to that fact that you just mentioned about the founders saying that they are going to give 80% of their personal equity, they're gonna donate it, uh, for the greater good, et cetera.

And I've kind of latched onto that and I thought, "Well, that's amazing. They're putting their money where their mouth is. They obviously really believe this," and I have always found that to be massively admirable. Then you read the IPO filing, and it basically says, um, "These commitments are voluntary, revocable expressions of each co-founder's philanthropic intent and do not constitute legally binding obligations." So once again, once again, we have the AI companies saying, "We believe this 100%. We absolutely believe this, and as a result, we are going to pinky promise that we're going to do something about it."

Justin: Well, I, I wouldn't get upset about that actually, Frank. Interesting, I would get... I mean, 'cause there, it's a particular type of legal, this is a legal document prospectus for an IPO. The- them pledging is slightly different. They could have a side agreement. Yeah, I don't know. I, I wouldn't get too worried about that.

I, I believe them

Frank: look, w- is-- if you, if you believe it and you're gonna put it out in the public realm that this is what you're gonna do, commit to it. What's it d- what's it doing in the IPO if it's, if it's not in any way legally binding whatsoever?

Justin: Okay, so I don't know, right? I'm neither a finance person nor a legal person, but the whole structure that you just laid in front of us there where you've got seven people who control one share, that's a nightmare by the way, getting seven people to agree on something. But anyway, so seven people that control one share and that controls half the company, so this is materially important to the future of the company.

And those seven people then have this side agreement where they're gonna give away 80% of their shares. I can imagine a, a future where four of the e- seven decide to give away 80%, and now who votes on what controls that one share that controls the entire... You know what I mean? It gets very complicated. So

Frank: we didn't-- we haven't even... Yeah, I mean, we haven't even covered some of the other complications around this because there's a whole other board, there's a whole other, um, group committee, I don't know what they, what they are, that's been formed that has control over the board. And then the thing that you said about, about the, um, the board, they're gonna be split into three groups, and they can only be voted out.

Uh, they all, they, they all have, like, um, stints of three years. So if you wanted to vote out the board, you could only ever vote out a third of them, basically. Um, and but that only comes into pl- being when the founders lose that 51%, uh, share, I think, so, you know, et cetera. So it's all super con- like, like I said, it's all these convula- convoluted hoops to try to ensure no hostile takeover by parties that they believe might not have the world's best interests at heart.

But yeah, I just, uh, when you, when you boil it all down, I'm just kinda like, this does not, to me, look like it's going to be good for AI safety.

Justin: Well, no, but, uh, yes. No. It's not good for AI safety, not because of all the convoluted things, I guess. It's not good for AI safety because they're now tied to a quarterly-- I don't know, they're such a strange company, tied to quarterly reporting, but actually I suspect that they don't even care. When you look at the stuff that happened with the American government a couple of months ago, they're like, "Yeah, we don't care.

We're doing what we think is right, and that's what we're gonna do." And that's cool if you're a private company. If you're a public company, um, it's not quite as cool. And, you know, think about-- Isn't it an interesting thing, right? So think about those two things, right? Um, so the disagreement they had with the Department of Defense six or eight months ago, which is-- seems to have disappeared now.

I haven't heard anything about it. And, and they're like, "No, we're gonna do what's best for humanity." Now, let's say they go IPO, and because it's the biggest IPO, um, and they're a very large company, they will probably immediately be on the Nasdaq, the, um, whatever, the FTSE, what-whatever, the, you know, New York Stock Exchange, the whole lot, right?

Therefore, my pension fund and the pension fund of lots of other people, it, by default, will buy some of their shares, right? So now I have, um, an interest in Anthropic, and if they decide they are gonna do something or not gonna do something for the good of humanity, that's gonna fin- financially impact me.

And as a public company, they're supposed to do what's in my interest, but as a public benefit corporation, they're supposed to do what's in the interest of humanity, and those two interests aren't always aligned.

Frank: Yeah.

Is Google Carbon a comeback we can’t use?

Frank: Um, kind of tangentially related, let's move on to the other story you wanted to talk about, which is Google Carbon, their latest frontier model. And again, this has made the news because basically Google have finally regained the lead in frontier models.

Justin: Have they?

Frank: well, accord- there's one, there's one YouTuber I, I follow and, um, whenever a company releases a model, they release all these benchmarks, and he refers to them as the trust me bro benchmarks because they've all been, they've all been done internally and they haven't been pub- publicly verified, et cetera.

So yeah, the trust-- But according to the trust me bro benchmarks, currently Carbon is, uh, uh, very much leading in the frontier models.

Justin: Let's test it out then, Frank

Frank: Let's test it out. Yes, how do we get access to it? Oh, hang on, we can't. They're doing the same kind of release as Anthropic did with Mythos, where it will first go to cybersecurity as, as small select group of cybersecurity people, then that, that cybersecurity panel will be expanded, and then eventually at some time or other, maybe businesses and, and consumers might, uh, get it at some point.

So people are a little bit disappointed about that. Um, but you know, it's interesting because you, you know, you mentioned the Genie model as well at the start, and people are excited about that world, the, the, the-- you can literally prompt-- Uh, we've seen, obviously this is like what, Genie 4 is it? We've seen previous models from Google where you can prompt it and it creates a whole world and you can walk through the world, you can control the character.

Um, and this one just looks like absolutely incredible, um, absolutely amazing. So between the, just the two things, like Google are, are very much on a, uh, a fast release cycle at the moment of some really impressive products,

Justin: Except they're not releasing them

Frank: And this is-- Well, yeah.

Justin: They're telling us about them though. They're in a fast telling us cycle

Frank: you can join the wait list for Genie, I believe, and, and try to-- At least with Genie, you, you, as a regular person, you can at least try and, and get on the-- get access.

Uh, but yeah, this is a problem with Google constantly. I mean, even just for stuff they do release, they have so many different packages and so many different ways that you can a-access their AI products. I never know what I have access to. Um, I have, you know, various-- I have a Google Workspace account, I have a personal account, I have totally different AI products available to me on both.

Last year, I spent months waiting for a product that, uh, support told me was just a slow rollout, only to find out that actually it wasn't in my plan at all. Um, and then two months later, they changed it and it was added to my plan. You just never know with Google how to access their products. It's crazy.

But the tangentially related bit that I mentioned coming into this is I find it kind of interesting. Now, I don't, I don't have any I, I, I, I've got my tinfoil hat on here a little bit, right? But, you know, Demis Hassabis stepped aside as CEO, and suddenly we seem to have all these models, and suddenly we have Google Carbon.

And, you know, if you couple the fact that Demis Hassabis stepped aside with we also have seen very similar AI safety and security, um, upheavals in Google as we've seen in other companies, and some-- there was one, uh, um, one woman was, uh, talking about why she was leaving because of the increased, um, Google's increased push for AI in the military or increased contracts with the military around AI.

Um, we saw they moved, um, they moved their, their security team, I think, from like London to the US, was that, was it? And then there was, so there was concerns that they would be too far removed from some of the development to actually have a meaningful impact. And I just-- I'm basically concerned that Google are pushing for commercialisation where Demis Hassabis might have been a little bit more of a, um, safety person, kind of slowing things down a little bit in the interests of safety.

And I could be wrong because obviously, look, y-you know, he left very recently. He was obviously involved in these models. But I don't know. It just, it already, even though he's only gone a short space of time, it feels different, and I wonder, are they falling into the same trap of rushing along with every other company

Justin: And to tie a bow in this for you, Frank, to bring us back to the start of the conversation and my question of why are Anthropic going public, Google today are down-- I don't know how much they're down. They're also being hit, right, 10, 15, 20% because of an earnings miss, right? They didn't earn as much money as they expected to, which is going to put more pressure on the AI division to release better models faster in order to make more money to get their share price up and keep up with the other model providers.

And after the IPO, Anthropic will be under the same sort of pressure. And I'm not concerned about it because I want better models faster, and I think it's great. But, you know, those who are more cautious among us might be going, "Yeah, really? Do we want to do that?" So it's an interesting one.

Is Grok’s Odyssey making Nolan nervous?

Frank: So we have one final segment of the show before we wrap up, Justin, and it is time for

Jingle: What's Elon up to?

Justin: Ooh. Bet you Elon would like that voice.

Frank: I picked that especially for Elon. I did.

Justin: Good lord. What is Elon up to?

Frank: Well, did you, do you remember in the summer there, um, Christopher Nolan released "The Odyssey," a three-hour epic, um, about, yeah, uh, the Odyssey essentially?

Justin: Yes

Frank: And did you know Elon Musk was not a fan?

Justin: Aw. Oh, well. There you go. Can't win them all

Frank: he was not happy about the casting. Um, he was not happy in particular about, uh, Elliot Page playing

Justin: Ja.

Frank: and he was not happy...

Uh, well, Elliot Page is a transgender actor, and Elon is not a fan, uh, of the, of any of it. Yes, absolutely. Um, so, uh, yeah, I, I, yeah, let's... Yes. God, I do, do not like Elon. Um, and he was not al- he was also not a fan of Lupita Nyong'o playing Helen of Troy, um, I presume, uh, because she's Black. And, uh,

Justin: Really? Did he actually give out about her?

Frank: uh, he retweeted a tweet, uh, he retweeted a tweet about that somebody else, um, uh, said, so he, yeah, he did that thing of, like, he made it clear what he believes, but he wasn't saying it himself.

Justin: Yeah

Frank: And so he says, "You know what? What I'm gonna do is I'm gonna get Grok to generate a full feature length, historically accurate version of 'The Odyssey'."

Um, and

Justin: Is he gonna release it in cinemas too?

Frank: he's-- well, he hasn't quite fulfilled his promise, uh, of creating a, a feature length, uh, version of "The Odyssey," but he has released the first... They're gonna do it episodically, so they've released the first seven-minute episode, and it was created by a London-based studio, and it took three people just under two weeks, I believe, and at a fraction of the conventional budget.

So I tried to do a bit of maths, right? And "The Odyssey," Christopher Nolan's "Odyssey" was around, uh, it was nearly three hours long. It cost $250 million. They, they're, they said that this c- the seven minutes, Grok's seven minutes cost tens of thousands of dollars. Now, is that $20,000 or is that $90,000? I mean, who knows.

But if we take it, if we take it to be, like, 50K, we'll just pick the, pick the middle ground and say 50K, it could take them, it could cost 0.34% of Nolan's budget to do

Justin: Less than half a percent. Good. This sounds like a, this sounds like progress

Frank: uh, it's seven minutes long and it felt longer than Christopher Nolan's "Odyssey."

Justin: Seven elapsed minute, but 50 years of my life have been sucked away from me

Frank: It is, uh, absolutely, um, shocking. It, the-- and look, on one hand, it's incredible that we've got to where we've got to. You know, it, it looks, it looks okay. Like, I follow some people on LinkedIn doing amazing things with AI video. What's, what's possible now is incredible. This is not-- Grok Imagine is not at the top level of AI video, but it's still, it's pretty impressive.

It's, you know, it's, it's amazing that they've managed to do what they've managed to do, but oh my God, it is not good. The voices are stilted, the AI performances are completely unconvincing, and nothing to do with AI particularly, like, th-there are AI problems, but there was much bigger problems in terms of this thing just did not know how to tell a story So

Justin: ever considered getting a job as a reviewer at The New York Times? "The fish was cold, and the

Frank: I,

Justin: was watery."

Frank: I do not think that Christopher Nolan is breaking a sweat right now about AI coming for his job

Justin: Well, well look, it's, it's progressing very fast. Who knows? This time next year maybe he will.

Frank: Maybe, maybe.

Justin: Frank, pleasure as always

Frank: Absolutely. I will chat to you next week and maybe episode two of "The Odyssey" will be out and maybe it'll be a hell of a lot better

Justin: Oh, indeed. I look forward to it. All right, have a good one

View all episodes

About The AI Argument

A weekly podcast where an approachable AI doomer and a techno-optimist argue over the latest AI news. Heavy topics, discussed lightly.

Portrait of Frank Prendergast

Frank Prendergast

The approachable doomer.

Portrait of Justin Collery

Justin Collery

The techno-overoptimist.