Name: How Competition Is Stifling AI Breakthroughs | Llion Jones | TED
Uploaded: 2026-03-02T21:34:41.000Z
Duration: 16 min 1 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

Adverbs of manner

So as mentioned, I'm probably most well known as one of the Transformers authors.

Transformers are of course the T in ChatGPT and are the architectures that run most of the state of the art artificial intelligence.

Adverbs of degree

If I think back to that time when we were working on the Transformers, I remember it as a very organic, bottom up kind of project where the idea came from talking over lunch or scribbling randomly, whiteboards in the office.

And importantly, when we felt like we did actually have a good idea, we had the freedom to actually spend the time and go and work on it.

And even more importantly, we didn't have any pressure that was coming down from management.

No pressure to work on any particular project, publish a number of papers, to push a certain number up.

So that's the image I want you to have in your mind, right?

That is the kind of environment that allowed the Transformer to come into existence.

An organic, open ended and with a lot of freedom to pursue the ideas that we thought were interesting and important.

And my deep concern is that right now in the AI industry we do not have this kind of environment.

And I want to talk about why not and what can we do about it.

So the main paradox that I see in artificial intelligence research or the industry in general right now is that despite the fact that there's never been so much interest and resources and money and talent, this has somehow caused a narrowing of the research that we're doing.

And to me, I think the reason is fairly obvious.

It's because the immense amount of pressure that comes with that, right?

Pressure from investors that are going to ask for a return on their investment, and

Because this is such an overcrowded industry right now, where it's very difficult to stand out.

And the researchers, they're really feeling this pressure, right?

If you're doing, let's say, standard AI research right now, you kind of have to assume that there's maybe three or four other groups doing something very similar, or maybe exactly the same.

So you have to spend the time checking to see if you've been scooped, to see if someone else has put your idea out there.

And even in academia, where you would hope you would have more freedom, there's pressure to publish and to have your papers published.

So if you have an interesting idea that could create something very interesting, or you have kind of a mediocre idea that no, it'll probably get a paper and probably get accepted, the temptation is to go for the low hanging fruit.

So unfortunately, this pressure damages the science because people are Rushing out papers and it's reducing the amount of creativity that we have.

So I want to take an analogy from AI itself.

So when we're designing AI search algorithms, we have to trade off something called the exploration exploitation.

When you're searching for a solution you can either spend your time exploring or exploiting.

If you spend all your time exploring, then you will probably only find a large number of medical solutions.

If you spend your time just exploiting, then you might lose out on finding other alternative solutions that you might be able to exploit better and improve better.

And we are almost certainly in that situation right now in the AI industry.

So all I really want to ask you today is to consider just changing that balance a little bit, just turning up the dial and exploring more.

So I actually remember what it was like just before the Transformers, and

I want to paint that picture for you as well.

Back then, my main memory was there were a lot of papers coming out and they were always permutating the current architecture, but

which was recurrent neural networks at the time, just endlessly trying different things, different gates, different layers, mostly for incremental gains.

And then after the transformer came out, all of that work that was spent on improving the recurrent neural network kind of felt a bit pointless.

How much time do you think those researchers would have spent trying to improve the recurrent neural network if they knew something like Transformers was around the corner?

It turned out we needed a longer conceptual leap.

We needed to throw away recurrence completely.

And again, I am worried that we're in that situation right now where we're just concentrating on one architecture and just commuting it and trying different things, where there might be a breakthrough just around the corner.

And if there is, then we should be acting like it.

The next breakthrough, almost by definition, has to come from this sort of open ended, much more speculative research.

And the only way to really hedge your bets against missing out on the next big thing is to invest in this kind of research.

So if I came up here and did nothing but just moan about the current situation, I don't think it would be a great talk.

So I want to give you a couple of suggestions.

First of all, in my company we champion having nature inspired.

So there are still things, plenty of things that the human brain can do that current state of the art AI can't do.

So maybe if we take some inspiration from nature, we can get some of those properties, but

You should follow what's interesting to you.

There's actually a quote I heard two weeks ago, and I thought, that's perfect.

I'm having that for my talk, and I think I'm stealing it from a guy called Brian Chung.

You should only do research that wouldn't happen if you weren't working on it.

And if we all did that, we wouldn't be stepping on each other's toes and we'd be exploring much more efficiently.

So I want to give you a concrete example.

There's a piece of research that we put out recently called the Continuous Thought Machine.

And all we did is we just took a little bit of inspiration from nature.

So in the human brain, synchronization is very important, and

we try to add this kind of synchronization into artificial neural networks.

I remember the employee coming to me with the idea, and

I said, okay, work on it for a week and we'll see what happens.

That employee later confided in me that in his previous employment, or even in his academic position before that, that he probably would have gotten skepticism and told not to waste his time.

But after that week, he started to find much more interesting properties of this model, and

We actually announced that we got the spotlight at NeurIPS this year.

And I think there's a couple of reasons for that.

I think there's a hunger for this kind of new and differentiated research.

And more interestingly, at no point when we were working on this project did we have to worry about being scooped so we could take our time to do the science properly and run the benchmarks that we wanted to run.

And I think that's the kind of research we should be doing.

So hopefully from that, you can tell that I'm not just up here trying to make a talk that sounds good.

I am putting my money where my mouth is, and

I am creating this kind of environment, the kind of environment that allow transformers to come into existence at my company.

I'm not sure if I should tell you this, because it's a bit of an advantage that the company has right now, but

it's a really, really good way of getting talent.

Talented, intelligent people, ambitious people will naturally seek out this kind of environment with high autonomy.

And some of our best hires recently have been explicitly because of this reason.

Oh, by the way, it works better than just money.

These superstars that are apparently being snapped up for literally a million dollars a year in some cases.

Do you think that when they start their new position, they feel empowered to try their mad ideas, their more speculative ideas?

Or do they feel immense pressure to prove their worth and will once again go for the low hanging fruit?

I think that maybe we're not exploring quite as efficiently as we should be.

And that's because Transformers are too good.

I know, modesty, but seriously, I mean, what can I mean by that?

What I mean is I think the punchline is going to be that when we look back at this point in AI history, the fact that the current technology is so powerful and flexible that it stopped us from looking for better, it makes sense, right?

If the current technology was worse, more people would be looking for better.

So there's two points I would like to clarify.

First of all, I'm not saying that there isn't already plenty of very interesting research happening.

I'm just saying that given the amount of talent and resources that we have currently, we can afford to do a lot more.

I and several other many other researchers believe we're not done and we should be looking for better.

But I'm also not saying that we should throw away the current technology.