20、 Obama
Advanced US 10889 Folder Collection
After playing the video, you can click or select the word to look it up in the dictionary.
Loading...
Report Subtitle Errors
I'm going to talk about my research
on the long term future of artificial intelligence.
In particular, I want to tell you
about a very important phenomenon called "Intelligence Explosion."
There are two reasons that I work on intelligence explosion
and that I think it's worth sharing.
The first is that it's a phenomenon of immense theoretical interest
for those who want to understand intelligence on a fundamental level.
The second reason is practical.
It has to do with the effects that intelligence explosion could have.
Depending on the conditions
under which an intelligence explosion could arise
and on the dynamics that it exhibits
it could mean that AI changes very rapidly
from a safe technology, relatively easy to handle,
to a volatile technology that is difficult to handle safely.
In order to navigate this hazard,
we need to understand intelligence explosion.
Intelligence explosion is a theoretical phenomenon.
In that sense, it's a bit
like a hypothetical particle in particle physics.
There are arguments that explain why it should exist,
but we have not been able to experimentally confirm it yet.
Nevertheless, the thought experiment
that explains what intelligence explosion would look like
is relatively simple.
And it goes like this.
Suppose we had a machine
that was much more capable than today's computers.
This machine, given a task,
could form hypotheses from observations,
use those hypotheses to make plans, execute the plans,
and observe the outcomes relative to the task,
and do it all efficiently within a reasonable amount of time.
This kind of machine could be given science and engineering tasks
to do on its own, autonomously.
And this is the key step in the thought experiment:
this machine could even be tasked with performing AI research,
designing faster and better machines.
Let's say our machine goes to work, and after a while,
produces blueprints for a second generation of AI,
that's more efficient, more capable, and more general than the first.
The second generation can be tasked once again
with designing improved machines,
leading to a third generation, a fourth, a fifth, and so on.
An outside observer would see
a very large and very rapid increase in the abilities of these machines,
and it's this large and rapid increase
that we call Intelligence Explosion.
Now if it's the case
that in order to undergo an intelligence explosion
many new pieces of hardware need to be build,
or new manufacturing technologies,
then an explosion will be more slow
- although still quite fast by historical standards.
However, looking at the history of algorithmic improvement
it turns out that just as much improvement
tends to come from new software as from new hardware.
This is true in areas like physics simulation, game playing,
image recognition, and many parts of machine learning.
What this means is that our outside observer may not see physical changes
in the machines that are undergoing an intelligence explosion.
They may just see a series of programs
writing successively more capable programs.
It stands to reason that this process could give rise to programs
that are much more capable at any number of intellectual tasks than any human is.
Just as we now build machines that are much stronger, faster, and more precise
at all kinds of physical tasks,
it's certainly possible to build machines
that are more efficient at intellectual tasks.
The human brain is not at the upper end of computational efficiency.
And it goes further than this.
There is no particular reason
to define our scale by the abilities of a single human or a single brain.
The largest thermonuclear bombs release more energy
in less than a second
than the human population of Earth does in a day.
It's not out of the question to think
that machines designed to perform intellectual tasks
and then honed over many generations of improvement
could similarly outperfom
the productive thinking of the human race.
This is the theoretical phenomenon called Intelligence Explosion.
We don't have a good theory of intelligence explosion yet,
but there is reason to think that it could happen at software speed
and could reach a level of capability
that's far greater than any human or group of humans
at any number of intellectual tasks.
The first time I encountered this argument,
I more or less ignored it.
Looking back it seems crazy for me, someone who takes AI seriously,
to walk away from intelligence explosion.
And I'll give you two reasons for that.
The first reason is a theorist's reason.
A theorist should be interested in the large-scale features of their field
in the contours of their phenomena of choice as determined by
the fundamental forces, or interactions, or building blocks of their subject.
As someone who aspires to be a good theorist of intelligence,
I can't, in good faith, ignore intelligence explosion
as a major feature
of many simple straightforward theories of intelligence.
What intelligence explosion means
is that intelligence improvement is not uniform.
There is a threshold below which improvements tend to peter out,
but above that threshold,
intelligence grows like compound interest increasing more and more.
This threshold would have to emerge from
any successful theory of intelligence.
The way phase transitions emerge from thermodynamics,
intelligence would effectively have a boiling point.
Seeing this way,
exploring intelligence explosion is exactly the kind of thing
a theorist wants to do, especially in a field like AI,
where we are trying to move from our current state
,partial theories, pseudotheories, arguments, and thought experiments,
toward a fully-fledged predictive theory of intelligence.
This is the intelligence explosion.
In its most basic form,
it relies on a simple premise
that AI research is not so different from other intellectual tasks
but can be performed by machines.
We don't have a good understanding yet,
but there's reason to think that it can happen at software speed
and reach levels of capability
far exceeding any human or group of humans.
The second reason which I alluded to at the start of the talk
is that intelligence explosion could change AI very suddenly
from being a benign technology to being a volatile technology
that requires significant thought into safety
before use or even development.
Today's AI, by contrast, is not volatile.
I don't mean that AI systems can't cause harm.
Weaponization of AI is ongoing, and accidental harms can arise
from unanticipated systemic effects or from faulty assumptions.
But on the whole, these sorts of harms should be manageable.
Today's AI is not so different from today's other technologies.
Intelligence explosion, however highlights an important fact:
AI will become more general, more capable, and more efficient
perhaps very quickly
and could become more so than any human or group of humans.
This kind of AI will require
a radically different approach to be used safely.
And small incidents could plausibly escalate to cause large amounts of harm.
To understand how AI could be hazardous,
let's consider an analogy to microorganisms.
There are two traits
that make microorganisms more difficult to handle safely than a simple toxin.
Microorganisms are goal-oriented,
and they are, what I'm going to call, chain reactive.
Goal-oriented means
that a microorganisms behaviors
tend to push towards some certain result.
In their case that's more copies of themselves.
Chain reactive means
that we don't expect a group of microorganisms to stay put.
We expect their zone of influence to grow,
and we expect their population to spread.
Hazards can arise, because a microorganisms
values don't often align with human goals and values.
I don't have particular use
for an infinite number of clones of this guy.
Chain reactivity can make this problem worse.
Since, small releases of a microorganism can balloon
into large population spending pandemics.
Very advanced AI, such as could arise from intelligence explosion,
could be quite similar in some ways to a microorganism.
Most AI systems are task-oriented.
They are designed by humans to complete a task.
Capable AIs will use many different kinds of actions
and many types of plans to accomplish their tasks.
And flexible AIs will be able to learn to thrive,
that is to make accurate predictions and effective plans
in a wide variaty of environments.
Since AIs will act to accomplish their tasks as well as possible,
they will also be chain reactive.
They'll have use for more resources, they'll want to improve themselves,
to spread to other computer systems, to make backup copies of themselves
in order to make sure that their task gets done.
Because of their task orientation and chain reactivity,
sharing an environment with this kind of AI would be hazardous.
They may use some of the things we care about,
our raw materials, and our stuff to accomplish their ends.
And there is no task that has yet been devised
that is compatible with human safety under these circumstances.
This hazard has made worse by intelligence explosion,
in which very volatile AI could arise quickly from benign AI.
Instead of a gradual learning period,
in which we come to terms with the power of very efficient AI,
we could be thrust suddenly into a world
where AI is much more powerful than it is today.
This scenario is not inevitable,
it's mostly dependent upon
some research group, or company, or government
walking into intelligence explosion blindly.
If we can understand intelligence explosion,
and if we have sufficient will and self-control as a society,
then we should be able to avoid an AI outbreak.
There is still the problem of chain reactivity though.
It would only take one group to release AI into the world
even if nearly all groups are careful.
One group walking into intelligence explosion accidently or on purpose
without taking proper precautions,
could release an AI that will self-improve
and cause immense amounts of harm to everyone else.
I'd like to close with four questions.
These are questions that I'd like to see answered
because they'll tell us more about the theory of artificial intelligence
and that theory is what will lead us understand intelligence explosion
well enough to mitigate the risks that it poses.
Some of these questions are being actively pursued
by researchers at my home institution,
The Future of Humanity Institute at Oxford,
and by others, like The Machine Intelligence Research Institute.
My first question is,
"Can we get a precise predictive theory of intelligence explosion?"
What happens when AI starts to do AI research?
In particular, I'd like to know
how fast software can improve its intellectual capabilities.
Many of the most volatile scenarios we've examined include
a rapid self-contained take off,
such as could only happen under a software improvement circumstance.
If there is some key resource that limits software improvement
or if it's the case that such improvement isn't possible
below a certain threshold of capability,
these would be very useful facts from a safety standpoint.
Question two:
what are our options, political or technological,
for dealing with the potential harms
from super efficient artificial intelligences?
One option, of course, is to not build them in the first place.
But this would require exceedingly good cooperation
between many governments, commercial entities, and even research groups.
That cooperation and that level of understanding isn't easy to come by.
It would also depend, to some extent, on an answer to question one
so that we know how to prevent intelligence explosion.
Another option would be to make sure
that everyone knows how to devise safe tasks.
It's intuitively plausible that there are some kinds of tasks
that can be assigned by a safety conscious team
without posing too much risk.
It's another question entirely
how these kinds of safety standards could be applied
uniformly and reliably enough all over the world
to prevent serious harm.
This leads into question three: very capable AIs,
if they can be programmed correctly,
should be able to determine
what is valuable
by modeling human preferences and philosophical arguments.
Is it possible to assign a task of learning what is valuable
and then acting to pursue that aim?
This turns out to be a highly technical problem.
Some of the ground work has been laid by researchers
like Eliezer Yudkowsky, Nick Bostrom,
Paul Christiano and myself
but we still have a long way to go.
My final question, as a machine self-improves it may make mistakes.
Even if the first AI is programed to pursue valuable ends,
later ones may not be.
Designing a stable and reliable self-improvement process
turns out to involve some open problems
in logic and in decision theory.
These problems are being actively pursued at research workshops
held by The Machine Intelligence Research Institute.
Those are my four questions.
I've only been able to cover the basics in this talk.
If you'd like to know more
about the long-term future of AI and about the intelligence explosion,
I can recommend David Chalmers' excellent paper,
"The Singularity of Philosophical Analysis,"
as well as a book forthcoming in 2014 called,
"Super Intelligence" by Nick Bostrom.
And of course there are links and references on my website.
I believe that managing
and understanding intelligence explosion
will be a critical concern
not just for the theory of AI but for safe use of AI
and possibly, for humanity as a whole.
Thank you.
(Applause)
    You must  Log in  to get the function.
Tip: Click on the article or the word in the subtitle to get translation quickly!

Loading…

Loading…

Advanced US

【TEDx】The long-term future of AI(and what we can do about it): Daniel Dewey at TEDxVienna

10889 Folder Collection
Michael Lu published on February 4, 2018    Bruce Hsu translated    Jerry reviewed

Learning Words

loading

Comments

Loading…
More Recommended Videos

  1. 1. Search word

    Select word on the caption to look it up in the dictionary!

  2. 2. Repeat single sentence

    Repeat the same sentence to enhance listening ability

  3. 3. Shortcut

    Shortcut!

  4. 4. Close caption

    Close the English caption

  5. 5. Embed

    Embed the video to your blog

  6. 6. Unfold

    Hide right panel

  1. Listening Quiz

    Listening Quiz!

  1. Click to open your notebook

  1. UrbanDictionary 俚語字典整合查詢。一般字典查詢不到你滿意的解譯，不妨使用「俚語字典」，或許會讓你有滿意的答案喔