Placeholder Image

Subtitles section Play video

  • Translator: Joseph Geni Reviewer: Morton Bast

  • My passions

  • are music, technology and making things.

  • And it's the combination of these things

  • that has led me to the hobby of sound visualization,

  • and, on occasion, has led me to play with fire.

  • This is a Rubens' tube. It's one of many I've made over the years,

  • and I have one here tonight.

  • It's about an 8-foot-long tube of metal,

  • it's got a hundred or so holes on top,

  • on that side is the speaker, and here

  • is some lab tubing, and it's connected to this tank

  • of propane.

  • So, let's fire it up and see what it does.

  • So let's play a 550-herz frequency

  • and watch what happens.

  • (Frequency)

  • Thank you. (Applause)

  • It's okay to applaud the laws of physics,

  • but essentially what's happening here

  • -- (Laughter) --

  • is the energy from the sound via the air and gas molecules

  • is influencing the combustion properties of propane,

  • creating a visible waveform,

  • and we can see the alternating regions of compression

  • and rarefaction that we call frequency,

  • and the height is showing us amplitude.

  • So let's change the frequency of the sound,

  • and watch what happens to the fire.

  • (Higher frequency)

  • So every time we hit a resonant frequency we get a standing wave

  • and that emergent sine curve of fire.

  • So let's turn that off. We're indoors.

  • Thank you. (Applause)

  • I also have with me a flame table.

  • It's very similar to a Rubens' tube, and it's also used

  • for visualizing the physical properties of sound,

  • such as eigenmodes, so let's fire it up

  • and see what it does.

  • Ooh. (Laughter)

  • Okay. Now, while the table comes up to pressure,

  • let me note here that the sound is not traveling

  • in perfect lines. It's actually traveling in all directions,

  • and the Rubens' tube's a little like bisecting those waves

  • with a line, and the flame table's a little like

  • bisecting those waves with a plane,

  • and it can show a little more subtle complexity, which is why

  • I like to use it to watch Geoff Farina play guitar.

  • (Music)

  • All right, so it's a delicate dance.

  • If you watch closely — (Applause)

  • If you watch closely, you may have seen

  • some of the eigenmodes, but also you may have seen

  • that jazz music is better with fire.

  • Actually, a lot of things are better with fire in my world,

  • but the fire's just a foundation.

  • It shows very well that eyes can hear,

  • and this is interesting to me because

  • technology allows us to present sound to the eyes

  • in ways that accentuate the strength of the eyes

  • for seeing sound, such as the removal of time.

  • So here, I'm using a rendering algorithm to paint

  • the frequencies of the song "Smells Like Teen Spirit"

  • in a way that the eyes can take them in

  • as a single visual impression, and the technique

  • will also show the strengths of the visual cortex

  • for pattern recognition.

  • So if I show you another song off this album,

  • and another, your eyes will easily pick out

  • the use of repetition by the band Nirvana,

  • and in the frequency distribution, the colors,

  • you can see the clean-dirty-clean sound

  • that they are famous for,

  • and here is the entire album as a single visual impression,

  • and I think this impression is pretty powerful.

  • At least, it's powerful enough that

  • if I show you these four songs,

  • and I remind you that this is "Smells Like Teen Spirit,"

  • you can probably correctly guess, without listening

  • to any music at all, that the song

  • a die hard Nirvana fan would enjoy is this song,

  • "I'll Stick Around" by the Foo Fighters,

  • whose lead singer is Dave Grohl,

  • who was the drummer in Nirvana.

  • The songs are a little similar, but mostly

  • I'm just interested in the idea that someday maybe

  • we'll buy a song because we like the way it looks.

  • All right, now for some more sound data.

  • This is data from a skate park,

  • and this is Mabel Davis skate park

  • in Austin, Texas. (Skateboard sounds)

  • And the sounds you're hearing came from eight

  • microphones attached to obstacles around the park,

  • and it sounds like chaos, but actually

  • all the tricks start with a very distinct slap,

  • but successful tricks end with a pop,

  • whereas unsuccessful tricks

  • more of a scratch and a tumble,

  • and tricks on the rail will ring out like a gong, and

  • voices occupy very unique frequencies in the skate park.

  • So if we were to render these sounds visually,

  • we might end up with something like this.

  • This is all 40 minutes of the recording,

  • and right away the algorithm tells us

  • a lot more tricks are missed than are made,

  • and also a trick on the rails is a lot more likely

  • to produce a cheer, and if you look really closely,

  • we can tease out traffic patterns.

  • You see the skaters often trick in this direction. The obstacles are easier.

  • And in the middle of the recording, the mics pick this up,

  • but later in the recording, this kid shows up,

  • and he starts using a line at the top of the park

  • to do some very advanced tricks on something

  • called the tall rail.

  • And it's fascinating. At this moment in time,

  • all the rest of the skaters turn their lines 90 degrees

  • to stay out of his way.

  • You see, there's a subtle etiquette in the skate park,

  • and it's led by key influencers,

  • and they tend to be the kids who can do the best tricks,

  • or wear red pants, and on this day the mics picked that up.

  • All right, from skate physics to theoretical physics.

  • I'm a big fan of Stephen Hawking,

  • and I wanted to use all eight hours

  • of his Cambridge lecture series to create an homage.

  • Now, in this series he's speaking with the aid of a computer,

  • which actually makes identifying the ends of sentences

  • fairly easy. So I wrote a steering algorithm.

  • It listens to the lecture, and then it uses

  • the amplitude of each word to move a point on the x-axis,

  • and it uses the inflection of sentences

  • to move a same point up and down on the y-axis.

  • And these trend lines, you can see, there's more questions

  • than answers in the laws of physics,

  • and when we reach the end of a sentence,

  • we place a star at that location.

  • So there's a lot of sentences, so a lot of stars,

  • and after rendering all of the audio, this is what we get.

  • This is Stephen Hawking's universe.

  • (Applause)

  • It's all eight hours of the Cambridge lecture series

  • taken in as a single visual impression,

  • and I really like this image,

  • but a lot of people think it's fake.

  • So I made a more interactive version,

  • and the way I did that is I used their position in time

  • in the lecture to place these stars into 3D space,

  • and with some custom software and a Kinect,

  • I can walk right into the lecture.

  • I'm going to wave through the Kinect here

  • and take control, and now I'm going to reach out

  • and I'm going to touch a star, and when I do,

  • it will play the sentence

  • that generated that star.

  • Stephen Hawking: There is one, and only one, arrangement

  • in which the pieces make a complete picture.

  • Jared Ficklin: Thank you. (Applause)

  • There are 1,400 stars.

  • It's a really fun way to explore the lecture,

  • and, I hope, a fitting homage.

  • All right. Let me close with a work in progress.

  • I think, after 30 years, the opportunity exists

  • to create an enhanced version of closed captioning.

  • Now, we've all seen a lot of TEDTalks online,

  • so let's watch one now with the sound turned off

  • and the closed captioning turned on.

  • There's no closed captioning for the TED theme song,

  • and we're missing it, but if you've watched enough of these,

  • you hear it in your mind's ear,

  • and then applause starts.

  • It usually begins here, and it grows and then it falls.

  • Sometimes you get a little star applause,

  • and then I think even Bill Gates takes a nervous breath,

  • and the talk begins.

  • All right, so let's watch this clip again.

  • This time, I'm not going to talk at all.

  • There's still going to be no audio,

  • but what I am going to do is I'm going to render the sound

  • visually in real time at the bottom of the screen.

  • So watch closely and see what your eyes can hear.

  • This is fairly amazing to me.

  • Even on the first view, your eyes will successfully

  • pick out patterns, but on repeated views,

  • your brain actually gets better

  • at turning these patterns into information.

  • You can get the tone and the timbre

  • and the pace of the speech,

  • things that you can't get out of closed captioning.

  • That famous scene in horror movies

  • where someone is walking up from behind

  • is something you can see,

  • and I believe this information would be something

  • that is useful at times when the audio is turned off

  • or not heard at all, and I speculate that deaf audiences

  • might actually even be better

  • at seeing sound than hearing audiences.

  • I don't know. It's a theory right now.

  • Actually, it's all just an idea.

  • And let me end by saying that sound moves in all directions,

  • and so do ideas.

  • Thank you. (Applause)

Translator: Joseph Geni Reviewer: Morton Bast

Subtitles and vocabulary

Operation of videos Adjust the video here to display the subtitles

B1 US TED frequency sound lecture skate fire

【TED】Jared Ficklin: New ways to see music (with color! and fire!) (Jared Ficklin: New ways to see music (with color! and fire!))

  • 44 0
    Zenn posted on 2017/01/05
Video vocabulary