Duration 35:30
16+
Play
Video

Sound Amplifier and the new Dynamics Processing Effect

Ricardo Garcia
Staff Software Engineer at Google
+ 1 speaker
  • Video
  • Table of contents
  • Video
2018 Google I/O
May 10, 2018, Mountain View, USA
2018 Google I/O
Video
Sound Amplifier and the new Dynamics Processing Effect
Available
In cart
Free
Free
Free
Free
Free
Free
Add to favorites
7.38 K
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

About speakers

Ricardo Garcia
Staff Software Engineer at Google
Brian Kemler
Product Manager at Google

Ricardo Garcia (rago) is a senior software engineer at Android Audio Framework team. He holds a BSEE from his native Colombia and two MS in Music Engineering (U. of Miami) and Media Arts & Sciences (MIT). He published work on audio watermarking, psychoacoustic models, genetic programming and hearing health. Worked at Chaoticom on low bitrate codecs and founded Base 10 Labs for custom DSP solutions.

View the profile

Brian joined Google in 2007 as one of Google Enterprise’s first sales engineers working on the Google Search Appliance and Google Apps. Brian bootstrapped Google’s expansion of Search and Gmail in Myanmar (Burma). He created the Google News Next Billion program launching 9 editions in underserved markets and then went on advance meetings experiences as a product manager on Chromebox for meetings. Brian currently manages accessibility product development for Android. He graduated from the George Washington U

View the profile

About the talk

Sound Amplifier is a new accessibility service built to demonstrate the capability of Android’s new Dynamics Processing Effect in the Audio Framework. Sound Amplifier helps users focus on the sounds they want to listen to regardless of poor acoustic conditions or distractive noise. Dynamics Processing offers powerful audio post-processing with a flexible API that allows developers to adjust an audio stream’s dynamic range and frequency response, with the same precision sound engineers can have with high end mixing boards. The API can be used to improve any listening experience as demonstrated by the Sound Amplifier. It has a multi-stage processing architecture including dual multi-band equalizers, multi-band compression, and a linked limiter. This session includes demos of the Sound Amplifier and dives into the API, as well as other potential uses by third party developers of the effect.

Share

Good afternoon, everybody. My name is Brian kamler. I'm a product manager and I work on Android accessibility in Android. How's everybody's last day of Ayo last last session? You made it through thank you so much for coming out. It was great. I owe on today. We have a lot of really exciting things to talk about primarily Geared for developers. We're going to be talking about a new sound enhancement feature in Android P called sound amplifier. We're going to be doing a little bit of a live demo to show how we can improve

the sound in any type of situation for users and how you can do the same as a developer. We're going to go and do a deep dive into the inner workings of sound amplifier and the Dynamics processing effect upon which is filled live with code examples. It's going to be really cool. Let's Dive Right In Listening is difficult even in empty field with a massive set of ears. But the everyday reality is we live work and play with an increasing amount of environmental noise. In fact unwanted sound is one of the most common environmental problems.

It's not only annoying but it prevents us from understanding our friends colleagues and loved ones. No matter how well we can hear most of us can relate to the following acoustically challenging situations. Trying to understand your date. It allowed restaurant or allowed bar. Trying to listen to a collar in a really loud airport lounge. Trying to listen to a speaker who's talking way too softly like me now environmental noise as a form of situational disability. But what if you can hear better using just your smartphone so you never miss a word in the conversation?

Now you can because today. We're introducing sound amplifier. What is sound amplifier sound amplifier is a new accessibility service that helps users focus on real-world conversations using only your Android smartphone and a set of headphones. Users can tune to hundreds of personalized levels to optimize their listening experience to the current environment. Is to sliders for loudness and tuning dynamically adjust over 100 audio presets in the background. These settings can be applied independently

teach are adjusting that improves the sound quality in an array of situations including the following enhancing sound in loud distant or otherwise acoustically challenging situations. Increasing the volume of somebody who speaks talks too Softly. Turning up the TV volume to one that's acceptable to everybody in the room without blasting everybody else. So now I'm going to go in and talk a little bit about how we built it the apis in the effect upon which it was built and I'll do some demos.

Sound amplifier is based on Android Keys new Dynamic processing effect. The effect is a four-stage signal processing architecture and I'm going to walk you through each stage and then Ricardo is going to come in a little bit and really deep diving into this and show a picture of the developers in the audience exactly how they can adapt this to their application. So first stage 1 Big one is pre Equalization. You can think of. Causation as an equalizer that you can programmatically adjust any

audio frequency bass midrange treble the entire audio Spectrum. Steve's to is a multi-band compressor. The multiband compressor is the heart and soul of the Dynamics processing attacked because what it does is really unique. It can simultaneously adjust down really loud or irritating sounds don't worry. I won't make any or and just adjust. Sounds that are too soft. He could do all of this without changing the characteristics of the underlying audio that you put

into the system. The third stage is also an equalization tages post Equalization allows you to fine-tune the output from the original NBC and the original on pre equalizer and lastly is a limiter. What the limiter does is protects a dish protect users from additional gain or additional volume above a certain developer designated threshold. So that there no loud drawing or uncomfortable noises. So now I'm going to switch switch over to a do a little demo.

So I'm going to play a very typical Google micro kitchen water cooler type of type of break where there's a loud espresso maker in the background and I'm in the sky star in this. They can hear the sound is if you can't really hear the conversation taking out the trash and start to hear the conversation. Remember, this is customizable tunable forever user. Doing this again again, I get to Star again. I'm really not this into myself. Hears me talking kind of rehearsing this talk right now and I'm speaking too quietly. So this will be another example of how I can adjust the sound

so that I can hear it better. The other thing that I also forgot to mention was use this with your headphones so you can think of what's in the video as you know, what you would be hearing in your headphones. So listening, Even under the best circumstances in a quiet field, no background noise and a large set of ears. I can hear the Boost button noisy conditions. The everyday reality is that we work play and live with an increasing amount of environmental noise. So listening is difficult this very easy to

fine-tune any acoustic situation. To the environment to your to your ears and so forth and Ricardo is going to Deep dive in and talk a little bit about little bit more about the Dynamics processing effect and how developers can start using this today. Thank you. Well now we saw the Deimos and we so I did I'd a sound amplifier actually working real life. So we are going to go down and see how the sounds of fire actually works. And what is the magic behind?

So for that, I want you to introduce to introduce to you the hidden treasure. I'm going to be in the supplies for a second and going to explain to you. What is here. What is the average student ratio for a human to human ratio is actually how much energy we need to have to be able to listen to the sound. So in this plot we can see from left to right from low frequency to high frequencies from 20/3 to 20 kg in the vertical axis. We can see the amplitude. We see how much energy do we actually will need to hear the sounds

so because this is the threshold of hearing and quiet these Pretty much telling that anything that is below the treasure would not be hurt by people. So for example, if we go and we look into the $4,000 we see the Declaration is lowering their that means that we need less energy we need and don't let sounds can be fainter. They can be softer and we will hear them but it was found that is about this. Level in around you will not be able to hear that. So this is important because many companies use this cursed this kind of tradition a human treasure to design their

equipment when you aren't deciding I don't know MP3 or CD players. It discovers our present or is you on TuneIn a headset speakers microphones. All of these girls are waiting for 10 to know what people will actually here and what they are sensitive to. midi skirts can chips So we can have kidney treasure shift due to so many circumstances that says the noise in the environment in here underwater treasure ship we needed to have sounds that we're way louder over that treasure to be able to listen to hear the sounds when they see turn

off. I mean we can speak softer and we feel. I want racial shifter down at the moment is a region that will need more energy for you to be able to notice that We have older ships. We can have cheats that are not as Broadband not all over the spectrum of the previous one. We can have shifts that are more localized this Fluker in here is showing us a shift that is around $500 real real really high. There is some noise of some of threats in there that is not going to allow you to hear things very well around

536 school be permanent. All all your lies are going to have the shift or could be temporary as Ryan mentioned could be a temporary situation of disability of time for not going to be able to hear things around those frequencies because either you can someone working will you learn of noisy environment your ears and nose working? Well at that moment. Sounds that are below the threshold that's shifted to racial. You are not going to be able to hear that and that's the important thing away from these if it's below the threshold. You cannot hear it. So what

happens when we have a sound of Interest they say that you're listening to music or someone is speaking or so something that interests you so independent. I just put a block that's a green blob in there is a Broadband sound cazadora frequencies and he has energy all over the place. But did you notice again around $500? A lot of the energy that is below. The threshold has shifted to the blue one is going to be lost. When you have a sound of interest that is below The Treasure of the ship to treasure is is going to be difficult to be her or 8:30 guilty is

really bad. You cannot understand this out. So this is the question that everybody's asking how can sound amplifier help us right now. So I'm glad that you asked is going to take the sounds and is going to actually boost them. Were you needing the most if we look at them and they in the range from $500 to 1 km. We took the sounds and This new process sound with to the sounds of with really soft. They didn't have that much energy and we boosted them all the way over the Chiefs the treasure. So now we can hear those but very important which of the sounds of winter

read-alouds and we move them up but not too much. So the loud sounds good but not too loud. So if you see this did the process that happened it was not uniform all over the Spectrum. It was shifting things and can I drink alcohol the day over at were shifted pressure? So it was doing a very intelligent boost in all of this processing is done by the new Dynamics processing affect the wind produced in Android and repeat and I'm going to talk more and going to show cause in a minute. I promise so Did I not explicit in effect

is the new process because 4 stages the preacher multiband compressor for SECU unlimited when we are using this effect. We have two big questions that we need to answer especially for our sound amplifier. First is what kind of processing we actually want to hear sounds to be able to depending of the needs that we have a weird and Loud environment or we are in a bad connection or we are in an airport or depending what kind of processing we need to make things aboard the ship the treasure that we have and how users

can actually go and move and find the right parameters that they need for you. So we have two big questions to ask. The first one for the processing what kind of process in which one of these with Google is really good at we took a lot of data and we start crunching Saturday. So which data from a human Treasures from regular and into a human in users so actually help people feel or how their ears response and we put them in there with two commercial ships from different environment places and different environment noises. We went to Airport restaurants and

places where we have no choice obstructionist things that are going to shit our treasure appearing and we also put them in in the blender if we went to try to get the audio content from the people actually want to hear a conversations movies Music Live concerts a lecture and we managed to pull all of this together and come up with a set of a recipes of el palomito. That would be good for processing the sound in many situations. When you do that you end up with I will say I move two dimensional space a bunch of

solutions that are pretty much fat does very complex Panorama test that you want to tell the Dynamics effects processor for this equation. This will be good to see if the sound for this situation will be. But that's on my devil is you want to tell that uses? What was the moving parameters? Like this will be hundreds of kilometres. I will be very difficult for the users to make something good with it. So we took it further steps. I did something that I can do all of those recipes move two dimensional space and flatten them out and place them in two dimensions in here in the building

the right you can see that we see each one of those is kind of formula reprocessing a recipe that we want a to feed the dynamic process to change the song. We did I mention Elite option we found two The tone and divorced the uses of the Souls II question that we have how the users are going to navigate this what they don't need to know. They all the parameters that we need. We just give them to slide the two slices that would you solve the demo by moving. Snyder. They're actually doing a very intelligent mapping and

going and moving at Towson removing for each channel in there when the user goes and it write using the slider. They're actually going through a bunch of a recipes are a bunch of a processing and they can find the one that suits them the best at that moment for that user in that location. That's what we wanted to do. So that was kind of like a bird. I wanna play fireworks now, we are going to go deeper and start talking about cold. What can you do with an economics affects a person's going to fix? So you can find the alignment process defects

in as a new library in the Android media. How do you fix Dynamics processing? And right now so let's stop talking about use cases. So when can you use a dynamic process in effect the first use case we saw a length you can do as an amplifier. So if you go today because this is now available in the Android a p developer preview. You can go tonight. I'm sure that everybody's going to skip the party and go and create you don't then I'm fine with the timer with alarm supposed to affect. We are expecting people to create their own son of the fires or

Creative Solutions to be able to hear Barry to listen better. I know there was something a little TV that can happen is it when you have a device on Android device and you want to tune the speaker? So the headphones many companies a half sword that equalizer do something to the headphones to make the sound a flutter better to have more bass something you want. Now with this effect. It will be easy for you or for your clients to do that. Another application that is really interesting is the TV remote if you're watching TV to a year and you cannot sleep but if you start

watching a movie and sometimes if someone is Whispering the movie just saying something rude aquatic animal jam music on Cinder all something really loud you woke up everybody in the house with a dynamic process. You can actually have something that they whispered a very soft parts of the movie can be raised up but the very loud music or gunshots or whatever has happened in the movie. He can bring down and everything is going to work on more level of sound in the world ever found another example can be for media players on media player and you want to

actually a do some loud is maximization or Master you want you want to listen to classical music Media Player and you are going to train you will like to have the controls to be able to again in squeeze the loud sounds and a bring up the stuff sounds so everything is working well for the diameter. Are you having dinner now? I'd like to talk a little bit about architecture and I promised cold is coming. But I want to show their protection. We are giving you a lot of power with this affect. So we make sure that we shall before we were kind of line is not a simple is a little bit more complex

architecture. Yes, we have four stages. We have a free cue multiband compressor for secuality, but we can do more things inside of each one of those hostages. We have a lot of parameters that you can control we have the bands and in the Freak You Move tea bag compress on a Pasa que we can have as many bones as you want and you can control each one of those bands and the parameters in those bands delimiter is a single unlimited but you can also control that all of these is it comes into each other? So I channel was it we have to find the channel as you have

a spiritual singular stereo signal has two channels left and right so we are going to have Channel 3. Channel one and you can control all those independently or you can have many more chatter. Let's say that you have a 5.1 same signal you actually have six channels that you can control you can index all of those channels and say I want to change the Moody one parameter the Liberty One controller by number 3 in the channel to and I want to change this. We are giving you an API to do all of those things that I want you to know. This is the limiters the limiters are linked

limiters. I'm going to talk more about that but is important when you are talking about Boogie channel to be able to a chainsaw delimiter scan of her once if Jeanette. So now finally code yes. We promised goes we have some code. So this is an instance institution example of the dynamic process in effect for this we decide because we have so many promises are so many things that can be configured to have a complete building. So you create a config Builder and then we are actually going to start a defect first with the concept builder in this example. We we can

have some parameters like the variance of the number of channels have it then we can enable or disable is one of the stages in this example. We have all the stages enabled. Yes, we want to use preview multiband compressor for SECU unlimited the number of bonds. We can say how many bands we want for the state's type so I can have all the previews can have less at the Museum on compressive strength bands and the Post Acute 13 bands. The only limitation is like this case knife needs to have the same number of bonds. But then you can choose any number of banks that you

want. We have some extra parameters as I mentioned the variance and the number of channels and they said prefer friend duration that you you can set in the config builder. Then in the last line. We are actually creating a configuration. We are saying built and we have a configuration object. Then I I don't want to use the configuration directly. Sometimes I want to finish the architecture of my effect and go and change the default parameters. So in this example, I'm getting a channel on getting my channel object in the first line from the config and I'm saying from channel

number 0 in this case get the multiband compressor. We have the multiband compressor station and then I eat the right through the 8th a multi One banks that I have in my booty on compressor and I went and changed the parameters in each. One of them. This is an example for a real world applications. You can go and do more things that you see that we can go and set each one of the things that we want the release their tag, the racial The Treasure of the way that you want for each one of the bands and finally we go Anniston shared a dynamic process can affect my science courses in

the face with a the config file that we just created decision. Best way to attach these effects to whatever day media player or the audio track that of interest for you and then you can start juice going in your brother and controlling every I'm going to instead of going on the AP. I'm just showing the rightful and going to go a little bit more metamorphine Septra to show these in graphics so old when we are running real time, we can have access to channels that I just show you. And we have many channels. So the

channel of dinner we can in this fictional by index number. So she don't want to 3 we can get their channel is channel object actually has some pain, just like the input game that you can go and change on the channel or you have a stages you can access the freak you multiband compressor prosecute or limited opening days till you can actually grab that you bought it used to look exactly the same from the API levels on the Viewpoint and you can go and change the parameters of the Q or change the parameters in each. One of the monster is going very deep there so we can go

from we can go from Back to the issue from Channel and then the collection of channels. So we have access to all the guns. We want them. The same thing for the multiband compressor we can access the multiband compressor on the side of the channel and then access the parameters that we have in each one of the banks will have banks in the limit of all the parameters there and we can access that so now I'm going to die a little bit more on each State's type. And what are they they accessories that would have the status and get us that we have to die. First Day

Collide most people are familiar with the equalizer but it wasn't weather in equalizer. That is not very quickly. When an equalizer. We are going to specify different frequency bands. We are going to have bangs from 0 to 5 on the 3rd. So I found it to 3003 thousand to 7000 and we are going to change the energy level or the game that we can finish one of those banks in here. We can set the bank anyway that we want so we can have many banks and we can have cutoff frequency is any characteristic that we want in this graph is kind of is leaving because all of them are the same a

week does not rule you can do any weights that you want and one side you set up the banks you can go and weed accessories that we have here we can get the game. What is the current gain or we can set the time we can change out of this also real time. We are going to a dent the multiband compressor. So first describe someone was pointing out your only showing one band in the multiband compressor. And yes, I'm showing One bank, but this is this is easy to show what the motive and compressors that has Brian manchin and we have mentioned when you have a compressor on multiband

compressor. The main goal is to take sounds that are really loud and make them softer and sounds that are very soft and making love the way that we have feared represent as we have the same sentence in the before we have about the sentence that is loud medium and soft in the top. And after we apply the processing of the multiband compressor is going to look like they after that is the last song is a little bit softer than they didn't want to stay medium and the self go stroller. The way that I move to ban compressor works of the parameters work a very lonely for a second or this

graphic is we have the input. What is the level of the of the signal ending in the horizontal axis? And the numbers are from -100 to 10 a day lower. The number like - 50 is way softer than May minus stand for example for my assistant will be louder loudest sound the compressor. What is doing this is analyzing the input and if the input is above the threshold we have a treasure pod is going to decide to say, you know, that's too loud. I'm going to bring you down. But if it happens to be below the threshold is not

touch. So for example in this graphic, it would choose - 40 people was - 40 level exactly Montessori. We are below the treasure. We're not going to modify that signal but The Treasure of here is around 25 - 25 I choose a value of -20 the actual output will be around a list of -10. So that's better - 10 as an input will math 2A - 22 the album then read that the sound became soft them just because he was above the treasure. With all these parameters with the compression ratio the treasure and add the game on the game that you put game gel to gain at the other parameters. We are able to do what we

have been promising that we are able to text house on Megan South allowed but take love songs and make them softer. And that's what a multi-band compressor is doing and is doing this in different bands. We can take Regents of frequency from 0 to 1000 curse and then and be safe in a different way that the region from 1000 to 5000 will be here. We are giving you all that power to control. Double-teaming compressor being one of the most complex ones you can get the settings and get it so you can get the exact time release time ratio all

these parameters that you need to actually come feed a compressor and a Jew disturbance. Everybody's asking why you have a free game Apples game. If you moved even compressor is before and after an equalization and it's true they're doing they're doing pretty much exactly the same but we did that for one reason. We talked to a lot of developer some people that actually have a lot of these algorithms in real life and some camp songs like a big portion of them

did really like to have an equalization and then a multi-band compressor to do some tuning of the microphone something speakers some of them want like to have a multi-band compressor and then an equalization stays so they easiest ways for us to make these effects vary. What's to a building somewhere. Nancy and do the API in such a way that they can easily for solder are grains to this. The last stage is the limiter delimiter is pretty much the same. The multiband compressor, but he's a single man so we don't have multiple bands and what this is doing is

pretty much the same is taking a shower and he's way too loud. If it's about certain level in the Treasure is ours going to squash that down. He's very useful and it's usually found at the end of any audio processing chain. Does it mean to protect the speakers? You don't want that the processing did something funky and the sound is really loud and you don't want that loud sound to come out of the speakers. So you go and tell him it is going to say, okay. Sorry. This is too loud. I'm not going to creep but I'm just going to squash it down. So it doesn't destroy the speaker. So the limiter is

really good and I promise to mention something about the link the limit inside link limiters and we have the link group. When we have multiple channels listen to We have two channels. We want a one of the channels are there in the same group and one of the channel to the last Channel something real loud having the channel is very very loud. If we squeeze all down only the Left Channel the stereo you much will shift all the way to the right because this the right one did nothing but if they are linked and for some reason one of the channel to go to loud sound both will come down

sooner than Isley and the stereo signal the stereo you match a rule notebook. It will stay in the center of the state where they needed to be so being able to control the link groups in a stereo signal or in a multi-channel environment. They said you want to link the stereo speakers the surround speakers in a different group so you can keep a juul spaceship image intact and there was another future I was asked for us to do. So someone comments about that. I'm supposed to have sex first one the real time controls all the controls and API that I try to summarize in here but

is very big. I will encourage you to go on with the API on the oldest controls. Most of them are real time controls you can use them and why is that your effect is running you can go and change pretty much any pedometer that you want. You can change the level think you can change the past. They release the ratio anything that you want in any of the channel seen any of the banks you can also use pretty much the same API for instance station. Sometimes you want to create an effect on how did Roy get to go in musically as soon as it starts playing is playing with all the parameters so it

will work like that. We also have some implementation. I just want to be called a favor frequency affect favorite time sometimes for this effects. If you have a frequency domain implementation is good because you can have very Precise frequency is bleaching the bands that you may need for Uruguay. I don't think that we are all so give me like the desert desire frame size a hint that you can give today audioengine today. If i n g to say, well, you know, I like frequency and I would like to have frames of about 10 milliseconds to seconds for

your processing. So also is not a guarantee. You can feel the energy. Hey, I will like to do that. Another word for something with this effective is built-in into a USB to USB Android p a s p is sleeping with this effect. If I am really wants to come with a snap your face there is nothing implementation or something. They are encouraged to do that. And if we have all them people and aliens, please talk to me after my spot. Yes, go ahead and do this and the only thing is

like, please do very cool effects with this. I'm very good quality effects thing is this effect is available for developers. You can go on implemented in your application that you have a media player. You have a music application you just go and use the Android media only affects Dynamic process and it will be available for you there in the NFL. So summary today, we learn a bunch of things. The first one we introduce the new fish of the sound amplifier. Then himself is a linear system data feature eat uses only the

smartphone and a set of headphones and feels you improve your listening experience Bryan show you a couple of very cold air mass and thank you for the same people that they turn off the AC at the moment so we could hear things a little bit better. We learn about the healing treasure and the healing treasure ships and how they power the way that the son of the fireworks on the last thing a we show that I am expressing affect the cold how the stages work and how can you go and create your own Dynamic process in effect in urine? So with that we want to give

thanks to a lot of people that were involved in this project. A Bryant's team is the Android accessibility my team down to Media franc worth in their research team at Google the song of the standing thing inside face that they are helping Plymouth applications that you saw today, and I want to thank you for staying still a the last day of Ohio on a please contact us if you have any questions

Cackle comments for the website

Buy this talk

Access to the talk “Sound Amplifier and the new Dynamics Processing Effect”
Available
In cart
Free
Free
Free
Free
Free
Free

Access to all the recordings of the event

Get access to all videos “2018 Google I/O”
Available
In cart
Free
Free
Free
Free
Free
Free
Ticket

Interested in topic “Software development”?

You might be interested in videos from this event

September 28, 2018
Moscow
16
159
app store, apps, development, google play, mobile, soft

Similar talks

Melissa Barnhart
User Experience (UX) Researcher on Android at Google
+ 4 speakers
Maya Ben Ari
Product Manager at Google
+ 4 speakers
Patrick Clary
Product Manager at Google
+ 4 speakers
Brian Kemler
Product Manager at Google
+ 4 speakers
Victor Tsaran
Technical program manager at Google
+ 4 speakers
Available
In cart
Free
Free
Free
Free
Free
Free
Marcelo Ferreiro
Director of Engineering at Jam City
+ 3 speakers
Benjamin Frenkel
Product Manager at Google
+ 3 speakers
Kyle Rushton
Director of product at Zynga
+ 3 speakers
Olivia Yu
Vice President at King
+ 3 speakers
Available
In cart
Free
Free
Free
Free
Free
Free
Vince Wu
Product Manager at Google
Available
In cart
Free
Free
Free
Free
Free
Free

Buy this video

Video

Access to the talk “Sound Amplifier and the new Dynamics Processing Effect”
Available
In cart
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
558 conferences
22059 speakers
8190 hours of content