Placeholder Image

Subtitles section Play video

  • Can you hear me?

  • Let's get started.

  • My name is Ahn Jae Man, and in this session

  • I am going to give a presentation about the Life-saving AI and Javascript.

  • Nice to meet you.

  • I work at AITRICS, a medical AI corporation, and I use AI

  • to predict the acute and critical diseases and emergencies of the inpatients,

  • And report the results to the medical team and help them to prepare for such incidents

  • We are provide solutions for such events.

  • Today I am here to walk you through the process and issues during the development of such solutions.

  • The topic itself is very broad.

  • When we say medical technology, it's a very broad term, same applies for the AI.

  • Likewise, there's a lot to talk about Javascript.

  • Our topics are extensive, and since I have such a limited time of 30 minutes,

  • I might briefly touch on some subjects on the surface level

  • On some topics, I may lack depth in my explanation.

  • and it might be hard to understand.

  • I apologize for such circumstances.

  • If you want further information

  • or if you would like to discuss some issues in depth, you can ask me,

  • or you can come talk to me in person later after the presentation,

  • I'll gladly discuss with you later on.

  • So in this session, what I would like to talk about is

  • The field of medical AI is not familiar with most people.

  • A lot of you might have heard of artificial intelligence so you'll might already be familiar with that.

  • but medical AI would be something most of you haven't heard of. So I will introduce you about this first

  • and then I will go over how to develop AI solutions,

  • the process of such development.

  • And then related to this topic, we are using the JS, so I will talk about Javascript related techniques

  • We use a lot of languages like Javascript, Python, and Go,

  • Today I would like to focus on the Javascript related issues.

  • And because the AI is different from regular softwares,

  • I will discuss several issues in relation to that.

  • This is the contents for today, and now I will start my presentation.

  • I will introduce the medical AI.

  • What if your code could save someone's life?

  • What if the code you created

  • Can contribute to raising the survival rate, even just 1%,

  • Your code will be able to save the life of one person out of a hundred.

  • In the U.S., 250,000 people lose their lives annually because of medical malpractice.

  • Now if your code may contribute to reducing such malpractice,

  • and increase the survival rate, even if it's just one percent,

  • You will be able to save 2,500 people annually, out of the 250,000.

  • Isn't it fascinating?

  • Now, I will talk about how the software engineering

  • and AI can save people's lives,

  • So one of the solutions we are working on right now is this:

  • This is a solution that predicts acute diseases in advance

  • and report the results to the doctors.

  • So what are the acute diseases and emergencies?

  • It can be death itself,

  • and an unexpected cardiac arrest,

  • and sepsis, or blood poisoning, which is not that widely known.

  • And many other diseases.

  • It may be hard to understand what sepsis is,

  • the sepsis refers to the same thing as blood poisoning.

  • So what exactly is it?

  • One of the common examples may be,

  • Let's say someone was bitten by a dog, and he was taken to the hospital, but passed away within a day.

  • Or,

  • Someone had an injection, like glucose solution, and suddenly passed away.

  • These are all cases caused by sepsis.

  • So sepsis, among many acute diseases,

  • induces the most deaths and incurs the highest cost,

  • And what's serious about it is, with each hour of delayed treatment,

  • you get 8% higher death rate.

  • More than 50% of deaths in the hospitals

  • are related to sepsis or blood poisoning.

  • Then if we can predict this sepsis and report it in advance,

  • we can intuitively understand that so many people can be saved.

  • So now we are working on sepsis,

  • the solution for predicting sepsis outbreaks.

  • Now I will introduce the development process for such a solution.

  • The overall process is shown in this figure.

  • The data of the patients, such as the heart rate, pulse, blood pressure, and body temperature,

  • and the blood test results, or X-ray, CT scan image, and prescriptions

  • So we collect all these,

  • and when we enter it into a machine learning model, and this model will, in 4 hours to 24 hours,

  • Calculate the probability of death or sepsis occurrence for the patient

  • Such results will be displayed on the dashboard,

  • Which is very simple.

  • It's a simple solution.

  • So this is the solution we are distributing right now, and

  • the mortality, or death, this solution decides that the person is risky of death in six hours,

  • and it notifies of the risk, and its evidences,

  • It offers the view of those information, such as the current state of the patient.

  • I summarized the process for AI solution development into five phases.

  • This is not the official 5-phase,

  • I thought it'd be easier for me to explain if I divide it into 5 phases.

  • I will walk you through each phases

  • First is data refining and model building,

  • this is an essential phase in building an AI.

  • You collect a lot of data

  • As you collect a massive amount of data,

  • you might encounter with strange data, or useless data,

  • Or irrelevant data.

  • So we do data cleansing and pre-processing.

  • And then for the outcome,

  • We have to define what we are going to predict.

  • When we predict death, death is obvious,

  • You would think, death is death,

  • You might jump into the conclusion that it's very simple to define the outcomes.

  • However, it is really complicated since we are predicting acute death

  • and we have to define what acute death is.

  • So the outcomes could be classified into many different definitions.

  • Let's say that the patient dies after 24 hours,

  • if you would like to predict this,

  • what about the patient who died after 25 hours?

  • What about 23 hours? These are all the issues that we have to keep in mind.

  • We think about those various circumstances,

  • and we struggle with many different data,

  • And we come up with the model that fits the data best.

  • This is what machine learning research usually does, and actually that's why

  • data refining and such takes up most of the time.

  • I would like to talk more about this process,

  • but I will skip this part for now.

  • And now the model, we developed this model,

  • and we have to deploy it,

  • and this is how the deployment is conducted.

  • Microservices, for example, Web API, most commonly,

  • or Rest API, we deploy models with these,

  • now that we have tensorflow JS, which enables us to

  • upload the model in the web browsers.

  • So with the tensorflow JS, we can deploy them on web browsers,

  • And of course processes like optimization for faster models, and compression, are necessary.

  • But in the deployment process, if we actually deploy them, it is reflected in the service,

  • so you have to make sure about this.

  • On my slide, it says it's about the safe usage of AI models,

  • but actually AI models do not always give the right output.

  • Even though it has 99% of accuracy,

  • It means it's accurate for 99 people out of 100, but it will be wrong for one person.

  • And this inaccuracy for one person is critical in many cases.

  • So what if the model suggests wrong output?

  • That's one issue,

  • And then when we have the output from the model,

  • why did the model produce such output?

  • How can we trust this output? This would be another issue.

  • So you have to defend such problems or consider such issues

  • to use the AI model safely.

  • So the drawback of the deep learning models is

  • They are very good at predicting the data it already had as the input

  • However, it cannot really predict well about the data it never learned.

  • For example, a deep learning model which distinguishes between dogs and cats,

  • if you put in a picture of a tiger, this model would,

  • It should say "this is neither a dog nor a cat"

  • but instead it will say that "it is a cat."

  • So when we develop machine learning models,

  • we test all of the possible inputs,

  • and even if there's some malfunctioning in the model

  • We need a lot of code to defend such cases in the solution phase.

  • If we provide machine learning models for this service,

  • we are usually concerned about these kind of things

  • so we need the test results of machine learning models for some random input values

  • This is usually called the property based testing

  • you test the model with random inputs

  • and observe what kinds of inputs fail the test

  • The test process for this is called property-based testing.

  • We conduct the property-based testing with Javascript

  • And the libraries like JS Verify or Fast Check supports this.

  • additionally, when the model produces an output,,

  • the issue of how can we trust the output,

  • It's about how accurate the results are.

  • So we have added a module that can be analyzed,

  • and so we were able to analyze and debug the module

  • If you look at the image on the right,

  • the image on the right bottom will be more intuitive

  • Here's a chest x-ray

  • Let's say I want to know if this person's health condition is in danger or not by looking at this x-ray

  • If you give this two images to the model,

  • and it says, 'the x-ray shows that he is in danger'

  • Then I need to know the exact reason why

  • to see if the model's answer is correct or not.

  • By putting an interpretable module, as we did for the right image,

  • it will tell us the reason why it's saying that. And we can see if it's working well or not.

  • So if we add an interpretable module,

  • we can use a much stabilized AI module.

  • Thus, when using interpreter modules

  • it's important to visualize it well when using the AI model.

  • And if I add some more explanation to the property based testing,

  • This is part of our solution code,

  • This code tests whether the people with sepsis actually

  • score higher on the sepsis risk test.

  • So when we find out the effect of sepsis,

  • the best way to diagnose sepsis is called lab blood culture feature.

  • If we give the value of 0.3,

  • the blood culture value is greater than 0.3. The rest of the value,

  • will be random values,

  • and it will automatically run a test,

  • and the result of the test must return a value greater than 0.2

  • And this shows that blood culture has some effect.

  • By testing it like this

  • we can run a machine learning module test randomly.

  • So by doing this,

  • I've introduced a way of deploying safely.

  • Next now that we have a model,

  • we have to find a way to use it with the actual data.

  • When we have data coming in live,

  • by using the calculator we have to find out the score,

  • for this we use data pipeline, like NodeJS,

  • it can be implemented very easily.

  • NodeJS supports asynchronous event,

  • when data flows in,

  • it tells the data to do a certain task,

  • so we were able to build a data pipeline easily using NodeJS.

  • During this progress, we can monitor the data flow,

  • or periodically check it's accuracy.

  • This is a part of our presentation during the PyCon.

  • So a data pipeline is something like this, when a data comes in from the hospital

  • the patients information comes in along with predictive acute disease solution.

  • By using the machine learning model, we can predict and show it on the Dashboard.

  • It will notify it in some way, such as the text message or the phone call.

  • Now you might think that we have simple data pipeline,

  • but this is actually very complicated,

  • If you look on top left corner, we can see that the patient's data comes from the hospital

  • Our synchronizer, which is a microservice, inputs the data into our database,

  • and if you look on the right side

  • you can see prediction, medical score,

  • and an alerting service

  • And if you see on top, you can see the things that must be done regularly.

  • Like backing up data,

  • a scheduler that trains the model,

  • and below that there's the monitor and controller.

  • I think we won't have enough time to go over everything,

  • so I'll start with this first,

  • I'll go back to other things later on

  • The monitor controller part

  • We use redis for this, and redis consumes a string,

  • and if we send it through StatsD

  • you can something like this on the right side of the screen.

  • You can see the medical score or EMR data.

  • By monitoring,

  • we can check if the data pipeline is working well or not.

  • As for the controller,

  • If we type in a command using slack,

  • we can check the status of the data pipeline

  • or rerun the data pipeline

  • you can do these kind of things.

  • It's easy when using node.js

  • to create this kind of things.

  • We can also use other things like python,

  • those things aren't event based,

  • so you have to check them continuously through threading, so there's a bit of a problem.

  • The next step is,

  • the most important part when making a service.

  • It's the frontend design,

  • You might think that frontend design

  • might not have any connection with machine learning

  • But actually,

  • Drawing from my experience,

  • In developing an AI model based product,

  • One of the most important parts is developing the frontend.

  • You might be wondering why,

  • So you get the output from the model,

  • It's crucial to interpret it and display it well

  • This will help you understand better.

  • The output from the machine learning model is

  • just that one number.

  • If you put the data of a patient, it will say, 0.015,

  • or 0.2, 0.03, hundreds of these numbers.

  • The doctors cannot make decisions out of these. These are not the sole decision maker.

  • If this patient gets 0.015, is she in risk or not,

  • or was she already in danger,

  • or has she developed the risk recently, these factors would be put into consideration

  • So what the medical team would like to see

  • would be the screen below here.

  • Producing that screen from the number 0.015 above, is what the frontend does.

  • It needs a lot of work and consideration

  • How should we interpret the model? We need to work on that as well,

  • For this process,

  • We draw these kind of charts a lot,

  • We check if our interpretation is actually accurate,

  • this is not the actual graph we use,

  • This is the kind of graph we use and I brought it from the E-chart docs

  • For example, the estimated score, and the actual outcome, like death and so on,

  • What was the relationship with the outcome and the input,

  • Those are drawn into graphs

  • to verify our intuition and results.

  • So when it's delivered to the patients, they warn that this is very risky,

  • like the figure is holding his heart,

  • we contemplate on the ways to deliver it more intuitively.

  • So far you might think that we finished making the AI product

  • But in fact, the AI models

  • are not always accurate when we actually put it into service.

  • It's always wrong, it is always wrong.

  • So if we dig down into the reason why it is wrong when it was successful during the development,

  • first, the data is a little different.

  • The aspects of the data is somewhat different.

  • For example, the patient in 2015 and 2019 are completely different.

  • Not completely maybe, but they are quite different. So that lowers the accuracy of the prediction

  • and people's actions are a little different as well.

  • Like when we predict sepsis

  • and report them 4 hours ahead of its predicted occurrence,

  • And the medical team reacts ahead according to it.

  • With that reaction, the pattern of the sepsis will be different,

  • And our AI model will lose accuracy because of it.

  • Because we intervene into people's behaviors and reactions, it results in less accurate prediction.

  • So the issue here is how do we make AI

  • learn from the data

  • that keeps changing.

  • We have to consider that as well.

  • You've seen this data pipeline before, right?

  • So this trainer here does that work,

  • The trainer microservice is not that big of a deal.

  • We have the original data,

  • We have data from September 1st, September 2nd, and the data keeps accumulating.

  • And we get the version 1 model from the original data.

  • If we include the data from September 1st, we have version 2,

  • and if we have less accuracy with it, we should discard it.

  • we decide to discard it,

  • And then we have other data on September 2nd,

  • and we train the model with it,

  • if we have increased accuracy with it, we decide to keep that data.

  • So we keep training the model regularly,

  • this is the microservice that enables the model to be updated automatically.

  • During such process, we verify the model with the data from other institutions

  • We keep verifying the model that it is applicable to many other hospitals and institutions

  • This described how to develop an AI product,

  • it was a primitive description.

  • And we cannot skip over tensorflow JS.

  • You might have heard a lot about the tensorflow JS.

  • This is, as you might know,

  • a library that enables machine learning with Javascript.

  • The one on the right is our code.

  • You would know it if you have used Python tensorflow,

  • the composition is very intuitive.

  • The code is not that different from the Python tensorflow,

  • So that you can use existing pre trained model

  • You can also convert and use a pre-made model with Python,

  • and it's also possible to train it on the browser or Node.js.

  • If you access the tensorflow website, you have a tutorial page,

  • and it's really good for beginners,

  • even if you know nothing about it, you can try and develop a cool product

  • So you learn about the already existing models here.

  • or you can retrain the existing models,

  • or use Javascript,

  • to develop the machine learning from the beginning, those are some things that you can do.

  • If you are interested, it would be fun to try it out.

  • There are many interesting examples.

  • You can play games, play a piano, there are a lot of fun demonstrations.

  • So, it would be good for you to try.

  • If you ask why we have to practice the machine learning on web browsers,

  • on the right, this is a patient and a doctor having a real time conversation

  • The doctor is constantly checking the patient's condition.

  • Like this, predicting something interactively in real time.

  • If you need this function,

  • getting back from the back-end is a slow process

  • and it doesn't react interactively.

  • So, if you put it on the browser using Tensorflow js,

  • very interactively,

  • you can use the machine learning model.

  • And of course, since it's ran on a web browser, the server load would be reduced.

  • And,

  • another benefit is that machine learning models could be visualized.

  • These are the advantages.

  • So we have used them a lot.

  • We were thinking of how to use the Tensorflow js.

  • Tensorflow doesn't support everything yet

  • and there are only few people using it in communities. So,

  • we made a simple model

  • and tried running them or making simple products and famous ones.

  • But we didn't try to use it for real production or anything like that.

  • When the community grows, we'll be able to try many other things.

  • So far, it's very good but it's not good enough to use right away.

  • This is a library called TensorSpace js.

  • When you run the machine learning model on a web browser, like this,

  • you can easily check how the machine learning model

  • actually moves about and what structure it takes.

  • It's also very easy to debug the machine learning model.

  • We've come to this point very quickly, now let's go back.

  • So the products we are making right now.

  • What kind of innovation could they actually make?

  • You might be wondering, are they actually saving people's lives?

  • In Korea, some hospitals are having a pilot test.

  • When I checked the feedback from the medical team, doctors and nurses,

  • there were some very good feedback.

  • For example, it helped distributing medical resources efficiently.

  • They could see the patients they couldn't see. Their work load have reduced so they could go home early.

  • Their sleeping hours increased by an hour. There were good feedbacks like these.

  • It's hard to say here whether it enhanced the survival rate or not.

  • because that has to go through a very scientific and statistical verification.

  • So it's ambiguous for me to say the exact percentage of it.

  • But based on the feedbacks, I can safely say that it is contributing to the survival rate.

  • So next, we are going to

  • show that AI solution can actually save human lives,

  • by verifying through a time-consuming experiments and observations.

  • And we should certainly save a lot of lives, right?

  • It's getting a bit cheesy so let's skip this part

  • To explain the technological details,

  • We are increasing the number of predictable diseases.

  • So that we could predict every disease that occur at the hospital.

  • We are trying to build that platform.

  • And when a medicine is injected to cure a disease,

  • predicting the reaction of the patient.

  • Medically, we are facing these types of challenges

  • To look at the machine learning, in terms of software,

  • since we are making many predictions,

  • it's hard to run all the machine learning models in the back-end.

  • We do machine learning even for the tiniest parts.

  • So we use the Tensorflow js to put it on a web browser or,

  • just by using Tensorflow api,

  • put it on mobile. We are using these processes.

  • The screen on the right shows something we have presented at Nvidea.

  • It's a platform that can accelerate machine learning research.

  • You can look up the details of the presentation.

  • Now, we are trying to

  • create a platform that can fasten the learning process of machine learning

  • We are working on that too.

  • Additionally, other than medical, we have other things to process machine learning

  • All of you, not just the people from our company, should utilize the machine learning in various fields

  • and try different things.

  • This could bring innovations that are never done before.

  • This is the end of my presentation, and the time's almost up.

  • Thank you for being such a wonderful audience.

Can you hear me?

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it