Home > Tracks > DSP & Machine Learning

The Past, Present, and Future of Embedded Machine Learning

Pete Warden - Watch Now

Pete Warden, from Google's TensorFlow Lite Micro project, will be talking about how machine learning on embedding devices began, and where it's heading. ML has been deployed to microcontrollers and DSPs for many years, but until recently it has been a niche solution for very particular problems. As deep learning has revolutionized the analysis of messy sensor data from cameras, microphones, and accelerometers it has begun to spread across many more applications. He will discuss how voice interfaces are leading the charge for ML on low-power, cheap devices, and what other uses are coming. He'll also look into the future of embedded machine learning to try to predict how hardware, software and applications will be evolving over the next few years.

M↓ MARKDOWN HELP
italicssurround text with
*asterisks*
boldsurround text with
**two asterisks**
hyperlink
[hyperlink](https://example.com)
or just a bare URL
code
surround text with
`backticks`
strikethroughsurround text with
~~two tilde characters~~
quote
prefix with
>

KVas
Score: 1 | 3 weeks ago | 1 reply

Hello Pete,
Thanks for the talk!.. Quite inspiring for me after listening to the Jan Jongboom and Your talk... Really looking forward to look into ML.. was wondering how to step into this field.. now that it is Embedded and ML, feels more closer and relevant... Have been into just embedded so kind of the opposite spectrum of what you mentioned, have not much idea about ML... wanted to ask, How much of machine learning knowledge would be required to start off working on TinyML / AI on the edge in general and any good starting projects?

petewardenSpeaker
Score: 0 | 3 weeks ago | 1 reply

That's a good question. We've tried to set up the examples so that you don't need any ML experience to make progress. You can check out the first six chapters of the TinyML book for free at https://tinymlbook.com/ if you'd like to see if it works for you.

KVas
Score: 0 | 3 weeks ago | no reply

Thanks for the reply... Luckily have a subscription at O'reilly.. Already added it to my bucket list after listening your talk..

DanOje
Score: 1 | 3 weeks ago | 1 reply

Hello Pete,
This was really a wonderful session for a newbie. Gave me a wonderful insight into what embedded ML is all about. Thanks so much.

petewardenSpeaker
Score: 0 | 3 weeks ago | no reply

Thanks for the kind words Dan! I look forward to seeing what you build.

daniel
Score: 1 | 2 months ago | 1 reply

Hello Pete,
thanks for the great talk!
Do you know of interesting projects using embedded ML in radar applications?

petewardenSpeaker
Score: 0 | 1 month ago | no reply

I don't know of any public projects, but I have seen some fascinating applications using radar sensors during the course of my work. It's definitely possible, I wish I had more documentation to share!

msaur
Score: 0 | 2 months ago | 1 reply

Checking the TensorFlow Lite (https://www.tensorflow.org/lite/microcontrollers) and it requires 32-bit microcontroller platform.
From your experience in creating TFLit, are there any limitations/restrictions on supporting it on 16-bit platforms? Or are there some recommendations on how to use it for 16-bit uC if I want to try?
In fact, for cost reasons, a big portion of 200billion+ devices would really be 16-bit (or even 8-bit).
Thanks

petewardenSpeaker
Score: 0 | 2 months ago | no reply

We have had reports of people successfully building the library for 16-bit microcontrollers. The biggest requirement is that the compiler toolchain supports modern C++ (at least C++11), but as long as that's met you should be able to port.

EEngstrom77
Score: 2 | 2 months ago | 1 reply

Pete we are glad to have you part of the Embedded community! Thank you for the gift of TinyML. I at times experienced some irrational guilt over sensor data that either was not collected or collected and having no idea what to do with it. ML still looks like a black box to me but excited to see how we can use it with our embedded sensor to revision the world around us.

petewardenSpeaker
Score: 0 | 2 months ago | no reply

Thanks for your kind words! Embedded ML can be a lot of fun, so I look forward to seeing what projects you create with it.

Pi
Score: 1 | 2 months ago | 1 reply

You mention how math-intensive these algorithms are, and I'm reminded of this article from DSP Related about how DSPs are perfectly suited for ML but chip manufacturers failed to capture that market.
What, if any, DSPs have optimized libraries for TinyML (or similar platforms)? Do you think they have an advantage over ARM and other general purpose chipsets, or is it application specific?

janjongboom
Score: 1 | 2 months ago | no reply

I'm not Pete, but I saw that this one went answered. We're starting to see them in the market now, e.g. Eta Compute ECM3532 has a separate DSP where they run their neural networks, which should be available publicly this year.

alexanderentinger
Score: 2 | 2 months ago | 1 reply

Arduino listening in and appreciating the friendly mention ;)

petewardenSpeaker
Score: 0 | 2 months ago | no reply

Happy to help spread the word, it's been great collaborating!

DaveN
Score: 2 | 2 months ago | 1 reply

Please recommend a good platform for experimentation (micro eval board with audio and enough processing/ram/etc)?

petewardenSpeaker
Score: 0 | 2 months ago | no reply

I recommend picking up an Arduino Nano BLE 33 Sense, since it has a microphone and IMU, is easy to program, and only costs $30. I'm using that for workshops myself, since it's easy to show people how to get running.

DaveN
Score: 3 | 2 months ago | 1 reply

Is training on embedded hopeless?

petewardenSpeaker
Score: 0 | 2 months ago | no reply

Not at all! That is a whole different talk that I need to do, but the summary is that there are all sorts of possible ways of training. The simplest is to just "personalize" by tuning the results of the last layer to match, for example, a particular user's voice on a wakeword model. Transfer learning can also be very powerful. Full backprop training isn't usually that useful on embedded models though, because you generally need a lot of labeled data, which you often don't have.

deltronics
Score: 1 | 2 months ago | 2 replies

Please do you know of any benchmarking applications and reference results for Tensorflow Lite for Microcontrollers i.e similar to benchmark_model from TFlite? Also, are there any Tensorflow Lite for Microcontroller models in place similar ( probably customized or cutdown) to existing standard models such as mobilenet, alexnet etc.

lorenzorizzello
Score: 0 | 2 months ago | 1 reply

hi, hope you don't mind if I join the conversation. I'll let Pete reply about benchmarking, but concerning standard models it depends on model size and type of operations (here the list of operations currently supported by TF lite micro). I have successfully used a pre-trained mobilenetv1, if you want, you can check my talk where I talk a bit about it.

deltronics
Score: 1 | 2 months ago | no reply

Thanks , I will check your talk.

petewardenSpeaker
Score: 0 | 2 months ago | 1 reply

We have started on benchmarking, you can see the initial two models we're focused on at https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/micro/benchmarks
One of them is a MobileNet v1 0.25 model for person detection, and we also have that in the examples folder.
We are working with the MLPerf group to expand and formalize the benchmarks, and we'd welcome help if you're interested.

deltronics
Score: 0 | 2 months ago | no reply

I had a quick look at this in the past but thought they were customized speech models for TLFM. I will have another look. Thank you.

PatrickMcDaid
Score: 5 | 2 months ago | 2 replies

Think your webcam may have frozen. didn't get to see that cat Dude. ;-)

petewardenSpeaker
Score: 0 | 2 months ago | no reply

I realized that, I think his cuteness overloaded the camera! :)
Here's an old video that's all about him though: https://petewarden.com/2014/04/08/how-to-add-a-brain-to-your-smart-phone/

ThomDenholm
Score: 0 | 2 months ago | no reply
This post has been deleted by the author
nicastellanos
Score: 1 | 2 months ago | 1 reply

Hi Peter, thanks for the session! Whats your opinion on the future of embedded applications in an Reinforcement Learning space. Do you see a possibility of using these technologies for edge RL agent systems and if so what advancements do you think will need to be made to make these systems possible. Thanks!

petewardenSpeaker
Score: 1 | 2 months ago | no reply

To be honest, I've not used RL myself, mostly because it seems so hard to get working well! I've seen good results in papers, but colleagues who've dabbled in it have found it very "touchy" and sensitive to initial conditions. The underlying math seems possible to support on embedded devices, but I've not focused on that area because getting the models working well for practical problems seems so tough. I'd love to hear about others' experiences in this area though.

JeanGoulet
Score: 4 | 2 months ago | 1 reply

Interesting point of view on how the Internet and Wifi in IoT, is a burden for the battery. I wonder to what extent 5G will change that (make it worse? better?). And yes indeed how comical yet sad that so many simple devices fail to operate (motion sensing lights that are worthless even though people are present; no reaction from so many defective motion-sensing faucets and soap dispensers and hand dryers). Are they mostly because of dead batteries? Or mostly terrible products? Hopefully, ML products of high quality, high intelligence, high longevity, and low cost, will one day change little things, and big things, for the better.

petewardenSpeaker
Score: 0 | 2 months ago | no reply

That's a great perspective, I am definitely interested in improving the everyday experience of users, and I do think ML can be more reliable than existing methods for things like faucets and lights. It will be up to us to make sure we get the engineering right of course, since ML's no magic bullet, but at least it gives us new options.

OUR SPONSORS