Home > On-Demand Archives > Talks >
The Past, Present, and Future of Embedded Machine Learning
Pete Warden - Watch Now - Duration: 45:30
Pete Warden, from Google's TensorFlow Lite Micro project, will be talking about how machine learning on embedding devices began, and where it's heading. ML has been deployed to microcontrollers and DSPs for many years, but until recently it has been a niche solution for very particular problems. As deep learning has revolutionized the analysis of messy sensor data from cameras, microphones, and accelerometers it has begun to spread across many more applications. He will discuss how voice interfaces are leading the charge for ML on low-power, cheap devices, and what other uses are coming. He'll also look into the future of embedded machine learning to try to predict how hardware, software and applications will be evolving over the next few years.
That's a good question. We've tried to set up the examples so that you don't need any ML experience to make progress. You can check out the first six chapters of the TinyML book for free at https://tinymlbook.com/ if you'd like to see if it works for you.
Thanks for the reply... Luckily have a subscription at O'reilly.. Already added it to my bucket list after listening your talk..
Hello Pete,
This was really a wonderful session for a newbie. Gave me a wonderful insight into what embedded ML is all about. Thanks so much.
Thanks for the kind words Dan! I look forward to seeing what you build.
Hello Pete,
thanks for the great talk!
Do you know of interesting projects using embedded ML in radar applications?
I don't know of any public projects, but I have seen some fascinating applications using radar sensors during the course of my work. It's definitely possible, I wish I had more documentation to share!
Checking the TensorFlow Lite (https://www.tensorflow.org/lite/microcontrollers) and it requires 32-bit microcontroller platform.
From your experience in creating TFLit, are there any limitations/restrictions on supporting it on 16-bit platforms? Or are there some recommendations on how to use it for 16-bit uC if I want to try?
In fact, for cost reasons, a big portion of 200billion+ devices would really be 16-bit (or even 8-bit).
Thanks
We have had reports of people successfully building the library for 16-bit microcontrollers. The biggest requirement is that the compiler toolchain supports modern C++ (at least C++11), but as long as that's met you should be able to port.
Pete we are glad to have you part of the Embedded community! Thank you for the gift of TinyML. I at times experienced some irrational guilt over sensor data that either was not collected or collected and having no idea what to do with it. ML still looks like a black box to me but excited to see how we can use it with our embedded sensor to revision the world around us.
Thanks for your kind words! Embedded ML can be a lot of fun, so I look forward to seeing what projects you create with it.
You mention how math-intensive these algorithms are, and I'm reminded of this article from DSP Related about how DSPs are perfectly suited for ML but chip manufacturers failed to capture that market.
What, if any, DSPs have optimized libraries for TinyML (or similar platforms)? Do you think they have an advantage over ARM and other general purpose chipsets, or is it application specific?
I'm not Pete, but I saw that this one went answered. We're starting to see them in the market now, e.g. Eta Compute ECM3532 has a separate DSP where they run their neural networks, which should be available publicly this year.
Arduino listening in and appreciating the friendly mention ;)
Happy to help spread the word, it's been great collaborating!
Please recommend a good platform for experimentation (micro eval board with audio and enough processing/ram/etc)?
I recommend picking up an Arduino Nano BLE 33 Sense, since it has a microphone and IMU, is easy to program, and only costs $30. I'm using that for workshops myself, since it's easy to show people how to get running.
Is training on embedded hopeless?
Not at all! That is a whole different talk that I need to do, but the summary is that there are all sorts of possible ways of training. The simplest is to just "personalize" by tuning the results of the last layer to match, for example, a particular user's voice on a wakeword model. Transfer learning can also be very powerful. Full backprop training isn't usually that useful on embedded models though, because you generally need a lot of labeled data, which you often don't have.
Please do you know of any benchmarking applications and reference results for Tensorflow Lite for Microcontrollers i.e similar to benchmark_model from TFlite? Also, are there any Tensorflow Lite for Microcontroller models in place similar ( probably customized or cutdown) to existing standard models such as mobilenet, alexnet etc.
hi, hope you don't mind if I join the conversation. I'll let Pete reply about benchmarking, but concerning standard models it depends on model size and type of operations (here the list of operations currently supported by TF lite micro). I have successfully used a pre-trained mobilenetv1, if you want, you can check my talk where I talk a bit about it.
Thanks , I will check your talk.
We have started on benchmarking, you can see the initial two models we're focused on at https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/micro/benchmarks
One of them is a MobileNet v1 0.25 model for person detection, and we also have that in the examples folder.
We are working with the MLPerf group to expand and formalize the benchmarks, and we'd welcome help if you're interested.
I had a quick look at this in the past but thought they were customized speech models for TLFM. I will have another look. Thank you.
Think your webcam may have frozen. didn't get to see that cat Dude. ;-)
I realized that, I think his cuteness overloaded the camera! :)
Here's an old video that's all about him though: https://petewarden.com/2014/04/08/how-to-add-a-brain-to-your-smart-phone/
Hi Peter, thanks for the session! Whats your opinion on the future of embedded applications in an Reinforcement Learning space. Do you see a possibility of using these technologies for edge RL agent systems and if so what advancements do you think will need to be made to make these systems possible. Thanks!
To be honest, I've not used RL myself, mostly because it seems so hard to get working well! I've seen good results in papers, but colleagues who've dabbled in it have found it very "touchy" and sensitive to initial conditions. The underlying math seems possible to support on embedded devices, but I've not focused on that area because getting the models working well for practical problems seems so tough. I'd love to hear about others' experiences in this area though.
Interesting point of view on how the Internet and Wifi in IoT, is a burden for the battery. I wonder to what extent 5G will change that (make it worse? better?). And yes indeed how comical yet sad that so many simple devices fail to operate (motion sensing lights that are worthless even though people are present; no reaction from so many defective motion-sensing faucets and soap dispensers and hand dryers). Are they mostly because of dead batteries? Or mostly terrible products? Hopefully, ML products of high quality, high intelligence, high longevity, and low cost, will one day change little things, and big things, for the better.
That's a great perspective, I am definitely interested in improving the everyday experience of users, and I do think ML can be more reliable than existing methods for things like faucets and lights. It will be up to us to make sure we get the engineering right of course, since ML's no magic bullet, but at least it gives us new options.
Hello Pete,
Thanks for the talk!.. Quite inspiring for me after listening to the Jan Jongboom and Your talk... Really looking forward to look into ML.. was wondering how to step into this field.. now that it is Embedded and ML, feels more closer and relevant... Have been into just embedded so kind of the opposite spectrum of what you mentioned, have not much idea about ML... wanted to ask, How much of machine learning knowledge would be required to start off working on TinyML / AI on the edge in general and any good starting projects?