Signal Processing and ML Inference on the Edge

I was giving an (online) talk last week at the C++ Meetup in Karlsruhe. The title of my talk was Signal Processing and ML Inference on the Edge where I talk a bit about the constraints and requirements for on-device online computing and some interesting approaches on how to implement a given algorithm using either the (already deprecated) RenderScript as well as Halide.

At the beginning of this talk I did recapitulate some essential audio processing basics and also introduced some typical ML related computations before discussing different approaches using Halide for my main example. There was even a live stream on YouTube available for this event (which is a first for me) that you can find here:

Following is the Agenda of this talk used for the announcement:

Agenda

The recent decade has been revolutionary regarding many technical aspects of our daily lives. We have experienced the transition from simplistic feature phones to fully featured little supercomputers carried along in our pockets, capable of performing uttermost computationally heavy tasks directly on the device. Moreover, we are currently already in the midst of what some may call the third AI renaissance, and are now able to solve pattern recognition problems with relatively little effort that in the past could only be solved by humans. The latter development has been a game changer for a variety of applications including anomaly detection, classification or speech recognition.

Despite the advanced computational resources of modern mobile devices, it is still challenging to obtain optimal throughput and minimal latency with signal processing implementations, which typically involves exploiting device specific acceleration techniques, such as vector intrinsics. However, this is not always feasible, especially when targeting a large variety of different architectures and target devices.

We will look at heterogeneous computing frameworks to accelerate the processing of otherwise performance intensive tasks which will get optimized across multi-core CPUs, GPUs or DPSs on the target device. Since Androids RenderScript is deprecated, we will focus on alternatives namely Halide.

I’m also available for software consultancy, training and mentoring. Please contact me, if you are interested in my services.