XMOS recently announced its all-new software development kit (SDK) for the artificial intelligence of things (AIoT). Incorporating TensorFlowLite for Microcontroller development tools, the SDK harnesses the versatility of xcore®.ai, making it easier for engineers to develop connected products that can sense, think, decide and act.
In this blog post we delve deeper into the components of the SDK, and highlight how it helps developers deploy custom or off-the-shelf AI models in a complete and secure application solution.
AI TOOLCHAIN EXTENSIONS
The AIoT SDK contains extensions to the XMOS toolchain, designed to support the deployment of a trained neural network model to the edge.. Included in these extensions is the xformer utility, which runs offline and converts a TensorFlow Lite model file into a model that is optimised for inference on xcore.ai. It does this by making many optimisation passes over the model – each pass looking for opportunities to minimise computational overhead and SRAM usage. There is no need to think about how to parallelise inference – the xformer takes care of that. When the xformer is finished with the model, it is ready to take full advantage of xcore.ai’s vector processing unit (VPU) and multi-core architecture.
The extensions are designed with both model designers and embedded software developers in mind. For model designers, the tools can be used to generate a library that can easily be integrated into an xcore.ai application. For embedded software developers, the tools can be used to generate source code from the optimised model. Source code that can easily be added into any embedded application project.
FAST ARITHMETIC & DSP LIBRARY
Neural network inference is not the only thing that xcore.ai can accelerate. Included with the SDK is a new arithmetic and DSP library which leverages the VPU to accelerate costly operations on vectors of 16- or 32-bit data. This includes operations such as add, subtract, multiply, sum, dot product, FFT, IFFT, and more.
The library has a low-level API that acts as the workhorse for the library. It stays close to the bare-metal and requires some knowledge of the VPU instruction set to use. Hard-core embedded developers will appreciate the power the low-level API gives them.
The library also comes with a high-level API, designed to be intuitive and easy to use. The high-level API is a block floating-point (BFP) API for performing accelerated operations. Each BFP vector is represented by a struct. The structure representing a BFP vector carries information about the vector’s data buffer and length as well as the vector’s exponent and headroom. The high-level API functions manages the headroom and exponent of the BFP vectors to determine when and how to best utilise the low-level API to maintain as much arithmetic precision as possible in the data.
MULTI-CORE FreeRTOS
xcore® has always been known as a platform that is well suited for hard real-time applications. xcore.ai builds on this tradition. Bare-metal application development is still a viable option for developers because not every application is appropriate for an RTOS. However, most are appropriate, including applications requiring real-time guarantees. This makes our FreeRTOS port a significant component of the AIoT SDK.
The standard FreeRTOS task scheduler gives the illusion of executing multiple tasks simultaneously. It allows for having more tasks than cores, but only one task is running at a given time. Our symmetric multiprocessing (SMP) implementation of the task scheduler takes advantage of xcore’s multi-core architecture, making executing multiple tasks simultaneously a reality. You decide how many cores to allocate to the task scheduler, create and prioritise your tasks, and the FreeRTOS kernel does the rest.
A customisable collection of soft peripheral I/O devices is provided with the FreeRTOS port. These peripheral devices run on bare-metal cores alongside the core(s) allocated to the FreeRTOS kernel and scheduler. Board support packages (BSP) provide software drivers for these peripherals that can be used by your FreeRTOS application. A BSP is provided with each xcore.ai Dev Kit.
FreeRTOS now makes it possible to program multi-threaded applications on XCORE exclusively in C. XC is still supported, but you no longer need to learn a new programming language to develop for the XCORE platform.
This blog touches on few of the exciting new software components we have for XCORE developers. We will go into more detail on additional new components in future articles, including, a voice front-end library that you can integrate into your applications. We are excited to make the heart of our VocalFusion® voice processing products available to developers.
The XMOS AIoT SDK will be available soon on GitHub. It is designed to be used in conjunction with the xcore.ai Explorer Kit, currently on limited release via xmos.com.
