PDM Microphone Aggregator Example#
This example provides a bridge between 16 PDM microphones to either TDM16 slave or USB Audio and targets the xcore-ai explorer board.
This application is to support cases where many microphone inputs need to be sent to a host where signal processing will be performed. Please see the other examples in sln_voice where signal processing is performed within the xcore in firmware.
This example uses a modified mic_array with multiple decimator threads to support 16 DDR microphones on a single 8 bit input port. The example is written as ‘bare-metal’ and runs directly on the XCORE device without an RTOS.
Obtaining the app files#
Download the main repo and submodules using:
$ git clone --recurse git@github.com:xmos/sln_voice.git
$ cd sln_voice/
Building the app#
First install and source the XTC version: 15.2.1 tools. The easiest way to source
the tools is to open the provided shortcut to XTC Tools 15.2.1 Command Prompt
.
Running the compiler binary xcc
will produce an output like this:
xcc --version
xcc: Build 19-198606c, Oct-25-2022
XTC version: 15.2.1
Copyright (C) XMOS Limited 2008-2021. All Rights Reserved.
Linux or Mac#
To build for the first time you will need to run cmake
to create the
make files:
$ mkdir build
$ cd build
$ cmake --toolchain ../xmos_cmake_toolchain/xs3a.cmake ..
$ make example_mic_aggregator_tdm -j
$ make example_mic_aggregator_usb -j
Following initial cmake
build, as long as you don’t add new source
files, you may just type:
$ make example_mic_aggregator_tdm -j
$ make example_mic_aggregator_usb -j
If you add new source files you will need to run the cmake
step
again.
Windows#
It is highly recommended to use Ninja
as the make system under
cmake
. Not only is it a lot faster than MSVC nmake
, it also
works around an issue where certain path names may cause an issue with
the XMOS compiler under windows.
To install Ninja, follow these steps:
Download
ninja.exe
from https://github.com/ninja-build/ninja/releases. This firmware has been tested with Ninja version v1.11.1.Ensure Ninja is on the command line path. You can add to the path permanently by following these steps https://www.computerhope.com/issues/ch000549.htm. Alternatively you may set the path in the current command line session using something like
set PATH=%PATH%;C:\Users\xmos\utils\ninja
To build for the first time you will need to run cmake
to create the
make files:
$ md build
$ cd build
$ cmake -G "Ninja" --toolchain ..\xmos_cmake_toolchain\xs3a.cmake ..
$ ninja example_mic_aggregator_tdm.xe -j
$ ninja example_mic_aggregator_usb.xe -j
Following initial cmake
build, as long as you don’t add new source
files, you may just type:
$ ninja example_mic_aggregator_tdm.xe -j
$ ninja example_mic_aggregator_usb.xe -j
If you add new source files you will need to run the cmake
step
again.
Running the app#
Connect the explorer board to the host and type:
$ xrun example_mic_aggregator_tdm.xe
$ xrun example_mic_aggregator_usb.xe
Optionally, you may use xrun --xscope
to provide debug output.
Required Hardware#
The application runs on the XCORE-AI Explorer board version 2 (with integrated XTAG debug adapter). You will require in addition:
The dual DDR microphone board that attaches via the flat flex connector.
Header pins soldered into:
J14, J10, SCL/SDA IOT, the I2S expansion header, MIC data and MIC clock.
Six jumper wires. Please see the microphone aggregator main documentation for details on how these are connected.
An oscilloscope will also be handy in case of hardware debug being needed.
Note
You will only be able to inject PDM data to two channels at a time due to a single pair of microphones on the HW.
If you wish to see all 16 microphones running then an external microphone board with 16 microphones (DDR connected to 8 data lines) is required.
Operation#
The design consists of a number of tasks connected via the xcore-ai silicon communication channels. The decimators in the microphone array are configured to produce a 48 kHz PCM output. The 16 output channels are loaded into a 16 slot TDM slave peripheral running at 24.576 MHz bit clock or a USB Audio Class 2 asynchronous interface and are optionally amplified. The TDM build also provides a simple I2C slave interface to allow gains to be controlled at run-time. The USB build supports USB Audio Class 2 compliant volume controls.
For the TDM build, a simple TDM16 master peripheral is included as well as a local 24.576 MHz clock source so that mic_array and TDM16 slave operation may be tested standalone through the use of jumper cables. These may be removed when integrating into a system with TDM16 master supplied.
Software Architecture#
The applications are written on bare metal and use logical cores (hardware threads) to implement the functional blocks. Each of the tasks are connected using channels provided in the xcore-ai architecture. The thread diagrams are shown in Fig. 1 and Fig. 2.
PDM Capture#
Both the TDM and USB aggregator examples share a common PDM front end. This consists of an 8 bit port with each data line connected to two PDM microphones each configured to provide data on a different clock edge. The 3.072 MHz clock for the PDM microphones is provided by the xcore-ai device on a 1 bit port and clocks all PDM microphones. The PDM clock is divided down from the 24.576 MHz local MCLK.
The data collected by the 8 bit port is sent to the lib_mic_array block which de-interleaves the PDM data streams and performs decimation of the PDM data down to 48 kHz 32 bit PCM samples. Due to the large number of microphones the PDM capture stage uses four hardware threads on tile[0]; one for the microphone capture and three for decimation. This is needed to divide the processing workload and meet timing comfortably.
Samples are forwarded to the next stage at a rate of 48 kHz resulting in a packet of 16 PCM samples per exchange.
Audio Hub#
The 16 channels of 48 kHz PCM streams are collected by Hub and are amplified using a
saturated gain stage. The initial gain is set to 100, since a gain of 1 sounds very
quiet due to the mic_array output being scaled to allow acoustic
overload of the microphones without clipping within the decimators. This value can be
overridden using the MIC_GAIN_INIT
define in app_conf.h.
Additionally for the TDM configuration, the Hub task also checks for control packets from I2C which may be used to dynamically update the individual gains at runtime.
A single hardware thread contains the task and a triple buffer scheme is used to ensure there is always a free buffer available to write into regardless of the relative phase between the production and consumption of microphone samples.
The Hub task has plenty of timing slack and is a suitable place for adding signal processing if needed.
TDM Host Connection#
The TDM build supports a 16-slot TDM slave Tx peripheral from the fwk_io sub-module. In this application it runs at 24.576 MHz bit clock which supports 16 channels of 32 bit, 48 kHz samples per frame.
The TDM component uses a single hardware thread.
For the purpose of debugging a simple TDM 16 Master Rx component is provided. This allows the transmitted TDM frames from the application to be received and checked without having to connect an external TDM Master. It may be deleted / disconnected without affecting the core application.
Note
The simple TDM 16 Master Rx component is not regression tested and is for evaluation of TDM 16 Slave Tx in this application only.
USB Host Connection#
As an alternative to TDM, a USB host connection is also supported. The USB connection uses the following specifications:
USB High Speed (480 Mbps)
USB Audio Class 2.0
Asynchronous mode (audio clock is provided by the firmware)
24 bit Audio slots
48 kHz Sample Rate
The USB host connection functionality is provided by lib_xua which is the core library of XMOS’s USB Audio solution.
The USB Audio subsection uses a total of four hardware threads in this application.
Resource Usage#
The xcore-ai device has a total resource count of 2 x 524288 Bytes of memory and 2 x 8 hardware threads across two tiles. This application uses around half of the processing resources and a tiny fraction of the available memory meaning there is plenty of space inside the chip for additional functionality if needed.
TDM Build#
Tile |
Memory |
Threads |
---|---|---|
0 |
25996 |
5 |
1 |
22812 |
2* |
Total |
48808 |
7 |
An additional debug TDM Master thread is used on Tile[1] by default which is not needed in a practical deployment.
USB Build#
Tile |
Memory |
Threads |
---|---|---|
0 |
24252 |
4 |
1 |
52116 |
5 |
Total |
76368 |
9 |
Board Configuration#
Make the following connections between headers using flying leads:
Host Connection |
Board Connection |
Note |
---|---|---|
MIC CLK |
J14 ‘00’ |
This is the microphone clock which is to be sent to the PDM microphones from J14. |
MIC DATA |
J14 ‘14’ |
This is the data line for microphones 0 and 8. See below. |
I2S LRCLK |
J10 ‘36’ |
This is the FSYCNH input for TDM slave. J10 ‘36’ is the TDM master FSYNCH output for the application. |
I2S MCLK |
I2S BCLK |
MCLK is the 24.576MHz clock which directly drives the BCLK input for the TDM slave. |
I2S DAC |
J10 ‘38’ |
I2S DAC is the TDM Slave Tx out which is read by the TDM Master Rx input on J10. |
To access other microphone inputs use the following:
Mic pair |
J14 pin |
---|---|
0, 8 |
14 |
1, 9 |
15 |
2, 10 |
16 |
3, 11 |
17 |
4, 12 |
18 |
5, 13 |
19 |
6, 14 |
20 |
7, 15 |
21 |
For I2C control, make the following connections:
Host Connection |
Board Connection |
---|---|
SCL IOL |
Your I2C host SCL. |
SDA IOL |
Your I2C host SDA. |
GND |
Your I2C host ground. |
The I2C slave is tested at 100 kHz SCL.
I2C Controlled Volume#
For the TDM build, there are 32 registers which control the gain of each of the 16 output channels. The 8 bit registers contain the upper 8 bit and lower 8 bit of the microphone gain respectively. The initial gain is set to 100, since 1 is quiet due to the mic_array output being scaled to allow acoustic overload of the microphones without clipping. Typically a gain of a few hundred works for normal conditions. The gain is only applied after the lower byte is written.
The gain applied is saturating so no overflow will occur, only clipping.
Register |
Value |
---|---|
0 |
Channel 0 upper gain byte |
1 |
Channel 0 lower gain byte |
2 |
Channel 1 upper gain byte |
3 |
Channel 1 lower gain byte |
4 |
Channel 2 upper gain byte |
5 |
Channel 2 lower gain byte |
6 |
Channel 3 upper gain byte |
7 |
Channel 3 lower gain byte |
8 |
Channel 4 upper gain byte |
9 |
Channel 4 lower gain byte |
10 |
Channel 5 upper gain byte |
11 |
Channel 5 lower gain byte |
12 |
Channel 6 upper gain byte |
13 |
Channel 6 lower gain byte |
14 |
Channel 7 upper gain byte |
15 |
Channel 7 lower gain byte |
16 |
Channel 8 upper gain byte |
17 |
Channel 8 lower gain byte |
18 |
Channel 9 upper gain byte |
19 |
Channel 9 lower gain byte |
20 |
Channel 10 upper gain byte |
21 |
Channel 10 lower gain byte |
22 |
Channel 11 upper gain byte |
23 |
Channel 11 lower gain byte |
24 |
Channel 12 upper gain byte |
25 |
Channel 12 lower gain byte |
26 |
Channel 13 upper gain byte |
27 |
Channel 13 lower gain byte |
28 |
Channel 14 upper gain byte |
29 |
Channel 14 lower gain byte |
30 |
Channel 15 upper gain byte |
31 |
Channel 15 lower gain byte |
If using a raspberry Pi as the I2C host you may use the following commands:
$ i2cset -y 1 0x3c 0 0 #Set the gain on mic channel 0 to 50
$ i2cset -y 1 0x3c 1 50 #Set the gain on mic channel 0 to 50
$ i2cget -y 1 0x3c 0 #Get the upper byte of gain on mic channel 0
$ i2cget -y 1 0x3c 1 #Get the lower byte of gain on mic channel 0
$ i2cset -y 1 0x3c 16 1 #Set the gain on mic channel 8 to 256
$ i2cset -y 1 0x3c 15 0 #Set the gain on mic channel 8 to 256