Modifying the Software#

Implementing the ASR API#

Begin your ASR port by creating a new folder under modules/asr/. The asr.h and device_memory.h files include comments detailing the public API methods and parameters. ASR ports that implement the public API defined can easily be added to current and future XCORE-VOICE example designs that support speech recognition.

Pay close attention to the functions: - asr_printf - devmem_malloc - devmem_free - devmem_read_ext - devmem_read_ext_async - devmem_read_ext_wait

ASR libraries should call asr_printf instead of printf or xcore’s debug_printf.

ASR libraries must not call malloc directly to allocate dynamic memory. Instead call the devmem_malloc and devmem_free functions. This allows the application to provide alternative implementations of these functions - like pvPortMalloc and vPortFree in a FreeRTOS application.

The devmem_read_ext function is provided to load data directly from external memory (QSPI flash or LPDDR) into SRAM. This is the recommended way to load coefficients or blocks of data from a model. It is far more efficient to load the data into SRAM and perform any math on the data while it is in SRAM. The devmem_read_ext function a signature similar to memcpy. The caller is responsible for allocating the destination buffer.

Like devmem_read_ext, the devmem_read_ext_async function is provided to load data directly from external memory (QSPI flash or LPDDR) into SRAM. devmem_read_ext_async differs in that it does not block the caller’s thread. Instead it loads the data in another thread. One must have a free core when calling devmem_read_ext_async or an exception will be raised. devmem_read_ext_async returns a handle that can later be used to wait for the load to complete. Call devmem_read_ext_wait to block the callers thread until the load is complete. Currently, each call to devmem_read_ext_async must be followed by a call to devmem_read_ext_wait. You can not have more than one read in flight at a time.

Note

XMOS provides an arithmetic and DSP library which leverages the XS3 Vector Processing Unit (VPU) to accelerate costly operations on vectors of 16- or 32-bit data. Included are functions for block floating-point arithmetic, fast Fourier transforms, discrete cosine transforms, linear filtering and more. See the XMath Programming Guide for more information.

Note

To minimize SRAM scratch space usage, some ASR ports load coefficients into SRAM in chunks. This is useful when performing a routine such as a vector matrix multiply as this operation can be performed on a portion of the matrix at a time.

When the port of the new ASR is complete, you can use the example in examples/speech_recognition to test it.

Note

You may also need to modify BRICK_SIZE_SAMPLES in app_conf.h to match the number of audio samples expected per process for your ASR port. In other example designs, this is defined by appconfINTENT_SAMPLE_BLOCK_LENGTH. This is set to 240 in the existing example designs.

In the current source code, the model data (and optional grammar data) are set in examples/speech_recognition/src/process_file.c. Modify these variables to reflect your data. The remainder of the API should be familiar to ASR developers. The API can be extended if necessary.

Flashing Models#

To flash your model, modify the --data argument passed to xflash command in the Flashing the Model section.

See examples/speech_recognition/asr_example/asr_example_model.h to see how the model’s flash address is defined.

Placing Models in SRAM#

Small models (near or under 100kB in size) may be placed in SRAM. See examples/speech_recognition/asr_example/asr_example_model.c for more information on placing your model in SRAM.

ASR API#

enum asr_error_enum#

Enumerator type representing error return values.

Values:

enumerator ASR_OK#

Ok.

enumerator ASR_ERROR#

General error

enumerator ASR_INSUFFICIENT_MEMORY#

Insufficient memory for given model.

enumerator ASR_NOT_SUPPORTED#

Function not supported for given model.

enumerator ASR_INVALID_PARAMETER#

Invalid Parameter.

enumerator ASR_MODEL_INCOMPATIBLE#

Model type or version is not compatible with the ASR library.

enumerator ASR_MODEL_CORRUPT#

Model malformed.

enumerator ASR_NOT_INITIALIZED#

Not Initialized.

enumerator ASR_EVALUATION_EXPIRED#

Evaluation period has expired.

typedef void *asr_port_t#

Typedef to the ASR port context struct.

An ASR port can store any data needed in the context. The context pointer is passed to all API methods and can be cast to any struct defined by the ASR port.

typedef int16_t asr_sample_t#

Typedef representing the base type of an audio sample.

typedef struct asr_attributes_struct asr_attributes_t#

Typedef to the ASR port and model attributes

typedef struct asr_result_struct asr_result_t#

Typedef to the ASR result

typedef enum asr_error_enum asr_error_t#

Enumerator type representing error return values.

void asr_printf(const char *format, ...)#

String output function that allows the application

to provide an alternative implementation.

ASR ports should call asr_printf instead of printf

asr_port_t asr_init(int32_t *model, int32_t *grammar, devmem_manager_t *devmem_ctx)#

Initialize an ASR port.

Parameters:

  • model – A pointer to the model data.

  • grammar – A pointer to the grammar data (Optional).

  • devmem_ctx – A pointer to the device manager (Optional). Save this pointer if calling any device manager API functions.

Returns:

the ASR port context.

asr_error_t asr_get_attributes(asr_port_t *ctx, asr_attributes_t *attributes)#

Get engine and model attributes.

Parameters:

  • ctx – A pointer to the ASR port context.

  • attributes – The attributes result.

Returns:

Success or error code.

asr_error_t asr_process(asr_port_t *ctx, int16_t *audio_buf, size_t buf_len)#

Process an audio buffer.

Parameters:

  • ctx – A pointer to the ASR port context.

  • audio_buf – A pointer to the 16-bit PCM samples.

  • buf_len – The number of PCM samples.

Returns:

Success or error code.

asr_error_t asr_get_result(asr_port_t *ctx, asr_result_t *result)#

Get the most recent results.

Parameters:

  • ctx – A pointer to the ASR port context.

  • result – The processed result.

Returns:

Success or error code.

asr_error_t asr_reset(asr_port_t *ctx)#

Reset ASR port (if necessary).

Called before the next call to asr_process.

Parameters:

  • ctx – A pointer to the ASR port context.

Returns:

Success or error code.

asr_error_t asr_release(asr_port_t *ctx)#

Release ASR port (if necessary).

The ASR port must deallocate any memory.

Parameters:

  • ctx – A pointer to the ASR port context.

Returns:

Success or error code.

struct asr_attributes_struct#
#include <asr.h>

Typedef to the ASR port and model attributes

struct asr_result_struct#
#include <asr.h>

Typedef to the ASR result

Device Memory API#

void *devmem_malloc(devmem_manager_t *ctx, size_t size)#

Memory allocation function that allows the application to provide an alternative implementation.

Call devmem_malloc instead of malloc

Parameters:

  • ctx – A pointer to the device memory context.

  • size – Number of bytes to allocate.

Returns:

A pointer to the beginning of newly allocated memory, or NULL on failure.

void devmem_free(devmem_manager_t *ctx, void *ptr)#

Memory deallocation function that allows the application to provide an alternative implementation.

Call devmem_free instead of free

Parameters:

  • ctx – A pointer to the device memory context.

  • ptr – A pointer to the memory to deallocate.

void devmem_read_ext(devmem_manager_t *ctx, void *dest, const void *src, size_t n)#

Synchronous extended memory read function that allows the application

to provide an alternative implementation. Blocks the callers thread until the read is completed.

Call devmem_read_ext instead of any other functions to read memory from flash, LPDDR or SDRAM. Modules are free to use memcpy if the dest and src are both SRAM addresses.

Parameters:

  • ctx – A pointer to the device memory context.

  • dest – A pointer to the destination array where the content is to be read.

  • src – A pointer to the word-aligned address of data to be read.

  • n – Number of bytes to read.

int devmem_read_ext_async(devmem_manager_t *ctx, void *dest, const void *src, size_t n)#

Asynchronous extended memory read function that allows the application

to provide an alternative implementation.

Call asr_read_ext_async instead of any other functions to read memory from flash, LPDDR or SDRAM.

Parameters:

  • ctx – A pointer to the device memory context.

  • dest – A pointer to the destination array where the content is to be read.

  • src – A pointer to the word-aligned address of data to be read.

  • n – Number of bytes to read.

Returns:

A handle that can be used in a call to devmem_read_ext_wait.

void devmem_read_ext_wait(devmem_manager_t *ctx, int handle)#

Wait in the caller’s thread for an asynchronous extended memory read to finish.

Parameters:

  • ctx – A pointer to the device memory context.

  • handle – The devmem_read_ext_asyc handle to wait on.

IS_SRAM(a)#
IS_SWMEM(a)#
IS_FLASH(a)#