AVNET RASynBoard Starter Kit Development User Guide

AVNET RASynBoard Starter Kit Development

Product Information

Specifications:

• Product Name: RASynBoard Starter Kit
• Document Version: 4.2
• Document Date: June 20, 2023
• Author: Peter Fenn
• Classification: Public

Product Usage Instructions

Hardware Setup for Application Development

1. Connect the 5V power input to power the RASynBoard Starter Kit.
2. If using a LiPo battery, connect it for input and charging.
3. Implement Debug UART for debugging purposes.

Software Installation

1. Utilize an SD Card for NDP120 firmware files and data recording.
2. Install NDP120 firmware files on a microSD Card.
3. Program the RA6M4 MCU using Renesas Flash Programmer (RFP).

Run the Pre-Built Alexa Demo Application

1. Perform keyword inference using the Syntiant Alexa model.

Build & Debug an Application using Renesas e2-studio IDE

1. Use the Renesas e2-studio IDE to build and debug applications.
2. Refer to notes regarding source code in the application.

I/O Board Expansion Connectors

1. Connect devices to the left-side connectors as needed.
2. Utilize the right-side connectors for additional expansion.

Hardware Requirements

• RASynBoard EVK (Starter Kit) : Avnet p/n: AES-RASYNB-120-SK-G
• 2mm Jumper : Handle version preferred for 2mm-pitch header
• PL2303TA based USB-to-Serial Debug Console Cable
• USB 2.0 micro SD Card Reader/Programmer
• USB-C to USB-A 3ft power+data cable

FAQ

• Q: Where can I find additional technical support for the RASynBoard Starter Kit?
  A: For additional technical support, please visit the official website or contact the manufacturer directly.

Document Control

• Document Version: 4.2
• Document Date: 06/20/2023
• Document Author: Peter Fenn
• Document Classification: Public
• Document Distribution: Public

Version History

Hardware Requirements

Listing of hardware items used during development (typical pricing shown )

Software Requirements

Overview

RASynBoard Core Board is a tiny (25mm x 30mm), ultra-low power, edge AI/ML board, based on Syntiant NDP120 Neural Decision Processor, a Renesas RA6M4 host MCU plus a power-efficient DA16600 Wi-Fi/BT combo module. The NDP120 subsystem with on-board digital microphone, IMU motion sensor and SPI Flash memory, achieves highly efficient processing of acoustic- and motion events. Battery and USB-C device connectors facilitate stand-alone use, while a compact under-board connector enables integration with custom OEM boards and additional sensors.
Renesas RA6M4 MCU application software development and debug is supported via Renesas e2 Studio IDE application, interfaced via the E2Lite debug interface implemented on RASynBoard’s IO carrier board. With a power-efficient 200 MHz Arm Cortex- M33 core, versatile set of peripheral interfaces and advanced security, the RA6M4 as a host microcontroller has much to offer, plus Wi-Fi and BLE connectivity via a companion power-efficient DA16600 wireless module, integrated onto the core board.
NDP120 application software and AI/ML models for popular use-cases (pre- engineered by Syntiant and others) are loaded from local microSD card or SPI Flash, for efficient execution on the ultra-low power NDP120 neural accelerator device, which is particularly well suited for Always-On Speech and Sensor-Fusion applications.

Core Board Features

• Syntiant NDP120 Neural Engine
  • Syntiant Core 2 Deep Neural Network
  • Arm Cortex M0 and HiFi 3 DSP
• Renesas RA6M4 Microcontroller
  • 1x Arm Cortex M33 (200 MHz)
  • 1 MB flash memory, 256 KB SRAM
  • USB 2.0 device interface
• Renesas DA16600 Wi-Fi/BT Combo Module
  • 802.11bgn 1×1 2.4 GHz Wi-Fi and BT 5.1
• Additional Onboard Memory
  • 2 MB SPI NOR Flash
• Battery Management
  • LiPo battery management and connector
• Onboard Sensors
  • IMU 6-axis motion sensor (ICM42670-P)
  • PDM digital microphone (MMICT5838)
• Expansion Connector & Dimensions
  • 2×28 pin board-to-board connector
  • 25 mm x 30 mm

Core Board Block Diagram

RASynBoard Starter Kit adds an IO Board (50mm x 30mm), for a versatile, compact, two-board evaluation kit assembly. This pins-out a subset of the NDP120 and RA6M4 I/Os to popular Pmod, Click header and expansion header footprints, enabling connection with additional external microphones and sensor options. An onboard debugger MCU (SWD and UART interfaces), button switches, RGB LED and removable MicroSD storage, further maximize the prototyping versatility and utility.

IO Board Features

• Onboard Debugger and USB Serial interface
  • Renesas E2 OB debugger MCU (USB C to SWD and serial interf ace)
  • 3.3V buck regulator for debugger circuits Expansion Interfaces and Storage
  • 2×28 pin board to board connector
  • 2×8 pin MikroE Click shuttle box header
  • 2×6 pin Pmod type 6A (I2C) socket
  • 2×7 pin MCU e xpansion header
  • 2×3 pin DMIC expansion headers (two) 3.3V level translated expansion interfaces
  • Micro SD card cage for removable storage
• User Interfaces
  • 2x Button Switches (Reset and User)
  • 1x User RGB LED
• Dimensions
  • 50 mm x 30 mm

I/O Board Block Diagram

Hardware Setup for Application Development

5V Power Input
One of three connectors can be used to power RASynBoard Which power source to use, depends on the use case

during software development

Core Board USB-C connector| If Core Board used standalone| Power & flash programming if Core Board is used standalone
Core Board battery connector| For LiPo battery operation| Battery- powered operation

While a 5V power source applied to either of the USB-C connectors can be used to power both boards, during application software development and testing for the RA6M4 MCU, simply power both boards from the development computer via the E2OB onboard-debugger USB-C connector on the

IO Board
LiPo Battery Input and Charging
The RASynBoard module is designed to also support operation from a Lithium Polymer battery On the Core Board two status LEDs only operate when external 5V power is connected:

Debug UART Implementation
During Starter Kit-based development and debugging of the RA6M4 application, it is convenient to use SCI4 assigned to P206 and P205 signals. To implement a debug UART, access these signals using fly-leads from the USB-UART cable, connected to the 3 pins of the J8 Pmod connector highlighted below. This UART is then accessed via the COM port that gets enumerated in Windows, using eg. Tera Term console application

Notes:

• The Tera Term console application should be configured for 115200 8N1 operation
• Pmod and Click connectors are factory-configured with I2C-related signals connected to I2C/UART pins of the RA6M4 microcontroller. Other signals are connected to the NDP120. (For optimum low-power performance, at a future date the sensors will be serviced directly by the NDP120)

Software Installation

1. Download and install the Renesas e2Studio IDE (version 2022-10 or later) https://www.renesas.com/e2studio
   Download link for Windows version for RA family:
   https://www.renesas.com/us/en/software-tool/e2studio-information-ra-family

2. Download and install Renesas Flash Programmer (version 3.11.01, 5 Jan 2023) https://www.renesas.com/rfp
   Download link for Windows version:
   https://www.renesas.com/us/en/document/swe/renesas-flash- programmer-v31101-windows

3. Download and install Tera Term (or equivalent) serial console application https://osdn.net/dl/ttssh2/teraterm-4.106.exe/

4. Download and install the PL2303 software driver for Windows 10 (for USB-Serial cable) prolific_usb_serial_3.8.28.0(station-drivers).zip

SD Card Use for NDP120 Firmware Files and Data Recording
The microSD card on underside of the IO Board serves multiple purposes:

• It provides a convenient way to load .synpkg firmware files into the NDP120 on startup
• It provides a way to copy .synpkg and .ini files into SPI flash on the Core board (ie for later standalone use of RASynBoard, where NDP120 firmware then gets loaded from SPI flash)
• It facilitates the recording of training data (eg. from onboard microphone or IMU) for training of the DNN model in 3rd party tools (eg. Edge Impulse Studio)

How to install NDP120 firmware files on microSD Card

On reset of RASynBoard, the RA6M4 application will attempt to load three .synpkg firmware files into the RAM memory of the NDP120 device.
By default, these .synpkg files load from uSDcard mass storage, based on settings in a config.ini file located on this uSDcard. What is to be loaded, is specified in the following format in this config file:

• MCU=mcu_fw_120.synpkg
• DSP=dsp_firmware_noaec_ff.synpkg
• DNN=menu_demo_512_rasyn_newph.synpkg

For backward compatibility, if this config.ini file is missing (or an older version of the demo application is used), then .synpkg files with the following default filenames must be present on the uSD card:

• mcu_fw_120.synpkg
• dsp_firmware.synpkg
• ei_model.synpkg

At this time, writing a new set of .synpkg files to uSDcard requires removal of uSDcard from the IO board, writing the files (from the computer) using a suitable SDcard adapter, then returning the uSDcard to the IO board. Explanation of the different sections in this config.ini file is provided in the Appendix of this User Guide

How to use Renesas Flash Programmer (RFP) to program RA6M4 MCU
If wanting to use a prebuilt RA6M4 binary file (supplied in .srec format), then use the following procedure:

1. Using the photos on page 7 prepare your hardware setup (do not power up RASynBoard yet)

2. To allow programming (and SWD debugging) of the RA6M4 MCU via the E2OB debugger on the IO Board:

3. Remove the 2mm shorting jumper from J3 on the IO Board

4. Remove the 2mm shorting jumper from J5 on the Core Board (if there is one fitted)

5. Connect USB cable from IO Board USB-C connector to a USB-A port on the development PC

6. Open Renesas Flash Programmer (version 3.11.01)

7. From the menu bar, select File → New Project → complete the Create New Project form as shown in the screenshot below (for Tool, select E2 Emulator Lite) then click the Connect button…
   ( Note : For the Interface type, either 2 wire UART (the default) or SWD can be used)

8. Select the Operation tab, then for Program File click Browse, then select the pre-built binary file (.SREC)

9. Click the large Start button to program the selected .srec file into the RA6M4 MCU flash memory…

10. Once successfully programmed, power-down RASynBoard by disconnecting the USB cable from the development computer.

11. Fit the 2mm shorting jumper across pins 1 & 2 of J3 on the IO Board (to strap E2OB in Reset mode)

12. Reconnect USB cable from USB-C on IO Board to the development computer or other +5V source

13. Provided the uSD card has the expected three .synpkg files, the RGB-Blue LED should illuminate for about 3 seconds while these files are read from the SD card, and loaded into the RAM of the NDP120

Run the Pre-Built Alexa Demo Application
Keyword inference using the Syntiant Alexa model

1. After reset of the RA6M4 MCU, it will read the MCU, DSP and DNN .synpkg firmware files from the file-system on the FAT32 formatted uSDcard and load these into the NDP120 RAM memory.
2. Wait until firmware has loaded (RGB LED turns-off), then speak the “Alexa” keyword If this is recognized by the NDP120, two forms of confirmation will be seen:
3. the RGB Green LED will briefly illuminate
4. a message is also output to the Tera Term console each time the keyword is recognized

Build & Debug an application using Renesas e2-studio IDE
The rasynboard_ew_demo project is the FreeRTOS-based RA6M4 application that is factory-programmed into RA6M4 MCU flash memory. It exercises multiple features of the RASynBoard Starter Kit. Originally developed for voice UI based demos at Embedded World, this application is useful to developers as it has evolved to support a wide range of RASynBoard functions, eg:

1. Load NDP120 firmware files from uSDcard (MCU, DSP, DNN .synpkg files)
2. Write recorded microphone audio to uSD card (.wav files)
3. Write NDP120 firmware to SPI Flash
4. Load NDP120 firmware from SPI Flash
5. Receive new voice-command inference results from the NDP120
6. Provide console debug output via UART (SCI4)
7. Service the local Status Green LED on the Core Board
8. Service the RGB_LED output on the IOBoard
9. Respond to button-press events on the IO Board
10. Establish wireless BLE comms with remote Raspberry Pi board that drives GUI display
11. Send inference results via BLE to remote Raspberry Pi board that drives GUI display
12. Implement Sleep Mode entry and exit (where only the NDP120 remains active)
13. etc…

To import and build the project rasynboard_ew_demo use the following steps:

1. On development computer, open e2-studio IDE, then select “File -> Import” to open the import dialog
2. Select the rasynboard_ew_demo directory in the subsequent Import dialog
3. After importing an existing project into your workspace, when using this for first time, you must open the (FSP) configuration and click on Generate Project Content, before attempting to build the project!
4. Click OK on the dialog warning that a different FSP version will be used, after the FSP panel opens-up, click on Generate Project Content (in top right-hand corner)
5. Now with the project name highlighted in Project Explorer left sidebar, click on the Build (hammer) icon on the toolbar to compile the project
6. An additional one-time setup step is needed before starting a debug session: With the project name selected, open the Run menu, then select Debug Configurations…
7. With the rasynboard_ew_demo Debug configuration selected in the left panel, open the Debugger tab, set the Debug hardware to E2 Lite (Arm), then select Connection Settings and make sure all settings match what is shown below…
8. Once the Debug Configuration is correctly set, apply these settings, then launch Debug (Debug can be launched from within this form, or by using the Debug icon at top left of the IDE)

Notes regarding Source Code in this Application

The src folder contains the user source files for this custom application
The ndp120 folder contains Syntiant NDP120 Tiny iLib SDK library files with APIs accessed in this project
The ndp120\syn_pkg_files folder contains the MCU, DSP, DNN .synpkg files and a sample config.ini file Note!

1. Edit the config.ini file to match the specific configuration that you want to exercise
2. Make sure to copy all files in this folder to your uSDCard before running the application!!!

For this FreeRTOS based implementation, four threads are serviced (located in files of matching names)

• led_threadx_entry(); // Manages Core Board Green Status LED
• ndp_thread_entry(); // NDP120 configuration and service functions
• system_cmd_thread_entry(); // Manages IO Board RGB LED and Sleep mode entry/exit
• ndp_record_thread_entry(); // Manages audio .wav file recording to uSDcard

Syntiant SDK configuration APIs are called from the ndp_thread_entry() function:

A Syntiant SDK API called in response to NDP interrupt, returns an index for the recognized voice keyword and processes this via a case statement in a while(1) loop later in same ndp_thread_entry() function

I/O Board Expansion Connectors

The IO Board provides multiple expansion interfaces, these are tabled here across two pages

Left-Side Connectors

Right-Side Connectors

Appendix – Development Notes

SDcard Config.ini File Settings
The config.ini file facilitates rapid reconfiguration of how RASynBoard is to be used (without rebuild of the application). The RA6M4 application reads settings for how it should operate, from this text file on startup.
Typical config.ini settings shown below are for applications using onboard microphone(s) or IMU sensors

Explanation of Fields within Config.ini File

• [NDP Firmware]
  Defines which set of firmware binary images are to be loaded into the NDP120 on startup. Three application use case modes are presently defined, using onboard microphone(s) or IMU sensors:

  • Single Microphone mode (eg. voice commands or audio events)
  • Dual Microphone mode (eg. voice commands or audio events)
  • IMU Sensor mode (eg. motion/vibration / motor-anomalies / hand gestures)

• [Single Mic]
  Defines the specific three firmware binary images to load into the NDP120 for single mic operation.

• Dual Mic]
  Defines the specific three firmware binary images to load into the NDP120 for dual mic operation. Note: An optional add-on microphone hardware accessory is needed to use this option

• [Circle Motion]
  Defines the specific three firmware binary images to load into the NDP120 for IMU motion detection (Note: At the time of RASynBoard Starter Kit release, only the circular motion gesture is detected)

• [Led]
  Defines for each supported inference index, the color which the RGB LED on IO board should illuminate when that specific inference gets identified by the NDP120

• [Debug Print]
  Defines to which UART the debug printf console output should be routed. Two options can presently be specified (see next section in the Appendix for more detail):

  • by UART
  • by USB-VCOM

• The three firmware binaries that load into the NDP120 on startup are as shown below:
  NDP120

Core

| Default Filenames| Comments
---|---|---
MCU| mcu_fw_120_notify.synpkg| Arm Cortex-M0 firmware image

for NDP120 peripherals, etc management

DSP| dsp_firmware.synpkg| Tensilica HiFi 3 DSP firmware image (eg. for audio preprocessing)
DNN| (depends on application)| Deep Neural Network firmware image (ie. the trained AI model)

Options for Implementing Debug UART Console Output

• Use of an external USB-to-Serial adapter cable (default method) A debug UART from the RA6M4 MCU is implemented in the application using SCI4 assigned to the P206 and P205 port signals. This UART however will no longer be available if P206 and P205 are needed to implement an I2C bus to the Pmod and/or Click connectors. In that case, alternative methods such as the following can be used

• Implement USB device CDC VCOM port via USB-C connector on Core board If console output is always needed when running the application, then within the application you can implement a USB VCOM port, over which the debug UART output is then transmitted
  Disadvantages of USB solution: – An additional USB connection between RASynBoard and the development PC is required – Power measurement will be more challenging: due to use of USB peripheral interface and because there will now be two power connections to the board during development

• Implement SEMIHOST for terminal output within e2 Studio IDE Simplest is to have printf trace output (ie. stdio) sent to a different UART (preferably one not utilized on the board), then use the Arm Cortex-M “semihost” feature, to output printf debug info from this UART to the terminal view within E2Studio IDE. How to implement this with the RA family is documented in the e2 studio IDE User Guide (Goto help then search for ”semihost”) Disadvantages of SEMIHOST solution: – Semihost is only accessible during development / while using the e2 studio IDE

Standalone Core Board Operation

It is possible to program and run RASynBoard Core Board “standalone” (without IO Board), note however:

• RA6M4 flash programming is only possible using the RFP application (with 1.27mm jumper on J5)
• SWD based debugging is not supported when Core Board is used standalone
• NDP120 firmware must load from SPI Flash (since microSD card storage not available)
• RGB LED, button switches, microSD card, Pmod and Click interfaces, etc are not available
  Note : The only time the smaller 1.27mm pitch jumper (taped to inside lid of the Starter Kit box) needs to be fitted to Core Board J5, is to set standalone (SCI boot mode) based USB-UART programming of the RA6M4 MCU, via the Core Board USB-C interface (- this not typically needed) J5 Jumper| RA6M4 Boot Mode
  ---|---

Removed

| Self-boot/Run mode

Note: This jumper must also be removed when using IO Board

Bridge J5

pin 1 & 2

| SCI boot mode for MCU flash programming via Core Board’s

USB-C interface

How to Transfer .synpkg files to SPI Flash (to direct boot NDP120 from SPI Flash)

From the running application, hold-down the USER button-switch for at least 3 seconds to copy the .synpkg firmware files (for MCU, DSP and DNN cores) and .ini settings from the uSDcard to the SPI flash
Direct boot of the NDP120, using these files on the SPI Flash, will be initiated if the uSDcard is not inserted

Mechanical Considerations

Technical Support

Online technical support is accessible via the RASynBoard product page
Use the link provided in the sidebar at https://avnet.me/rasynboard

Documentation for key components can be found at:

• Syntiant NDP120 AI/ML page www.syntiant.com/ndp120
• Renesas RA6M4 MCU page www.renesas.com/RA6M4
• Renesas DA16600MOD page www.renesas.com/DA16600MOD

Ordering Part Numbers

https://avnet.me/rasynboard
AES-RASYNB-120B-G| RASynBoard NDP120 Core Board only| Pricing on Request

Out-of-Box Application (rasynboard_ew_demo)
RASynBoard Starter Kit – Uses NDP120 for key-word spotting AI model, that has been trained with five command words – On recognition of a valid voice command, it confirms by illuminating onboard RGB LED for 1 sec (The RGB LED color identifies which command word has been recognized) – Sends inference results via BLE wireless to remote SBC board that generates a GUI response

Remote Single Board Computer (Raspberry Pi400)

• Receives inference results from RASynBoard via BLE wireless
• Drives the Voice-UI GUI on a large HDMI display screen

RASynBoard Power Consumption
Total power consumption for the RASynBoard 2-board assembly is monitored in two modes, using an inline USB power meter

• NDP120-only (listening for wake-word) – RA6M4 MCU is in SLEEP mode – BLE is OFF
• Wake Word recognized (all cores now up) – RA6M4 MCU is awake – BLE is ON!

Entry / Exit of Low-Power Sleep Mode
NDP120-only low-power mode is entered by repeating the DOWN voice command This low-power mode is exited by subsequent use of using any of the 5 supported voice commands

References

• DA16600MOD - Ultra-Low Power Wi-Fi + Bluetooth® Low Energy Combo Modules for Battery Powered IoT Devices | Renesas
• RA6M4 - 200MHz Arm® Cortex®-M33 TrustZone®, High Integration with Ethernet and OctaSPI | Renesas
• Hardware — Syntiant
• RASynBoard | Avnet Boards
• RZBoard V2L | Avnet Boards
• CQRobot 100 Pieces Black Standard Computer Jumper Caps with Handle Pin Shunt Short Circuit 2-Pin Connector 2.0mm. for Arduino Raspberry Pi PCB PC DVD HDD Motherboard Shorting and Other Project.
• CLEEFUN USB C Cable [3ft, 5-Pack], USB A to Type C Cable Fast Charging Charger Cord Braided for iPhone 15 Pro Max/Pro/Plus, for Samsung Galaxy S24 S23 S22 S21 S20 Ultra S10, Moto, Pixel : Electronics
• IOGEAR USB 2.0 SD Portable Card Reader - Dual Slot - Rate Up To 480Mbps - USB Powered - SDXC/SDHC/SD/Micro SDXC/Micro SD/Micro SDHC/M2/MS/CF/UHS-I - Mac/Win/Chrome - GFR204SD : Electronics
• EVISWIY PL2303TA USB to TTL Serial Cable Debug Console Cable for Raspberry Pi 3 Pack : Electronics
• DA16600MOD - Ultra-Low Power Wi-Fi + Bluetooth® Low Energy Combo Modules for Battery Powered IoT Devices | Renesas
• e² studio | Renesas
• RA6M4 - 200MHz Arm® Cortex®-M33 TrustZone®, High Integration with Ethernet and OctaSPI | Renesas
• Renesas Flash Programmer (Programming GUI) | Renesas
• Log In | Renesas Electronics Corporation
• e² studio -information for RA Family | Renesas
• Hardware — Syntiant

Read User Manual Online (PDF format)

Download This Manual (PDF format)

Download this manual  >>