ESP32-C3 Wireless Adventure

ESP32-C3 Wireless Adventure

A Comprehensive Guide to IoT

Espressif Systems June 12, 2023

Specifications

• Product: ESP32-C3 Wireless Adventure
• Manufacturer: Espressif Systems
• Date: June 12, 2023

Product Usage Instructions

Preparation

Before using the ESP32-C3 Wireless Adventure, make sure you are
familiar with the concepts and architecture of IoT. This will help
you understand how the device fits into the larger IoT ecosystem
and its potential applications in smart homes.

Introduction and Practice of IoT Projects

In this section, you will learn about typical IoT projects,
including the basic modules for common IoT devices, basic modules
of client applications, and common IoT cloud platforms. This will
provide you with a foundation for understanding and creating your
own IoT projects.

Practice: Smart Light Project

In this practice project, you will learn how to create a smart
light using the ESP32-C3 Wireless Adventure. The project structure,
functions, hardware preparation, and development process will be
explained in detail.

Project Structure

The project consists of several components, including the
ESP32-C3 Wireless Adventure, LEDs, sensors, and a cloud
backend.

Project Functions

The smart light project allows you to control the brightness and
color of the LEDs remotely through a mobile app or web
interface.

Hardware Preparation

To prepare for the project, you will need to gather the
necessary hardware components, such as the ESP32-C3 Wireless
Adventure board, LEDs, resistors, and a power supply.

Development Process

The development process involves setting up the development
environment, writing code to control the LEDs, connecting to the
cloud backend, and testing the functionality of the smart
light.

Introduction to ESP RainMaker

ESP RainMaker is a powerful framework for developing IoT
devices. In this section, you will learn what ESP RainMaker is and
how it can be implemented in your projects.

What is ESP RainMaker?

ESP RainMaker is a cloud-based platform that provides a set of
tools and services for building and managing IoT devices.

The Implementation of ESP RainMaker

This section explains the different components involved in
implementing ESP RainMaker, including the claiming service,
RainMaker Agent, cloud backend, and RainMaker Client.

Practice: Key Points for Developing with ESP RainMaker

In this practice section, you will learn about the key points to
consider when developing with ESP RainMaker. This includes device
claiming, data synchronization, and user management.

Features of ESP RainMaker

ESP RainMaker offers various features for user management, end
users, and administrators. These features allow for easy device
setup, remote control, and monitoring.

Setting Up Development Environment

This section provides an overview of ESP-IDF (Espressif IoT
Development Framework), which is the official development framework
for ESP32-based devices. It explains the different versions of
ESP-IDF and how to set up the development environment.

Hardware and Driver Development

Hardware Design of Smart Light Products based on ESP32-C3

This section focuses on the hardware design of smart light
products based on the ESP32-C3 Wireless Adventure. It covers the
features and composition of smart light products, as well as the
hardware design of the ESP32-C3 core system.

Features and Composition of Smart Light Products

This subsection explains the features and components that make
up smart light products. It discusses the different functionalities
and design considerations for creating smart lights.

Hardware Design of ESP32-C3 Core System

The hardware design of the ESP32-C3 core system includes power
supply, power-on sequence, system reset, SPI flash, clock source,
and RF and antenna considerations. This subsection provides
detailed information on these aspects.

FAQ

Q: What is ESP RainMaker?

A: ESP RainMaker is a cloud-based platform that provides tools
and services for building and managing IoT devices. It simplifies
the development process and allows for easy device setup, remote
control, and monitoring.

Q: How can I set up the development environment for

ESP32-C3?

A: To set up the development environment for ESP32-C3, you need
to install ESP-IDF (Espressif IoT Development Framework) and
configure it according to the provided instructions. ESP-IDF is the
official development framework for ESP32-based devices.

Q: What are the features of ESP RainMaker?

A: ESP RainMaker offers various features, including user
management, end user features, and admin features. User management
allows for easy device claiming and data synchronization. End user
features enable remote control of devices through a mobile app or
web interface. Admin features provide tools for device monitoring
and management.

ESP32-C3 Wireless Adventure
A Comprehensive Guide to IoT
Espressif Systems June 12, 2023

Contents

I Preparation

1

1 Introduction to IoT

3

1.1 Architecture of IoT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2 IoT Application in Smart Homes . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Introduction and Practice of IoT Projects

9

2.1 Introduction to Typical IoT Projects . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1.1 Basic Modules for Common IoT Devices . . . . . . . . . . . . . . . . . 9

2.1.2 Basic Modules of Client Applications . . . . . . . . . . . . . . . . . . . 10

2.1.3 Introduction to Common IoT Cloud Platforms . . . . . . . . . . . . . . 11

2.2 Practice: Smart Light Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2.1 Project Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.2.2 Project Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.2.3 Hardware Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.2.4 Development Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3 Introduction to ESP RainMaker

19

3.1 What is ESP RainMaker? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.2 The Implementation of ESP RainMaker . . . . . . . . . . . . . . . . . . . . . . 21

3.2.1 Claiming Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.2.2 RainMaker Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.2.3 Cloud Backend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.2.4 RainMaker Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.3 Practice: Key Points for Developing with ESP RainMaker . . . . . . . . . . . . 25

3.4 Features of ESP RainMaker . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.4.1 User Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.4.2 End User Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.4.3 Admin Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4 Setting Up Development Environment

31

4.1 ESP-IDF Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.1.1 ESP-IDF Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3

4.1.2 ESP-IDF Git Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 4.1.3 Choosing a Suitable Version . . . . . . . . . . . . . . . . . . . . . . . . 34 4.1.4 Overview of ESP-IDF SDK Directory . . . . . . . . . . . . . . . . . . . . 34 4.2 Setting Up ESP-IDF Development Environment . . . . . . . . . . . . . . . . . 38 4.2.1 Setting up ESP-IDF Development Environment on Linux . . . . . . . . 38 4.2.2 Setting up ESP-IDF Development Environment on Windows . . . . . . 40 4.2.3 Setting up ESP-IDF Development Environment on Mac . . . . . . . . . 45 4.2.4 Installing VS Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 4.2.5 Introduction to Third-Party Development Environments . . . . . . . . 46 4.3 ESP-IDF Compilation System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 4.3.1 Basic Concepts of Compilation System . . . . . . . . . . . . . . . . . . 47 4.3.2 Project File Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 4.3.3 Default Build Rules of the Compilation System . . . . . . . . . . . . . 50 4.3.4 Introduction to the Compilation Script . . . . . . . . . . . . . . . . . . 51 4.3.5 Introduction to Common Commands . . . . . . . . . . . . . . . . . . . 52 4.4 Practice: Compiling Example Program “Blink” . . . . . . . . . . . . . . . . . . 53 4.4.1 Example Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 4.4.2 Compiling the Blink Program . . . . . . . . . . . . . . . . . . . . . . . 56 4.4.3 Flashing the Blink Program . . . . . . . . . . . . . . . . . . . . . . . . 59 4.4.4 Serial Port Log Analysis of the Blink Program . . . . . . . . . . . . . . 60 4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

II Hardware and Driver Development

65

5 Hardware Design of Smart Light Products based on ESP32-C3

67

5.1 Features and Composition of Smart Light Products . . . . . . . . . . . . . . . 67

5.2 Hardware Design of ESP32-C3 Core System . . . . . . . . . . . . . . . . . . . 70

5.2.1 Power Supply . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

5.2.2 Power-on Sequence and System Reset . . . . . . . . . . . . . . . . . . 74

5.2.3 SPI Flash . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

5.2.4 Clock Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

5.2.5 RF and Antenna . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

5.2.6 Strapping Pins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

5.2.7 GPIO and PWM Controller . . . . . . . . . . . . . . . . . . . . . . . . . 79

5.3 Practice: Building a Smart Light System with ESP32-C3 . . . . . . . . . . . . . 80

5.3.1 Selecting Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

5.3.2 Configuring GPIOs of PWM Signals . . . . . . . . . . . . . . . . . . . . 82

5.3.3 Downloading Firmware and Debugging Interface . . . . . . . . . . . . 82

5.3.4 Guidelines for RF Design . . . . . . . . . . . . . . . . . . . . . . . . . . 84 5.3.5 Guidelines for Power Supply Design . . . . . . . . . . . . . . . . . . . 86 5.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

6 Driver Development

87

6.1 Driver Development Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

6.2 ESP32-C3 Peripheral Applications . . . . . . . . . . . . . . . . . . . . . . . . . 88

6.3 LED Driver Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

6.3.1 Color Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

6.3.2 LED Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

6.3.3 LED Dimming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

6.3.4 Introduction to PWM . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

6.4 LED Dimming Driver Development . . . . . . . . . . . . . . . . . . . . . . . . 96

6.4.1 Non-Volatile Storage (NVS) . . . . . . . . . . . . . . . . . . . . . . . . 97

6.4.2 LED PWM Controller (LEDC) . . . . . . . . . . . . . . . . . . . . . . . 98

6.4.3 LED PWM Programming . . . . . . . . . . . . . . . . . . . . . . . . . . 100

6.5 Practice: Adding Drivers to Smart Light Project . . . . . . . . . . . . . . . . . 103

6.5.1 Button Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

6.5.2 LED Dimming Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

III Wireless Communication and Control

109

7 Wi-Fi Configuration and Connection

111

7.1 Basics of Wi-Fi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

7.1.1 Introduction to Wi-Fi . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

7.1.2 Evolution of IEEE 802.11 . . . . . . . . . . . . . . . . . . . . . . . . . 111

7.1.3 Wi-Fi Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

7.1.4 Wi-Fi Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

7.2 Basics of Bluetooth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

7.2.1 Introduction to Bluetooth . . . . . . . . . . . . . . . . . . . . . . . . . 123

7.2.2 Bluetooth Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

7.2.3 Bluetooth Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

7.3 Wi-Fi Network Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

7.3.1 Wi-Fi Network Configuration Guide . . . . . . . . . . . . . . . . . . . . 131

7.3.2 SoftAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

7.3.3 SmartConfig . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

7.3.4 Bluetooth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

7.3.5 Other Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

7.4 Wi-Fi Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 7.4.1 Wi-Fi Components in ESP-IDF . . . . . . . . . . . . . . . . . . . . . . . 139 7.4.2 Exercise: Wi-Fi Connection . . . . . . . . . . . . . . . . . . . . . . . . 141 7.4.3 Exercise: Smart Wi-Fi Connection . . . . . . . . . . . . . . . . . . . . . 145
7.5 Practice: Wi-Fi Configuration in Smart Light Project . . . . . . . . . . . . . . . 156 7.5.1 Wi-Fi Connection in Smart Light Project . . . . . . . . . . . . . . . . . 156 7.5.2 Smart Wi-Fi Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 157
7.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

8 Local Control

159

8.1 Introduction to Local Control . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

8.1.1 Application of Local Control . . . . . . . . . . . . . . . . . . . . . . . . 161

8.1.2 Advantages of Local Control . . . . . . . . . . . . . . . . . . . . . . . . 161

8.1.3 Discovering Controlled Devices through Smartphones . . . . . . . . . . 161

8.1.4 Data Communication Between Smartphones and Devices . . . . . . . . 162

8.2 Common Local Discovery Methods . . . . . . . . . . . . . . . . . . . . . . . . 162

8.2.1 Broadcast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

8.2.2 Multicast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

8.2.3 Comparison Between Broadcast and Multicast . . . . . . . . . . . . . . 176

8.2.4 Multicast Application Protocol mDNS for Local Discovery . . . . . . . . 176

8.3 Common Communication Protocols for Local Data . . . . . . . . . . . . . . . 179

8.3.1 Transmission Control Protocol (TCP) . . . . . . . . . . . . . . . . . . . 179

8.3.2 HyperText Transfer Protocol (HTTP) . . . . . . . . . . . . . . . . . . . 185

8.3.3 User Datagram Protocol (UDP) . . . . . . . . . . . . . . . . . . . . . . 189

8.3.4 Constrained Application Protocol (CoAP) . . . . . . . . . . . . . . . . 192

8.3.5 Bluetooth Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

8.3.6 Summary of Data Communication Protocols . . . . . . . . . . . . . . . 203

8.4 Guarantee of Data Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

8.4.1 Introduction to Transport Layer Security (TLS) . . . . . . . . . . . . . 207

8.4.2 Introduction to Datagram Transport Layer Security (DTLS) . . . . . . . 213

8.5 Practice: Local Control in Smart Light Project . . . . . . . . . . . . . . . . . . 217

8.5.1 Creating a Wi-Fi-based Local Control Server . . . . . . . . . . . . . . . 217

8.5.2 Verifying Local Control Functionality using Scripts . . . . . . . . . . . 221

8.5.3 Creating a Bluetooth-based Local Control Server . . . . . . . . . . . . 222

8.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

9 Cloud Control

225

9.1 Introduction to Remote Control . . . . . . . . . . . . . . . . . . . . . . . . . . 225

9.2 Cloud Data Communication Protocols . . . . . . . . . . . . . . . . . . . . . . 226

9.2.1 MQTT Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 9.2.2 MQTT Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 9.2.3 MQTT Message Format . . . . . . . . . . . . . . . . . . . . . . . . . . 228 9.2.4 Protocol Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 9.2.5 Setting Up MQTT Broker on Linux and Windows . . . . . . . . . . . . 233 9.2.6 Setting Up MQTT Client Based on ESP-IDF . . . . . . . . . . . . . . . . 235 9.3 Ensuring MQTT Data Security . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 9.3.1 Meaning and Function of Certificates . . . . . . . . . . . . . . . . . . . 237 9.3.2 Generating Certificates Locally . . . . . . . . . . . . . . . . . . . . . . 239 9.3.3 Configuring MQTT Broker . . . . . . . . . . . . . . . . . . . . . . . . . 241 9.3.4 Configuring MQTT Client . . . . . . . . . . . . . . . . . . . . . . . . . 241 9.4 Practice: Remote Control through ESP RainMaker . . . . . . . . . . . . . . . . 243 9.4.1 ESP RainMaker Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 9.4.2 Node and Cloud Backend Communication Protocol . . . . . . . . . . . 244 9.4.3 Communication between Client and Cloud Backend . . . . . . . . . . . 249 9.4.4 User Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 9.4.5 Basic Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 9.4.6 Smart Light Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 9.4.7 RainMaker App and Third- Party Integrations . . . . . . . . . . . . . . . 262 9.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

10 Smartphone App Development

269

10.1 Introduction to Smartphone App Development . . . . . . . . . . . . . . . . . . 269

10.1.1 Overview of Smartphone App Development . . . . . . . . . . . . . . . 270

10.1.2 Structure of the Android Project . . . . . . . . . . . . . . . . . . . . . . 270

10.1.3 Structure of the iOS Project . . . . . . . . . . . . . . . . . . . . . . . . 271

10.1.4 Lifecycle of an Android Activity . . . . . . . . . . . . . . . . . . . . . . 272

10.1.5 Lifecycle of iOS ViewController . . . . . . . . . . . . . . . . . . . . . . 273

10.2 Creating a New Smartphone App Project . . . . . . . . . . . . . . . . . . . . . 275

10.2.1 Preparing for Android Development . . . . . . . . . . . . . . . . . . . 275

10.2.2 Creating a New Android Project . . . . . . . . . . . . . . . . . . . . . . 275

10.2.3 Adding Dependencies for MyRainmaker . . . . . . . . . . . . . . . . . 276

10.2.4 Permission Request in Android . . . . . . . . . . . . . . . . . . . . . . 277

10.2.5 Preparing for iOS Development . . . . . . . . . . . . . . . . . . . . . . 277

10.2.6 Creating a New iOS Project . . . . . . . . . . . . . . . . . . . . . . . . 278

10.2.7 Adding Dependencies for MyRainmaker . . . . . . . . . . . . . . . . . 279

10.2.8 Permission Request in iOS . . . . . . . . . . . . . . . . . . . . . . . . . 280

10.3 Analysis of the App’s Functional Requirements . . . . . . . . . . . . . . . . . . 281

10.3.1 Analysis of the Project’s Functional Requirements . . . . . . . . . . . . 282

10.3.2 Analysis of User Management Requirements . . . . . . . . . . . . . . . 282 10.3.3 Analysis of Device Provisioning and Binding Requirements . . . . . . . 283 10.3.4 Analysis of Remote-Control Requirements . . . . . . . . . . . . . . . . 283 10.3.5 Analysis of Scheduling Requirements . . . . . . . . . . . . . . . . . . . 284 10.3.6 Analysis of User Centre Requirements . . . . . . . . . . . . . . . . . . 285 10.4 Development of User Management . . . . . . . . . . . . . . . . . . . . . . . . 285 10.4.1 Introduction to RainMaker APIs . . . . . . . . . . . . . . . . . . . . . . 285 10.4.2 Initiating Communication via Smartphone . . . . . . . . . . . . . . . . 286 10.4.3 Account Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286 10.4.4 Account Login . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 10.5 Development of Device Provisioning . . . . . . . . . . . . . . . . . . . . . . . 292 10.5.1 Scanning Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 10.5.2 Connecting Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295 10.5.3 Generating Secret Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . 298 10.5.4 Getting Node ID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298 10.5.5 Provisioning Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300 10.6 Development of Device Control . . . . . . . . . . . . . . . . . . . . . . . . . . 302 10.6.1 Binding Devices to Cloud Accounts . . . . . . . . . . . . . . . . . . . . 303 10.6.2 Getting a List of Devices . . . . . . . . . . . . . . . . . . . . . . . . . . 305 10.6.3 Getting Device Status . . . . . . . . . . . . . . . . . . . . . . . . . . . 308 10.6.4 Changing Device Status . . . . . . . . . . . . . . . . . . . . . . . . . . 310 10.7 Development of Scheduling and User Centre . . . . . . . . . . . . . . . . . . . 313 10.7.1 Implementing Scheduling Function . . . . . . . . . . . . . . . . . . . . 313 10.7.2 Implementing User Centre . . . . . . . . . . . . . . . . . . . . . . . . . 315 10.7.3 More Cloud APIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318 10.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319

11 Firmware Upgrade and Version Management

321

11.1 Firmware Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321

11.1.1 Overview of Partition Tables . . . . . . . . . . . . . . . . . . . . . . . . 322

11.1.2 Firmware Boot Process . . . . . . . . . . . . . . . . . . . . . . . . . . . 324

11.1.3 Overview of the OTA Mechanism . . . . . . . . . . . . . . . . . . . . . 326

11.2 Firmware Version Management . . . . . . . . . . . . . . . . . . . . . . . . . . 329

11.2.1 Firmware Marking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329

11.2.2 Rollback and Anti-Rollback . . . . . . . . . . . . . . . . . . . . . . . . 331

11.3 Practice: Over-the-air (OTA) Example . . . . . . . . . . . . . . . . . . . . . . . 332

11.3.1 Upgrade Firmware Through a Local Host . . . . . . . . . . . . . . . . . 332

11.3.2 Upgrade Firmware Through ESP RainMaker . . . . . . . . . . . . . . . 335

11.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342

IV Optimisation and Mass Production

343

12 Power Management and Low-Power Optimisation

345

12.1 ESP32-C3 Power Management . . . . . . . . . . . . . . . . . . . . . . . . . . . 345

12.1.1 Dynamic Frequency Scaling . . . . . . . . . . . . . . . . . . . . . . . . 346

12.1.2 Power Management Configuration . . . . . . . . . . . . . . . . . . . . 348

12.2 ESP32-C3 Low-Power Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348

12.2.1 Modem-sleep mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349

12.2.2 Light-sleep Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351

12.2.3 Deep-sleep mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356

12.2.4 Current Consumption in Different Power Modes . . . . . . . . . . . . . 358

12.3 Power Management and Low-Power Debugging . . . . . . . . . . . . . . . . . 359

12.3.1 Log Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360

12.3.2 GPIO Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362

12.4 Practice: Power Management in Smart Light Project . . . . . . . . . . . . . . . 363

12.4.1 Configuring Power Management Feature . . . . . . . . . . . . . . . . . 364

12.4.2 Use Power Management Locks . . . . . . . . . . . . . . . . . . . . . . 365

12.4.3 Verifying Power Consumption . . . . . . . . . . . . . . . . . . . . . . . 366

12.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367

13 Enhanced Device Security Features

369

13.1 Overview of IoT Device Data Security . . . . . . . . . . . . . . . . . . . . . . . 369

13.1.1 Why Securing IoT Device Data? . . . . . . . . . . . . . . . . . . . . . . 370

13.1.2 Basic Requirements for IoT Device Data Security . . . . . . . . . . . . 371

13.2 Data Integrity Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372

13.2.1 Introduction to Integrity Verification Method . . . . . . . . . . . . . . 372

13.2.2 Integrity Verification of Firmware Data . . . . . . . . . . . . . . . . . . 373

13.2.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374

13.3 Data Confidentiality Protection . . . . . . . . . . . . . . . . . . . . . . . . . . 374

13.3.1 Introduction to Data Encryption . . . . . . . . . . . . . . . . . . . . . . 374

13.3.2 Introduction to Flash Encryption Scheme . . . . . . . . . . . . . . . . . 376

13.3.3 Flash Encryption Key Storage . . . . . . . . . . . . . . . . . . . . . . . 379

13.3.4 Working Mode of Flash Encryption . . . . . . . . . . . . . . . . . . . . 380

13.3.5 Flash Encryption Process . . . . . . . . . . . . . . . . . . . . . . . . . . 381

13.3.6 Introduction to NVS Encryption . . . . . . . . . . . . . . . . . . . . . . 383

13.3.7 Examples of Flash Encryption and NVS Encryption . . . . . . . . . . . 384

13.4 Data Legitimacy Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386

13.4.1 Introduction to Digital Signature . . . . . . . . . . . . . . . . . . . . . 386

13.4.2 Overview of Secure Boot Scheme . . . . . . . . . . . . . . . . . . . . . 388

13.4.3 Introduction to Software Secure Boot . . . . . . . . . . . . . . . . . . . 388 13.4.4 Introduction to Hardware Secure Boot . . . . . . . . . . . . . . . . . . 390 13.4.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394 13.5 Practice: Security Features In Mass Production . . . . . . . . . . . . . . . . . . 396 13.5.1 Flash Encryption and Secure Boot . . . . . . . . . . . . . . . . . . . . . 396 13.5.2 Enabling Flash Encryption and Secure Boot with Batch Flash Tools . . 397 13.5.3 Enabling Flash Encryption and Secure Boot in Smart Light Project . . . 398 13.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398

14 Firmware Burning and Testing for Mass Production

399

14.1 Firmware Burning in Mass Production . . . . . . . . . . . . . . . . . . . . . . 399

14.1.1 Defining Data Partitions . . . . . . . . . . . . . . . . . . . . . . . . . . 399

14.1.2 Firmware Burning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402

14.2 Mass Production Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403

14.3 Practice: Mass Production Data in Smart Light Project . . . . . . . . . . . . . 404

14.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404

15 ESP Insights: Remote Monitoring Platform

405

15.1 Introduction to ESP Insights . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405

15.2 Getting Started with ESP Insights . . . . . . . . . . . . . . . . . . . . . . . . . 409

15.2.1 Getting Started with ESP Insights in the esp-insights Project . . . . . . 409

15.2.2 Running Example in the esp-insights Project . . . . . . . . . . . . . . . 411

15.2.3 Reporting Coredump Information . . . . . . . . . . . . . . . . . . . . . 411

15.2.4 Customising Logs of Interest . . . . . . . . . . . . . . . . . . . . . . . . 412

15.2.5 Reporting Reboot Reason . . . . . . . . . . . . . . . . . . . . . . . . . 413

15.2.6 Reporting Custom Metrics . . . . . . . . . . . . . . . . . . . . . . . . . 413

15.3 Practice: Using ESP Insights in Smart Light Project . . . . . . . . . . . . . . . 416

15.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417

Introduction
ESP32-C3 is a single-core Wi-Fi and Bluetooth 5 (LE) microcontroller SoC, based on the open-source RISC-V architecture. It strikes the right balance of power, I/O capabilities, and security, thus offering the optimal cost- effective solution for connected devices. To show various applications of the ESP32-C3 family, this book by Espressif will take you on an interesting journey through AIoT, starting from the basics of IoT project development and environment setup to practical examples. The first four chapters talk about IoT, ESP RainMaker and ESP-IDF. Chapter 5 and 6 brief on hardware design and driver development. As you progress, you’ll discover how to configure your project through Wi-Fi networks and mobile Apps. Finally, you’ll learn to optimize your project and put it into mass production.
If you are an engineer in related fields, a software architect, a teacher, a student, or anyone who has an interest in IoT, this book is for you.
You may download the code example used in this book from Espressif’s site on GitHub. For latest information on IoT development, please follow our official account.

Preface
An Informatising World
Riding the wave of Internet, Internet of Things (IoT) made its grand debut to become a new type of infrastructure in digital economy. To bring the technology closer to the public, Espressif Systems works for the vision that developers from all walks of life can use IoT to solve some of the most pressing problems of our times. A world of “Intelligent Network of All Things” is what we are expecting from the future.
Designing our own chips makes a critical component of that vision. It is to be a marathon, requiring constant breakthroughs against technological boundaries. From the “Game Changer” ESP8266 to the ESP32 series integrating Wi-Fi and Bluetoothr (LE) connectivity, followed by ESP32-S3 equipped by AI acceleration, Espressif never stops researching and developing products for AIoT solutions. With our open-source software, such as the IoT Development Framework ESP-IDF, Mesh Development Framework ESP-MDF, and Device Connectivity Platform ESP RainMaker, we have created an independent framework for building AIoT applications.
As of July 2022, the cumulative shipments of Espressif’s IoT chipsets have exceeded 800 million, leading in the Wi-Fi MCU market and powering up a huge number of connected devices worldwide. Pursuit for excellence makes every Espressif product a big hit for its high level of integration and cost efficiency. The release of ESP32-C3 marks a significant milestone of Espressif’s self-developed technology. It is a single-core, 32-bit, RISC-V-based MCU with 400KB of SRAM, which can run at 160MHz. It has integrated 2.4 GHz Wi-Fi and Bluetooth 5 (LE) with a long-range support. It strikes a fine balance of power, I/O capabilities, and security, thus offering the optimal cost-effective solution for connected devices. Based on such powerful ESP32-C3, this book is intended to help readers understand IoT- related knowledge with detailed illustration and practical examples.
Why we wrote this book?
Espressif Systems is more than a semiconductor company. It is also an IoT platform company, which always strives for breakthroughs and innovations in the field of technology. At the same time, Espressif has open-sourced and shared its self-developed operating system and software framework with the community, forming a unique ecosystem. Engineers, makers, and technology enthusiasts actively develop new software applications based on Espressif’s products, freely communicate, and share their experience. You can see developers’ fascinating ideas on various platforms all the time, such as YouTube and GitHub. The popularity of Espressif’s products has stimulated an increasing number of authors who have produced over 100 books based on Espressif chipsets, in more than ten languages, including English, Chinese, German, French, and Japanese.

It is the support and trust of community partners that encourages Espressif’s continuous innovation. “We strive to make our chips, operating systems, frameworks, solutions, Cloud, business practices, tools, documentation, writings, ideas, etc., ever more relevant to the answers people need in contemporary life’s most pressing problems. This is Espressif’s highest ambition and moral compass.” said Mr. Teo Swee Ann, Founder and CEO of Espressif.
Espressif values reading and ideas. As the continuous upgrading of IoT technology poses higher requirements on engineers, how can we help more people to quickly master IoT chips, operating systems, software frameworks, application schemes and cloud service products? As the saying goes, it is better to teach a man how to fish than to give him fish. In a brainstorming session, it occurred to us that we could write a book to systematically sort out the key knowledge of IoT development. We hit it off, quickly gathered a group of senior engineers, and combined the experience of the technical team in embedded programming, IoT hardware and software development, all contributing to the publishing of this book. In the process of writing, we tried our best to be objective and fair, stripped of the cocoon, and use concise expressions to tell the complexity and charm of the Internet of Things. We carefully summarised the common questions, referred to the feedback and suggestions of the community, in order to clearly answer the questions encountered in the development process, and provide practical IoT development guidelines for relevant technicians and decision-makers.
Book Structure
This book takes an engineer-centred perspective and expounds the necessary knowledge for IoT project development step by step. It is composed of four parts, as follows:
· Preparation (Chapter 1­4): This part introduces the architecture of IoT, typical IoT project framework, the ESP RainMakerr cloud platform, and the development environment ESP-IDF, so as to lay a solid foundation for IoT project development.
· Hardware and Driver Development (Chapter 5­6): Based on the ESP32-C3 chipset, this part elaborates on the minimum hardware system and driver development, and implements the control of dimming, colour grading, and wireless communication.
· Wireless Communication and Control (Chapter 7­11): This part explains the intelligent Wi-Fi configuration scheme based on ESP32-C3 chip, local & cloud control protocols, and local & remote control of devices. It also provides schemes for developing smartphone apps, firmware upgrade, and version management.
· Optimisation and Mass Production (Chapter 12-15): This part is intended for advanced IoT applications, focusing on optimisation of products in power management, low-power optimisation, and enhanced security. It also introduces firmware burning and testing in mass production, and how to diagnose the running status and logs of device firmware through the remote monitoring platform ESP Insights.

About the Source Code
Readers can run the example programmes in this book, either by entering the code manually or by using the source code that accompanies the book. We emphasise the combination of theory and practice, and thus set a Practice section based on the Smart Light project in almost every chapter. All the codes are open-sourced. Readers are welcome to download the source code and discuss it in the sections related to this book on GitHub and our official forum esp32.com. The open-sourced code of this book is subject to the terms of Apache License 2.0.
Author’s Note
This book is officially produced by Espressif Systems and is written by the company’s senior engineers. It is suitable for managers and R&D personnel in IoT-related industries, teachers and students of related majors, and enthusiasts in the field of Internet of Things. We hope that this book can serve as a work manual, a reference, and a bedside book, to be like a good tutor and friend.
While compiling this book, we referred to some relevant research results of experts, scholars, and technicians at home and abroad, and we did our best to cite them according to academic norms. However, it is unavoidable that there should be some omissions, so here we would like to express our deep respect and gratitude to all the relevant authors. In addition, we have quoted information from the Internet, so we would like to thank the original authors and publishers and apologise that we cannot indicate the source of every piece of information.
In order to produce a book of high quality, we have organised rounds of internal discussions, and learned from the suggestions and feedback of trial readers and publisher editors. Here, we would like to thank you again for your help which all contributed to this successful work.
Last, but the most importantly, thanks to everyone at Espressif who has worked so hard for the birth and popularization of our products.
The development of IoT projects involves a wide range of knowledge. Limited to the length of the book, as well as the level and experience of the author, omissions are unavoidable. Therefore, we kindly request that experts and readers criticise and correct our mistakes. If you have any suggestions for this book, please contact us at book@espressif.com. We look forward to your feedback.

How to use this book?
The code of the projects in this book has been open sourced. You can download it from our GitHub repository and share your thoughts and questions on our official forum. GitHub: https://github.com/espressif/book-esp32c3-iot-projects Forum: https://www.esp32.com/bookc3 Throughout the book, there will be parts highlighted as shown below.
Source code In this book, we emphasise the combination of theory and practice, and thus set a Practice section about the Smart Light project in almost every chapter. Corresponding steps and source page will be marked between two lines beginning with the tag Source code.
NOTE/TIPS This is where you may find some critical information and reminding for successfully debugging your programme. They will be marked between two thick lines beginning with the tag NOTE or TIPS.
Most of the commands in this book are executed under Linux, prompted by the character “$”. If the command requires superuser privileges to execute, the prompt will be replaced by “#”. The command prompt on Mac systems is “%”, as used in Section 4.2.3 Installing ESP-IDF on Mac.
The body text in this book will be printed in Charter, while the code examples, components, functions, variables, code file names, code directories, and strings will be in Courier New.
Commands or texts that need to be input by the user, and commands that can be entered by pressing the “Enter” key will be printed in Courier New bold. Logs and code blocks will be presented in light blue boxes .
Example:
Second, use esp-idf/components/nvs flash/nvs partition generator/nvs partition gen.py to generate the NVS partition binary file on the development host with the following command:
$ python $IDF PATH/components/nvs flash/nvs partition generator/nvs partition gen.py –input mass prod.csv –output mass prod.bin –size NVS PARTITION SIZE

Chapter 1

Introduction

to

IoT

At the end of the 20th century, with the rise of computer networks and communication technologies, Internet rapidly integrated into people’s lives. As Internet technology continues to mature, the idea of Internet of Things (IoT) was born. Literally, IoT means an Internet where things are connected. While the original Internet breaks the limits of space and time and narrows the distance between “person and person”, IoT makes “things” an important participant, bringing “people” and “things” closer together. In the foreseeable future, IoT is set to become the driving force of the information industry.
So, what is the Internet of Things?
It is hard to accurately define the Internet of Things, as its meaning and scope are constantly evolving. In 1995, Bill Gates first brought up the idea of IoT in his book The Road Ahead. Simply put, IoT enables objects to exchange information with each other through Internet. Its ultimate goal is to establish an “Internet of Everything”. This is an early interpretation of IoT, as well as a fantasy of future technology. Thirty years later, with the rapid development of economy and technology, the fantasy is coming into reality. From smart devices, smart homes, smart cities, Internet of Vehicles and wearable devices, to the “metaverse” supported by IoT technologies, new concepts are constantly emerging. In this chapter, we will begin with an explanation of the architecture of Internet of Things, and then introduce the most common IoT application, the smart home, in order to help you get a clear understanding of IoT.
1.1 Architecture of IoT
Internet of Things involves multiple technologies which have different application needs and forms in different industries. In order to sort out the structure, the key technologies and application characteristics of IoT, it is necessary to establish a unified architecture and a standard technical system. In this book, the architecture of IoT is simply divided into four layers: perception & control layer, network layer, platform layer, and application layer.
Perception & Control Layer As the most basic element of the IoT architecture, perception & control layer is the core to realise the comprehensive sensing of IoT. Its main function is to collect, identify and control information. It consists of a variety of devices with the ability of perception,
3

identification, control and execution, and is responsible for retrieving and analysing data such as material properties, behavioural trends, and device status. In this way, IoT gets to recognise the real physical world. Besides, the layer is also able to control the status of the device.
The most common devices of this layer are various sensors, which play an important role in information collection and identification. Sensors are like human sensory organs, such as photosensitive sensors equalling to vision, acoustic sensors to hearing, gas sensors to smelling, and pressure- and temperature-sensitive sensors to touching. With all these “sensory organs”, objects become “alive” and capable of intelligent perception, recognition and manipulation of the physical world.
Network Layer The main function of the network layer is to transmit information, including data obtained from the perception & control layer to specified target, as well as commands issued from the application layer back to the perception & control layer. It serves as an important communication bridge connecting different layers of an IoT system. To set up a basic model of Internet of Things, it involves two steps to integrate objects into a network: access to Internet and transmission through Internet.
Access to Internet Internet enables interconnection between person and person, but fails to include things into the big family. Before the advent of IoT, most things were not “network-able”. Thanks to the continuous development of technology, IoT manages to connect things to the Internet, thus realizing interconnection between “people and things”, and “things and things”. There are two common ways to implement Internet connection: wired network access and wireless network access.
Wired network access methods include Ethernet, serial communication (e.g., RS-232, RS-485) and USB, while wireless network access depends on wireless communication, which can be further divided into short-range wireless communication and long-range wireless communication.
Short-range wireless communication includes ZigBee, Bluetoothr, Wi-Fi, Near- Field Communication (NFC), and Radio Frequency Identification (RFID). Long- range wireless communication includes Enhanced Machine Type Communication (eMTC), LoRa, Narrow Band Internet of Things (NB-IoT), 2G, 3G, 4G, 5G, etc.
Transmission through Internet Different methods of Internet access lead to corresponding physical transmission link of data. The next thing is to decide which communication protocol to use to transmit the data. Compared with Internet terminals, most IoT terminals currently have fewer
4 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

available resources, such as processing performance, storage capacity, network rate, etc., so it is necessary to choose a communication protocol that occupies fewer resources in IoT applications. There are two communication protocols that are widely used today: Message Queuing Telemetry Transport (MQTT) and Constrained Application Protocol (CoAP).
Platform Layer The platform layer mainly refers to IoT cloud platforms. When all IoT terminals are networked, their data need to be aggregated on an IoT cloud platform to be calculated and stored. The platform layer mainly supports IoT applications in facilitating access and management of massive devices. It connects IoT terminals to the cloud platform, collects terminal data, and issues commands to terminals, so as to implement remote control. As an intermediate service to assign equipment to industry applications, the platform layer plays a connecting role in the entire IoT architecture, carrying abstract business logic and standardized core data model, which can not only realize rapid access of devices, but also provide powerful modular capabilities to meet various needs in industry application scenarios. The platform layer mainly includes functional modules such as device access, device management, security management, message communication, monitoring operation and maintenance, and data applications.
· Device access, realising the connection and communication between terminals and IoT cloud platforms.
· Device management, including functions such as device creation, device maintenance, data conversion, data synchronization, and device distribution.
· Security management, ensuring the security of IoT data transmission from the perspectives of security authentication and communication security.
· Message communication, including three transmission directions, that is, the terminal sends data to the IoT cloud platform, the IoT cloud platform sends data to the server side or other IoT cloud platforms, and the server side remotely controls IoT devices.
· Monitoring O&M, involving monitoring and diagnosis, firmware upgrade, online debugging, log services, etc.
· Data applications, involving the storage, analysis and application of data.
Application Layer The application layer uses the data from the platform layer to manage the application, filtering and processing them with tools such as databases and analysis software. The resulting data can be used for real-world IoT applications such as smart healthcare, smart agriculture, smart homes, and smart cities.
Of course, the architecture of IoT can be subdivided into more layers, but no matter how many layers it consists of, the underlying principle remains essentially the same. Learning
Chapter 1. Introduction to IoT 5

about the architecture of IoT helps deepen our understanding of IoT technologies and build fully functional IoT projects.
1.2 IoT Application in Smart Homes
IoT has penetrated into all walks of life, and the most closely related IoT application to us is the smart home. Many traditional appliances are now equipped with one or more IoT devices, and many newly built houses are designed with IoT technologies from the start. Figure 1.1 shows some common smart home devices.
Figure 1.1. Common smart home devices The development of smart home can be simply divided into smart product stage, scene interconnection stage and intelligent stage, as shown in Figure 1.2.
Figure 1.2. Development stage of smart home 6 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

The first stage is about smart products. Different from traditional homes, in smart homes, IoT devices receive signals with sensors, and are networked through wireless communication technologies such as Wi-Fi, Bluetooth LE, and ZigBee. Users can control smart products in a variety of ways, such as smartphone apps, voice assistants, smart speaker control, etc. The second stage focuses on scene interconnection. In this stage, developers are no longer considering controlling single smart product, but interconnecting two or more smart products, automating to a certain extent, and finally forming a custom scene mode. For example, when the user presses any scene mode button, the lights, curtains, and air conditioners will be automatically adapted to the presets. Of course, there is the prerequisite that the linkage logic are readily set up, including trigger conditions and execution actions. Imagine that the air conditioning heating mode is triggered when the indoor temperature drops below 10°C; that at 7 o’clock in the morning, music is played to wake up the user, smart curtains are opened, and the rice cooker or bread toaster starts through a smart socket; as the user gets up and finishes washing, breakfast is already served, so that there will be no delay in going to work. How convenient has our life become! The third stage goes to intelligence stage. As more smart home devices are accessed, so will the types of data generated. With the help of cloud computing, big data and artificial intelligence, it is like a “smarter brain” has been planted into smart homes, which no longer require frequent commands from the user. They collect data from previous interactions and learn the user’s behaviour patterns and preferences, so as to automate activities, including providing recommendations for decision-making. Currently, most smart homes are at the scene interconnection stage. As the penetration rate and intelligence of smart products increase, barriers between communication protocols are being removed. In the future, smart homes are bound to become really “smart”, just like the AI system Jarvis in Iron Man, which can not only help the user control various devices, handle daily affairs, but also have super computing power and thinking ability. In the intelligent stage, human beings will receive better services both in quantity and quality.
Chapter 1. Introduction to IoT 7

8 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Chapter Introduction and Practice of 2 IoT Projects
In Chapter 1, we introduced the architecture of IoT, and the roles and interrelationships of the perception & control layer, network layer, platform layer, and application layer, as well as the development of smart home. However, just like when we learn to paint, knowing the theoretical knowledge is far from enough. We have to “get our hands dirty” to put IoT projects into practice in order to truly master the technology. In addition, when a project moves to the mass production stage, it is necessary to consider more factors such as network connection, configuration, IoT cloud platform interaction, firmware management and updates, mass production management, and security configuration. So, what do we need to pay attention to when developing a complete IoT project? In Chapter 1, we mentioned that smart home is one of the most common IoT application scenarios, and smart lights are one of the most basic and practical appliances, which can be used in homes, hotels, gyms, hospitals, etc. Therefore, in this book, we will take the construction of a smart light project as the starting point, explain its components and features, and provide guidance on project development. We hope that you can draw inferences from this case to create more IoT applications.
2.1 Introduction to Typical IoT Projects
In terms of development, basic functional modules of IoT projects can be classified into software and hardware development of IoT devices, client application development, and IoT cloud platform development. It is important to clarify the basic functional modules, which will be further described in this section.
2.1.1 Basic Modules for Common IoT Devices
Software and hardware development of IoT devices include the following basic modules: Data collection
As the bottom layer of the IoT architecture, the IoT devices of the perception & control layer connect sensors and devices through their chips and peripherals to achieve data collection and operation control.
9

Account binding and initial configuration For most IoT devices, account binding and initial configuration are completed in one operational process, for example, connecting devices with users by configuring Wi-Fi network.
Interaction with IoT cloud platforms To monitor and control IoT devices, it is also necessary to connect them to IoT cloud platforms, in order to give commands and report status through interaction between each other.
Device control When connected with IoT cloud platforms, devices can communicate with the cloud and be registered, bound, or controlled. Users can query product status and carry out other operations on the smartphone app through IoT cloud platforms or local communication protocols.
Firmware upgrade IoT devices can also achieve firmware upgrade based on manufacturers’ needs. By receiving commands sent by the cloud, firmware upgrade and version management will be realized. With this firmware upgrade feature, you can continuously enhance the functions of IoT devices, fix defects, and improve user experience.
2.1.2 Basic Modules of Client Applications
Client applications (e.g., smartphone apps) mainly include the following basic modules:
Account system and authorisation It supports account and device authorisation.
Device control Smartphone apps are usually equipped with controlling functions. Users can easily connect to IoT devices, and manage them anytime, anywhere through smartphone apps. In a real-world smart home, devices are mostly controlled through smartphone apps, which not only enables intelligent management of devices, but also saves the cost of manpower. Therefore, device control is a must for client applications, such as device function attribute control, scene control, scheduling, remote control, device linkage, etc. Smart home users can also customise scenes according to personal needs, controlling lighting, home appliances, entrance, etc., to make home life more comfortable and convenient. They can time air conditioning, turn off it remotely, set the hallway light on automatically once the door is unlocked, or switch to the “theater” mode with one single button.
Notification Client applications update real-time status of IoT devices, and send alerts when devices go abnormal.
10 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

After-sales customer service Smartphone apps can provide after-sales services for products, to solve problems related to IoT device failures and technical operations in a timely manner.
Featured functions To meet the needs of different users, other functions may be added, such as Shake, NFC, GPS, etc. GPS can help set the accuracy of scene operations according to location and distance, while the Shake function allows users to set the commands to be executed for specific device or scene by shaking.
2.1.3 Introduction to Common IoT Cloud Platforms
IoT cloud platform is an all-in-one platform which integrates functions such as device management, data security communication, and notification management. According to their target group and accessibility, IoT cloud platforms can be divided into public IoT cloud platforms (hereinafter referred to as “public cloud”) and private IoT cloud platforms (hereinafter referred to as “private cloud”).
Public cloud usually indicates shared IoT cloud platforms for enterprises or individuals, operated and maintained by platform providers, and shared through the Internet. It can be free or low-cost, and provides services throughout the open public network, such as Alibaba Cloud, Tencent Cloud, Baidu Cloud, AWS IoT, Google IoT, etc. As a supporting platform, public cloud can integrate upstream service providers and downstream end users to create a new value chain and ecosystem.
Private cloud is built for enterprise use only, thus guaranteeing the best control over data, security, and service quality. Its services and infrastructure are maintained separately by enterprises, and the supporting hardware and software are also dedicated to specific users. Enterprises can customise cloud services to meet the needs of their business. At present, some smart home manufacturers have already got private IoT cloud platforms and developed smart home applications based on them.
Public cloud and private cloud have their own advantages, which will be explained later.
To achieve communication connectivity, it is necessary to complete at least embedded development on the device side, alongwith business servers, IoT cloud platforms, and smartphone apps. Facing such a huge project, public cloud normally provides software development kits for device-side and smartphone apps to speed up the process. Both public and private cloud provide services including device access, device management, device shadow, and operation and maintenance.
Device access IoT cloud platforms need to provide not only interfaces for device access using protocols
Chapter 2. Introduction and Practice of IoT Projects 11

such as MQTT, CoAP, HTTPS, and WebSocket, but also the function of device security authentication to block forged and illegal devices, effectively reducing the risk of being compromised. Such authentication usually supports different mechanisms, so when devices are mass-produced, it is necessary to pre-assign the device certificate according to the selected authentication mechanism and burn it into the devices.
Device management The device management function provided by IoT cloud platforms can not only help manufacturers monitor the activation status and online status of their devices in real time, but also allows options such as adding / removing devices, retrieving, adding / deleting groups, firmware upgrade, and version management.
Device shadow IoT cloud platforms can create a persistent virtual version (device shadow) for each device, and the status of the device shadow can be synchronised and obtained by smartphone app or other devices through Internet transmission protocols. Device shadow stores the latest reported status and expected status of each device, and even if the device is offline, it can still obtain the status by calling APIs. Device shadow provides always-on APIs, which makes it easier to build smartphone apps that interact with devices.
Operation and maintenance The O&M function includes three aspects: · Demonstrating statistical information about IoT devices and notifications. · Log management allows information retrieval about device behavior, up / down message flow, and message content. · Device debugging supports command delivery, configuration update, and checking the interaction between IoT cloud platforms and device messages.
2.2 Practice: Smart Light Project
After the theoretical introduction in each chapter, you will find a practice section related to the Smart Light project to help you get hands-on experience. The project is based on Espressif’s ESP32-C3 chip and ESP RainMaker IoT Cloud Platform, and covers wireless module hardware in smart light products, embedded software for smart devices based on ESP32C3, smartphone apps, and ESP RainMaker interaction.
Source code For better learning and developing experience, the project in this book has been opensourced. You can download the source code from our GitHub repository at https://github. com/espressif/book-esp32c3-iot-projects.
12 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

2.2.1 Project Structure
The Smart Light project consists of three parts: i. Smart light devices based on ESP32-C3, responsible for interacting with IoT cloud platforms, and controlling the switch, brightness and color temperature of the LED lamp beads. ii. Smartphone apps (including tablet apps running on Android and iOS), responsible for network configuration of smart light products, as well as querying and controlling their status.
iii. An IoT cloud platform based on ESP RainMaker. For simplification, we consider the IoT cloud platform and business server as a whole in this book. Details about ESP RainMaker will be provided in Chapter 3.
The correspondence between the Smart Light project structure and the architecture of IoT is shown in Figure 2.1.
Figure 2.1. Structure of smart light project
2.2.2 Project Functions
Divided according to the structure, functions of each part are as follows. Smart light devices
· Network configuration and connection. · LED PWM control, such as switch, brightness, color temperature, etc. · Automation or scene control, e.g., time switch. · Encryption and secure boot of the Flash. · Firmware upgrade and version management.
Chapter 2. Introduction and Practice of IoT Projects 13

Smartphone apps · Network configuration and device binding. · Smart light product control, such as switch, brightness, color temperature, etc. · Automation or scene settings, e.g., time switch. · Local/remote control. · User registration, login, etc.
ESP RainMaker IoT cloud platform · Enabling IoT device access. · Providing device operation APIs accessible to smartphone apps. · Firmware upgrade and version management.
2.2.3 Hardware Preparation
If interested in putting the project into practice, you will also need the following hardware: smart lights, smartphones, Wi-Fi routers, and a computer that meets the installation requirements of the development environment. Smart lights
Smart lights are a new type of bulbs, whose shape is the same as the general incandescent bulb. A smart light is composed of capacitor step-down regulated power supply, wireless module (with built-in ESP32-C3), LED controller and RGB LED matrix. When connected to power, the 15 V DC voltage output after capacitor step-down, diode rectification, and regulation provides energy to the LED controller and LED matrix. The LED controller can automatically send high and low levels at certain intervals, switching the RGB LED matrix between closed (lights on) and open (lights off), so that it can emit cyan, yellow, green, purple, blue, red, and white light. The wireless module is responsible for connecting to the Wi-Fi router, receiving and reporting the status of smart lights, and sending commands to control the LED.
Figure 2.2. A simulated smart light
In the early development stage, you can simulate a smart light using the ESP32-C3DevKitM-1 board connected with RGB LED lamp beads (see Figure 2.2). But you should
14 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

note that this is not the only way to assemble a smart light. The hardware design of the project in this book only contains a wireless module (with built-in ESP32-C3), but not a complete smart light hardware design. In addition, Espressif also produces a ESP32-C3-based audio development board ­ ESP32C3-Lyra ­ for controlling lights with audio. The board has interfaces for microphones and speakers and can control LED strips. It can be used for developing ultra-low-cost, highperformance audio broadcasters and rhythm light strips. Figure 2.3 shows a ESP32-C3Lyra board linked with a strip of 40 LED lights.
Figure 2.3. ESP32-C3-Lyra linked with a strip of 40 LED lights
Smartphones (Android/iOS) The Smart Light project involves the development of a smartphone app for setting up and controlling smart light products.
Wi-Fi routers Wi-Fi routers convert wired network signals and mobile network signals into wireless network signals, for computers, smartphones, tablets, and other wireless devices to connect to the network. For example, broadband in the home only needs to be connected to a Wi-Fi router to achieve wireless networking of Wi-Fi devices. The mainstream protocol standard supported by Wi- Fi routers is IEEE 802.11n, with an average TxRate of 300 Mbps, or 600 Mbps at maximum. They are backward compatible with IEEE 802.11b and IEEE 802.11g. The ESP32-C3 chip by Espressif supports IEEE 802.11b/g/n, so you can choose a single-band (2.4 GHz) or dual-band (2.4 GHz and 5 GHz) Wi-Fi router.
A computer (Linux/macOS/Windows) Development environment will be introduced in Chapter 4. Chapter 2. Introduction and Practice of IoT Projects 15

2.2.4 Development Process
Figure 2.4. Steps of developing the Smart Light project
Hardware design Hardware design of IoT devices is essential to an IoT project. A complete smart light project is intended to produce a lamp working under mains supply. Different manufacturers produce lamps of different styles and driver types, but their wireless modules are usually of the same function. To simplify the development process of the Smart Ligh project, this book only covers the hardware design and software development of wireless modules.
IoT cloud platform configuration To use IoT cloud platforms, you need to configure projects on the backend, such as creating products, creating devices, setting device properties, etc.
Embedded software development for IoT devices Implement expected functions with ESP-IDF, Espressif’s device-side SDK, including connecting to IoT cloud platforms, developing LED drivers, and upgrading firmware.
Smartphone app development Develop smartphone apps for Android and iOS systems to realise user registration and login, device control and other functions.
IoT device optimisation Once the basic development of IoT device functions is completed, you may turn to optimisation tasks, such as power optimisation.
Mass production testing Carry out mass production tests according to related standards, such as equipment function test, aging test, RF test, etc.
Despite the steps listed above, a Smart Light project is not necessarily subject to such procedure as different tasks can also be carried out at the same time. For example, embedded software and smartphone apps can be developed in parallel. Some steps may also need to be repeated, such as IoT device optimisation and mass production testing.
16 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

2.3 Summary
In this chapter, we first expounded on the basic components and functional modules of an IoT project, then introduced the Smart Light case for practice, refering to its structure, functions, hardware preparation, and development process. Readers can draw inferences from the practice and become confident to carry out IoT projects with minimum mistakes in the future.
Chapter 2. Introduction and Practice of IoT Projects 17

18 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Chapter 3

Introduction

to

ESP

RainMaker

The Internet of Things (IoT) offers endless possibilities to change the way people live, yet the development of IoT engineering is full of challenges. With public clouds, terminal manufacturers can implement product functionality through the following solutions:
Based on solution providers’ cloud platforms In this way, terminal manufacturers only need to design the product hardware, then connect the hardware to the cloud using provided communication module, and configure the product functions following the guidelines. This is an efficient approach since it eliminates the need for server-side and application-side development and operations and maintenance (O&M). It allows terminal manufacturers to focus on hardware design without having to consider cloud implementation. However, such solutions (e.g., device firmware and App) are generally not open source, so the product functions will be limited by provider’s cloud platform which cannot be customized. Meanwhile, the user and device data also belong to the cloud platform.
Based on cloud products In this solution, after completing the hardware design, terminal manufacturers not only need to implement cloud functions using one or more cloud products provided by the public cloud, but also need to link the hardware with the cloud. For example, to connect to Amazon Web Services (AWS), terminal manufacturers need to use AWS products such as Amazon API Gateway, AWS IoT Core, and AWS Lambda to enable device access, remote control, data storage, user management, and other basic functions. It not only asks terminal manufacturers to flexibly use and configure cloud products with in-depth understanding and rich experience, but also requires them to consider the construction and maintenance cost for initial and later stages This poses great challenges to the company’s energy and resources.
Compared with public clouds, private clouds are usually built for specific projects and products. Private cloud developers are given highest level of freedom in protocol design and business logic implementation. Terminal manufacturers can make products and design schemes at will, and easily integrate and empower user data. Combining the high security, scalability and reliability of public cloud with the advantages of private cloud, Espressif launched ESP
19

RainMaker, a deeply integrated private cloud solution based on Amazon cloud. Users can deploy ESP RainMaker and build private cloud simply with an AWS account.
3.1 What is ESP RainMaker?
ESP RainMaker is a complete AIoT platform built with multiple mature AWS products. It provides various services required for mass production such as device cloud access, device upgrade, backend management, third-party login, voice integration, and user management. By using the Serverless Application Repository (SAR) provided by AWS, terminal manufacturers can quickly deploy ESP RainMaker to their AWS accounts, which is time-efficient and easy to operate. Managed and maintained by Espressif, the SAR used by ESP RainMaker helps developers reduce cloud maintenance costs and accelerate the development of AIoT products, thus building secure, stable, and customizable AIoT solutions. Figure 3.1 shows the architecture of ESP RainMaker.
Figure 3.1. Architecture of ESP RainMaker
The ESP RainMaker public server by Espressif is free for all ESP enthusiasts, makers, and educators for solution evaluation. Developers can log in with Apple, Google, or GitHub accounts, and quickly build their own IoT application prototypes. The public server integrates Alexa and Google Home, and provides voice control services, which are supported by Alexa Skill and Google Actions. Its semantic recognition function is also powered by third parties. RainMaker IoT devices only respond to specific actions. For an exhaustive list of supported voice commands, please check the third-party platforms. In addition, Espressif offers a public RainMaker App for users to control the products through smartphones. 20 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

3.2 The Implementation of ESP RainMaker
As shown in Figure 3.2, ESP RainMaker consists of four parts: · Claiming Service, enabling RainMaker devices to dynamically obtain certificates. · RainMaker Cloud (also known as cloud backend), providing services such as message filtering, user management, data storage, and third-party integrations. · RainMaker Agent, enabling RainMaker devices to connect to RainMaker Cloud. · RainMaker Client (RainMaker App or CLI scripts), for provisioning, user creation, device association and control, etc.
Figure 3.2. Structure of ESP RainMaker
ESP RainMaker provides a complete set of tools for product development and mass production, including: RainMaker SDK
RainMaker SDK is based on ESP-IDF and provides the source code of the device- side agent and related C APIs for firmware development. Developers only need to write the application logic and leave the rest to the RainMaker framework. For more information about C APIs, please visit https://bookc3.espressif.com/rm/c-api-reference. RainMaker App The public version of RainMaker App allows developers to complete device provisioning, and control and query the status of devices (e.g., smart lighting products). It is available on both iOS and Android app stores. For more details, please refer to Chapter 10. REST APIs REST APIs help users build their own applications similar to the RainMaker App. For more information, please visit https://swaggerapis.rainmaker.espressif.com/.
Chapter 3. Introduction to ESP RainMaker 21

Python APIs A Python-based CLI, which comes with the RainMaker SDK, is provided to implement all functions similar to smartphone features. For more information about Python APIs, please visit https://bookc3.espressif.com/rm /python-api-reference.
Admin CLI Admin CLI, with higher level of access, is provided for ESP RainMaker private deployment to generate device certificates in bulk.
3.2.1 Claiming Service
All communication between RainMaker devices and the cloud backend is carried out through MQTT+TLS. In the context of ESP RainMaker, “Claiming” is the process in which devices obtain certificates from the Claiming Service to connect to the cloud backend. Note that Claiming Service is only applicable to the public RainMaker service, while for private deployment, the device certificates need to be generated in bulk through Admin CLI. ESP RainMaker supports three types of Claiming Service: Self Claiming
The device itself fetches the certificates through a secret key pre-programmed in eFuse after connecting to the Internet. Host Driven Claiming The certificates are obtained from the development host with the RainMaker account. Assisted Claiming The certificates are obtained via smartphone applications during provisioning.
3.2.2 RainMaker Agent
Figure 3.3. Structure of RainMaker SDK The primary function of the RainMaker Agent is to provide connectivity and assist the application layer to process uplink/downlink cloud data. It is built through the RainMaker SDK 22 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

and developed based on the proven ESP-IDF framework, using ESP-IDF components such as RTOS, NVS, and MQTT. Figure 3.3 shows the structure of the RainMaker SDK.
The RainMaker SDK includes two major features.
Connection
i. Cooperating with Claiming Service to obtain device certificates.
ii. Connecting to the cloud backend using the secure MQTT protocol to provide remote connectivity and implement remote control, message reporting, user management, device management, etc. It uses the MQTT component in ESP-IDF by default and provides an abstraction layer to interface with other protocol stacks.
iii. Providing wifi provisioning component for Wi-Fi connection and provisioning, esp https ota component for OTA upgrades, and esp local ctrl component for local device discovery and connection. All these objectives can be achieved through simple configuration.
Data processing
i. Storing the device certificates issued by Claiming Service and the data needed when running RainMaker, by default using the interface provided by the nvs flash component, and providing APIs for developers for direct use.
ii. Using the callback mechanism to process uplink/downlink cloud data and automatically unblocking the data to the application layer for easy processing by developers. For example, the RainMaker SDK provides rich interfaces for establishing TSL (Thing Specification Language) data, which are required to define TSL models to describe IoT devices and implement functions such as timing, countdown, and voice control. For basic interactive features such as timing, RainMaker SDK provides a development-free solution which can be simply enabled when needed. Then, the RainMaker Agent will directly process the data, send it to the cloud through the associated MQTT topic, and feed back the data changes in the cloud backend through callback mechanism.
3.2.3 Cloud Backend
The cloud backend is built on AWS Serverless Computing and achieved through AWS Cognito (identity management system), Amazon API Gateway, AWS Lambda (serverless computing service), Amazon DynamoDB (NoSQL database), AWS IoT Core (IoT access core that provides MQTT access and rule filtering), Amazon Simple Email Service (SES simple mail service), Amazon CloudFront (fast delivery network), Amazon Simple Queue Service (SQS message queuing), and Amazon S3 (bucket storage service). It is aimed to optimize scalability and security. With ESP RainMaker, developers can manage devices without having to write code in the cloud. Messages reported by devices are transparently transmitted to
Chapter 3. Introduction to ESP RainMaker 23

application clients or other third-party services. Table 3.1 shows the AWS cloud products and functions used in the cloud backend, with more products and features under development.
Table 3.1. AWS cloud products and functions used by the cloud backend

AWS Cloud Product Used by RainMaker

Function

AWS Cognito

Managing user credentials and supporting third-party logins

AWS Lambda

Implementing the core business logic of the cloud backend

Amazon Timestream Storing time series data

Amazon DynamoDB Storing customers’ private information

AWS IoT Core

Supporting MQTT communication

Amazon SES

Providing email sending services

Amazon CloudFront Accelerating the management of backend website access

Amazon SQS

Forwarding messages from AWS IoT Core

3.2.4 RainMaker Client
RainMaker clients, such as App and CLI, communicate with the cloud backend through REST APIs. Detailed information and instructions about REST APIs can be found in the Swagger documentation provided by Espressif. RainMaker’s mobile application client is available for both iOS and Android systems. It allows device provisioning, control, and sharing, as well as creating and enabling countdown tasks and connecting to third-party platforms. It can automatically load UI and icons according to the configuration reported by the devices and fully display the device TSL.
For example, if a smart light is built on the RainMaker SDK-provided examples, the icon and UI of the bulb light will be loaded automatically when the provisioning is completed. Users can change the color and brightness of the light through the interface and achieve third-party control by linking Alexa Smart Home Skill or Google Smart Home Actions to their ESP RainMaker accounts. Figure 3.4 shows the icon and UI examples of the bulb light respectively on Alexa, Google Home, and ESP RainMaker App.

24 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

(a) Example – Alexa

(b) Example – Google Home

(c) Example – ESP RainMaker
Figure 3.4. Examples of icon and UI of the bulb light on Alexa, Google Home, and ESP RainMaker App
3.3 Practice: Key Points for Developing with ESP RainMaker
Once the device driver layer has been completed, developers may start to create TSL models and process downlink data using the APIs provided by RainMaker SDK, and enable the ESP RainMaker basic services based on the product definition and requirements.
Chapter 3. Introduction to ESP RainMaker 25

Section 9.4 of this book will explain the implementation of the LED smart light in RainMaker. During debugging, developers can use the CLI tools in the RainMaker SDK to communicate with the smart light (or call REST APIs from Swagger).
Chapter 10 will elaborate the usage of REST APIs in developing smartphone applications. The OTA upgrades of LED smart lights will be covered in Chapter 11. If developers have enabled the ESP Insights remote monitoring, the ESP RainMaker management backend will display the ESP Insights data. Details will be presented in Chapter 15.
ESP RainMaker supports private deployment, which differs from the public RainMaker server in the following ways:
Claiming Service To generate certificates in private deployments, it is required to use the RainMaker Admin CLI instead of Claiming. With public server, developers must be given admin rights to implement firmware upgrade, but it is undesirable in commercial deployments. Therefore, neither separate authentication service can be provided for self-claiming, nor admin rights for host driven or assisted claiming.
Phone apps In private deployments, applications need to be configured and compiled separately to ensure that the account systems are not interoperable.
3rd party logins and voice integration Developers have to configure separately via Google and Apple Developer accounts to enable 3rd party logins, as well as the Alexa Skill and Google Voice Assistant integration.
TIPS For details about cloud deployment, please visit https://customer.rainmaker.espressif. com. In terms of firmware, migration from public server to private server only requires replacing device certificates, which greatly improves migration efficiency and reduces the cost of migration and secondary debugging.
3.4 Features of ESP RainMaker
ESP RainMaker features are mainly targeted at three aspect – user management, end users, and admins. All features are supported in both public and private servers unless otherwise stated.
3.4.1 User Management
The user management features allow end users to register, log in, change passwords, retrieve passwords, etc.
26 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Register and log in The registration and login methods supported by RainMaker include: · Email id + Password · Phone number + Password · Google account · Apple account · GitHub account (public server only) · Amazon account (private server only)
NOTE Sign up using Google/Amazon shares the user’s email address with RainMaker. Sign up using Apple shares a dummy address that Apple assigns for the user specifically for the RainMaker service. A RainMaker account will be automatically created for users signing in with a Google, Apple, or Amazon account for the first time.
Change password Valid only for Email id/Phone number based logins. All other active sessions will be logged out after password is changed. As per AWS Cognito behaviour, the logged-out sessions can stay active upto 1 hour.
Retrieve password Valid only for Email id/Phone number based logins.
3.4.2 End User Features
Features open to end users include local and remote control and monitoring, scheduling, device grouping, device sharing, push notifications, and third- party integrations.
Remote control and monitoring · Query configuration, parameter values, and connection status for one or all devices. · Set parameters for single or multiple devices.
Local control and monitoring Mobile phone and the device need to be connected to the same network for local control.
Scheduling · Users pre-set certain actions at a specific time. · No Internet connection required for the device while executing the schedule. · One time or repeat (by specifying days) for single or multiple devices.
Device grouping Supports multi-level abstract grouping Group metadata can be used to create a Home Room structure.
Chapter 3. Introduction to ESP RainMaker 27

Device sharing One or more devices can be shared with one or more users.
Push notifications End users will receive push notifications for events such as · New device(s) added/removed · Device connected to cloud · Device disconnected from cloud · Device sharing requests created/accepted/declined · Alert messages reported by devices
Third party integrations Alexa and Google Voice Assistant are supported to control RainMaker devices, including lights, switches, sockets, fans, and temperature sensors.
3.4.3 Admin Features
Admin features allow administrators to implement device registration, device grouping, and OTA upgrades, and to view statistics and ESP Insights data.
Device registration Generate device certificates and register with Admin CLI (private server only).
Device grouping Create abstract or structured groups based on device information (private server only).
Over-the-Air (OTA) upgrades Upload firmware based on version and model, to one or more devices or a group Monitor, cancel, or archive OTA jobs.
View statistics Viewable statistics include: · Device registrations (certificates registered by the admin) · Device activations (device connected for the first time) · User accounts · User-device association
View ESP Insights data Viewable ESP Insights data include: · Errors, warnings, and custom logs · Crash reports and analysis · Reboot reasons · Metrics like memory usage, RSSI, etc. · Custom metrics and variables
28 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

3.5 Summary
In this chapter, we introduced some key differences between the public RainMaker deployment and the private deployment. The private ESP RainMaker solution launched by Espressif is highly reliable and extensible. All ESP32 series chips have been connected and adapted to AWS, which greatly reduces the cost. Developers can focus on prototype verification without having to learn about AWS cloud products. We also explained the implementation and features of ESP RainMaker, and some key points for development using the platform.
Scan to download ESP RainMaker for Android Scan to download ESP RainMaker for iOS
Chapter 3. Introduction to ESP RainMaker 29

30 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Chapter Setting Up 4 Development Environment
This chapter focuses on ESP-IDF, the official software development framework for ESP32-C3. We’ll explain how to set up the environment on various operating systems, and introduce the project structure and build system of ESP-IDF, as well as the usage of related development tools. Then we’ll present the compiling and running process of an example project, while offering a detailed explanation of the output log at each stage.
4.1 ESP-IDF Overview
ESP-IDF (Espressif IoT Development Framework) is a one-stop IoT development framework provided by Espressif Technology. It uses C/C++ as the main development language and supports cross-compilation under mainstream operating systems such as Linux, Mac, and Windows. The example programs included in this book are developed using ESP-IDF, which offers the following features: · SoC system-level drivers. ESP-IDF includes drivers for ESP32, ESP32-S2, ESP32-C3,
and other chips. These drivers encompass peripheral low level (LL) library, hardware abstraction layer (HAL) library, RTOS support and upper-layer driver software, etc. · Essential components. ESP-IDF incorporates fundamental components required for IoT development. This includes multiple network protocol stacks such as HTTP and MQTT, a power management framework with dynamic frequency modulation, and features like Flash Encryption and Secure Boot, etc. · Development and production tools. ESP-IDF provides commonly used tools for building, flash, and debugging during development and mass production (see Figure 4.1), such as the building system based on CMake, the cross-compilation tool chain based on GCC, and the JTAG debugging tool based on OpenOCD, etc. It is worth noting that the ESP-IDF code primarily adheres to the the Apache 2.0 open-source license. Users can develop personal or commercial software without restrictions while complying with the terms of the open-source license. Additionally, users are granted permanent patent licenses free of charge, without the obligation to open-source any modifications made to the source code.
31

Figure 4.1.

Building, flashing, and debug-

ging tools for development and mass production

4.1.1 ESP-IDF Versions
The ESP-IDF code is hosted on GitHub as an open-source project. Currently, there are three major versions available: v3, v4, and v5. Each major version usually contains various subversions, such as v4.2, v4.3, and so on. Espressif Systems ensures a 30-month support for bug fixes and security patches for each released sub-version. Therefore, revisions of subversions are also released regularly, such as v4.3.1, v4.2.2, etc. Table 4.1 shows the support status of different ESP-IDF versions for Espressif chips, indicating whether they are in a preview stage (offering support for preview versions, which may lack certain features or documentation) or are officially supported.

Table 4.1. Support status of different ESP-IDF versions for Espressif chips

Series ESP32 ESP32-S2 ESP32-C3 ESP32-S3 ESP32-C2 ESP32-H2

v4.1 supported

v4.2 supported supported

v4.3 supported supported supported

v4.4 supported supported supported supported
preview

v5.0 supported supported supported supported supported preview

32 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

The iteration of major versions often involves adjustments to the framework structure and updates to the compilation system. For example, the major change from v3. to v4. was the gradual migration of the build system from Make to CMake. On the other hand, iteration of minor versions typically entails the addition of new features or support for new chips.
It is important to distinguish and understand the relationship between stable versions and GitHub branches. Versions labeled as v. or v..* represent stable versions that have passed complete internal testing by Espressif. Once fixed, the code, tool chain, and release documents for the same version remain unchanged. However, GitHub branches (e.g., the release/v4.3 branch) undergo frequent code commits, often on a daily basis. Therefore, two code snippets under the same branch may differ, necessitating developers to promptly update their code accordingly.
4.1.2 ESP-IDF Git Workflow
Espressif follows a specific Git workflow for ESP-IDF, outlined as follows:
· New changes are made on the master branch, which serves as the main development branch. The ESP-IDF version on the master branch always carries a -dev tag to indicate that it is currently under development, such as v4.3-dev. Changes on the master branch will first be reviewed and tested in Espressif’s internal repository, and then pushed to GitHub after automated testing is complete.
· Once a new version has completed feature development on the master branch and met the criteria for entering beta testing, it transitions to a new branch, such as release/ v4.3. In addition, this new branch is tagged as a pre-release version, like v4.3-beta1. Developers can refer to the GitHub platform to access the complete list of branches and tags for ESP-IDF. It’s important to note that the beta version (pre-release version) may still have a significant number of known issues. As the beta version undergoes continuous testing, bug fixes are added to both this version and the master branch simultaneously. Meanwhile, the master branch may have already begun developing new features for the next version. When testing is nearly complete, a release candidate (rc) label is added to the branch, indicating that it is a potential candidate for the official release, such as v4.3-rc1. At this stage, the branch remains a pre-release version.
· If no major bugs are discovered or reported, the pre-release version eventually receives a major version label (e.g., v5.0) or a minor version label (e.g., v4.3) and becomes an official release version, which is documented in the release notes page. Subsequently, any bugs identified in this version are fixed on the release branch. After manual testing is completed, the branch is assigned a bug-fix version label (e.g., v4.3.2), which is also reflected on the release notes page.
Chapter 4. Setting Up Development Environment 33

4.1.3 Choosing a Suitable Version
Since ESP-IDF officially began supporting ESP32-C3 from version v4.3, and v4.4 has not yet been officially released at the time of writing this book, the version used in this book is v4.3.2, which is a revised version of v4.3. However, it is important to note that by the time you read this book, v4.4 or newer versions may already be available. When selecting a version, we recommend the following:
· For entry-level developers, it is advisable to choose the stable v4.3 version or its revised version, which aligns with the example version used in this book.
· For mass production purposes, it is recommended to use the latest stable version to to benefit from the most up-to-date technical support.
· If you intend to experiment with new chips or explore new product features, please use the master branch. The latest version contains all the latest features, but keep in mind that there may be known or unknown bugs present.
· If the stable version being used does not include the desired new features and you wish to minimise the risks associated with the master branch, consider using the corresponding release branch, such as the release/v4.4 branch. Espressif’s GitHub repository will first create the release/v4.4 branch and subsequently release the stable v4.4 version based on a specific historical snapshot of this branch, after completing all feature development and testing.
4.1.4 Overview of ESP-IDF SDK Directory
The ESP-IDF SDK consists of two main directories: esp-idf and .espressif. The former contains ESP-IDF repository’s source code files and compilation scripts, while the latter mainly stores compilation tool chains and other software. Familiarity with these two directories will help developers make better use of available resources and speed up the development process. The directory structure of ESP-IDF is described below:
(1) ESP-IDF repository code directory (/esp/esp-idf), as shown in Figure 4.2.
a. Component directory components
This core directory integrates numerous essential software components of ESP- IDF. No project code can be compiled without relying on the components within this directory. It includes driver support for various Espressif chips. From the LL library and HAL library interfaces for peripherals to the upper-level Driver and Virtual File System (VFS) layer support, developers can choose the appropriate components at different levels for their development needs. ESP- IDF also supports multiple standard network protocol stacks such as TCP/IP, HTTP, MQTT, WebSocket, etc. Developers can utilise familiar interfaces like Socket to build network applications. Components provide comprehen-
34 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Figure 4.2. ESP-IDF repository code directory
sive functionality and can be easily integrated into applications, allowing developers to focus solely on the business logic. Some common components include: · driver: This component contains peripheral driver programs for various Espressif
chip series, such as GPIO, I2C, SPI, UART, LEDC (PWM), etc. The peripheral driver programs in this component offer chip-independent abstract interfaces. Each peripheral has a common header file (such as gpio.h), eliminating the need to deal with different chip-specific support questions. · esp_wifi: Wi- Fi, as a special peripheral, is treated as a separate component. It includes multiple APIs such as initialisation of various Wi-Fi driver modes, parameter configuration, and event processing. Certain functions of this component are provided in the form of static link libraries. ESP-IDF also provides comprehensive driver documentation for ease of use.
Chapter 4. Setting Up Development Environment 35

· freertos: This component contains the complete FreeRTOS code. Apart from providing comprehensive support for this operating system, Espressif has also extended its support to dual-core chips. For dual-core chips like ESP32 and ESP32-S3, users can create tasks on specific cores.
b. Document directory docs
This directory contains ESP-IDF related development documents, including the Get Started Guide, API Reference Manual, Development Guide, etc.
NOTE After being compiled by automated tools, the contents of this directory are deployed at https://docs.espressif.com/projects/esp-idf. Please ensure to switch the document target to ESP32-C3 and select the specified ESP-IDF version.
c. Script tool tools
This directory contains commonly used compilation front-end tools such as idf.py, and the monitor terminal tool idf_monitor.py, etc. The sub-directory cmake also contains core script files of the compilation system, serving as the foundation for implementing ESP-IDF compilation rules. When adding the environment variables, the contents within the tools directory are added to the system environment variable, allowing idf.py to be executed directly under the project path.
d. Example program directory examples
This directory comprises a vast collection of ESP-IDF example programs that demonstrate the usage of component APIs. The examples are organised into various subdirectories based on their categories:
· get-started: This sub-directory includes entry-level examples like “hello world” and “blink” to help users grasp the basics.
· bluetooth: You can find Bluetooth related examples here, including Bluetooth LE Mesh, Bluetooth LE HID, BluFi, and more.
· wifi: This sub-directory focuses on Wi-Fi examples, including basic programs like Wi-Fi SoftAP, Wi-Fi Station, espnow, as well as proprietary communication protocol examples from Espressif. It also includes multiple application layer examples based on Wi-Fi, such as Iperf, Sniffer, and Smart Config.
· peripherals: This extensive sub-directory is further divided into numerous subfolders based on peripheral names. It mainly contains peripheral driver examples for Espressif chips, with each example featuring several sub- examples. For instance, the gpio sub-directory includes two examples: GPIO and GPIO matrix keyboard. It’s important to note that not all examples in this directory are applicable to ESP32-C3.
36 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

For example, the examples in usb/host are only applicable to peripherals with USB Host hardware (such as ESP32-S3), and ESP32-C3 does not have this peripheral. The compilation system typically provides prompts when setting the target. The README file of each example lists the supported chips. · protocols: This sub-directory contains examples for various communication protocols, including MQTT, HTTP, HTTP Server, PPPoS, Modbus, mDNS, SNTP, covering a wide range of communication protocol examples required for IoT development. · provisioning: Here, you’ll find provisioning examples for different methods, such as Wi-Fi provisioning and Bluetooth LE provisioning. · system: This sub-directory includes system debugging examples (e.g., stack tracing, runtime tracing, task monitoring), power management examples (e.g., various sleep modes, co-processors), and examples related to common system components like console terminal, event loop, and system timer. · storage: Within this sub-directory, you’ll discover examples of all file systems and storage mechanisms supported by ESP-IDF (such as reading and writing of Flash, SD card and other storage media), as well as examples of non-volatile storage (NVS), FatFS, SPIFFS and other file system operations. · security: This sub- directory contains examples related to flash encryption. (2) ESP-IDF compilation tool chain directory (/.espressif), as shown in Figure 4.3.
Figure 4.3. ESP-IDF compilation tool chain directory
Chapter 4. Setting Up Development Environment 37

a. Software distribution directory dist
The ESP-IDF tool chain and other software are distributed in the form of compressed packages. During the installation process, the installation tool first downloads the compressed package to the dist directory, and then extracts it to the specified directory. Once the installation is complete, the contents in this directory can be safely removed.
b. Python virtual environment directory python env
Different versions of ESP-IDF rely on specific versions of Python packages. Installing these packages directly on the same host can lead to conflicts between package versions. To address this, ESP-IDF utilises Python virtual environments to isolate different package versions. With this mechanism, developers can install multiple versions of ESP-IDF on the same host and easily switch between them by importing different environment variables.
c. ESP-IDF compilation tool chain directory tools
This directory mainly contains cross-compilation tools required to compile ESP-IDF projects, such as CMake tools, Ninja build tools, and the gcc tool chain that generates the final executable program. Additionally, this directory houses the standard library of the C/C++ language along with the corresponding header files. If a program references a system header file like

include <stdio.h>, the compilation tool chain will locate the stdio.h file

within this directory.
4.2 Setting Up ESP-IDF Development Environment
The ESP-IDF development environment supports mainstream operating systems such as Windows, Linux, and macOS. This section will introduce how to set up the development environment on each system. It is recommended to develop ESP32-C3 on Linux system, which will be introduced in detail here. Many instructions are applicable across platforms due to the similarity of the development tools. Therefore, it is advised to carefully read the content of this section.
NOTE You can refer to the online documents available at https://bookc3.espressif.com/esp32c3, which provide the commands mentioned in this section.
4.2.1 Setting up ESP-IDF Development Environment on Linux
The GNU development and debugging tools required for the ESP-IDF development environment are native to the Linux system. Additionally, the command-line terminal in Linux is powerful and user-friendly, making it an ideal choice for ESP32-C3 development. You can
38 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

select your preferred Linux distribution, but we recommend using Ubuntu or other Debianbased systems. This section provides guidance on setting up the ESP-IDF development environment on Ubuntu 20.04.
1. Install required packages
Open a new terminal and execute the following command to install all necessary packages. The command will automatically skip packages that are already installed.
$ sudo apt-get install git wget flex bison gperf python3 python3-pip python3setuptools cmake ninja-build ccache libffi-dev libssl-dev dfu-util libusb-1.0-0
TIPS You need to use the administrator account and password for the command above. By default, no information will be displayed when entering the password. Simply press the “Enter” key to continue the procedure.
Git is a key code management tool in ESP-IDF. After successfully setting up the development environment, you can use the git log command to view all code changes made since the creation of ESP-IDF. In addition, Git is also used in ESP-IDF to confirm version information, which is necessary for installing the correct tool chain corresponding to specific versions. Along with Git, other important system tools include Python. ESP-IDF incorporates numerous automation scripts written in Python. Tools such as CMake, Ninja-build, and Ccache are widely used in C/C++ projects and serve as the default code compilation and building tools in ESP-IDF. libusb-1.0-0 and dfu-util are the main drivers used for USB serial communication and firmware burning. Once the software packages are installed, you can use the apt show command to obtain detailed descriptions of each package. For example, use apt show git to print the description information for the Git tool.
Q: What to do if the Python version is not supported? A: ESP-IDF v4.3 requires a Python version that is not lower than v3.6. For older versions of Ubuntu, please manually download and install a higher version of Python and set Python3 as the default Python environment. You can find detailed instructions by searching for the keyword update-alternatives python.
2. Download ESP-IDF repository code
Open a terminal and create a folder named esp in your home directory using the mkdir command. You can choose a different name for the folder if you prefer. Use the cd command to enter the folder.
Chapter 4. Setting Up Development Environment 39

$ mkdir -p /esp $ cd /esp
Use the git clone command to download the ESP-IDF repository code, as shown below:
$ git clone -b v4.3.2 –recursive https://github.com/espressif/esp-idf.git
In the command above, the parameter -b v4.3.2 specifies the version to download (in this case, version 4.3.2). The parameter –recursive ensures that all sub-repositories of ESP-IDF are downloaded recursively. Information about sub-repositories can be found in the .gitmodules file.
3. Install the ESP-IDF development tool chain
Espressif provides an automated script install.sh to download and install the tool chain. This script checks the current ESP-IDF version and operating system environment, and then downloads and installs appropriate version of Python tool packages and compilation tool chains. The default installation path for the tool chain is /.espressif. All you need to do is to navigate to the esp-idf directory and run install.sh.
$ cd /esp/esp-idf $ ./install.sh
If you install the the tool chain successfully, the terminal will display:
All done!
At this point, you have successfully set up the ESP-IDF development environment.
4.2.2 Setting up ESP-IDF Development Environment on Windows
1. Download ESP-IDF tools installer
TIPS It is recommended to set up the ESP-IDF development environment on Windows 10 or above. You can download the installer from https://dl.espressif.com/dl/esp-idf/. The installer is also an open-source software, and its source code can be viewed at https: //github.com/espressif /idf-installer.
· Online ESP-IDF tools installer
This installer is relatively small, around 4 MB in size, and other packages and code will be downloaded during the installation process. The advantage of the online installer is that not only can software packages and code be downloaded on demand during the installation process, but also allows the installation of all available releases of ESP-IDF and the latest branch of GitHub code (such as the master branch). The disadvantage is that it requires a network connection during the installation process, which may cause installation failure due to network problems.
40 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

· Offline ESP-IDF tools installer This installer is larger, about 1 GB in size, and contains all the software packages and code required for environment set up. The main advantage of the offline installer is that it can be used on computers without Internet access, and generally has a higher installation success rate. It should be noted that the offline installer can only install stable releases of ESP-IDF identified by v. or v..*.
2. Run the ESP-IDF tools installer After downloading a suitable version of the installer (take ESP-IDF Tools Offline 4.3.2 for example here), double- click the exe file to launch the ESP-IDF installation interface. The following demonstrates how to install ESP-IDF stable version v4.3.2 using the offline installer.
(1) In the “Select installation language” interface shown in Figure 4.4, select the language to be used from the drop-down list.
Figure 4.4. “Select installation language” interface (2) After selecting the language, click “OK” to pop up the “License agreement” interface
(see Figure 4.5). After carefully reading the installation license agreement, select “I accept the agreement” and click “Next”.
Figure 4.5. “License agreement” interface Chapter 4. Setting Up Development Environment 41

(3) Review the system configuration in the “Pre-installation system check” interface (see Figure 4.6). Check the Windows version and the installed antivirus software information. Click “Next” if all the configuration items are normal. Otherwise, you can click “Full log” for solutions based on key items.
Figure 4.6. “System check before installation” interface TIPS
You can submit logs to https://github.com/espressif/idf-installer/issues for help. (4) Select the ESP-IDF installation directory. Here, select D:/.espressif, as shown in
Figure 4.7, and click “Next”. Please note that .espressif here is a hidden directory. After the installation is completed, you can view the specific contents of this directory by opening the file manager and displaying hidden items.
Figure 4.7. Select the ESP-IDF installation directory 42 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

(5) Check the components that need to be installed, as shown in Figure 4.8. It is recommended to use the default option, that is, complete installation, and then click “Next”.
Figure 4.8. Select the components to install (6) Confirm the components to be installed and click “Install” to start the automated in-
stallation process, as shown in Figure 4.9. The installation process may last tens of minutes and the progress bar of the installation process is shown in Figure 4.10. Please wait patiently.
Figure 4.9. Preparing for installation (7) After the installation is complete, it is recommended to check “Register the ESP-IDF
Tools executables as Windows Defender exclusions…” to prevent antivirus software from deleting files. Adding exclusion items can also skip frequent scans by antivirus
Chapter 4. Setting Up Development Environment 43

Figure 4.10. Installation progress bar software, greatly improving the code compilation efficiency of the Windows system. Click “Finish” to complete the installation of the development environment, as shown in Figure 4.11. You can choose to check “Run ESP-IDF PowerShell environment” or “Run ESP-IDF command prompt”. Run the compilation window directly after installation to ensure that the development environment functions normally.
Figure 4.11. Installation completed (8) Open the installed development environment in the program list (either ESP-IDF 4.3
CMD or ESP-IDF 4.3 PowerShell terminal, as shown in Figure 4.12), and the ESP- IDF environment variable will be automatically added when running in the terminal. After that, you can use the idf.py command for operations. The opened ESP-IDF 4.3 CMD is shown in Figure 4.13. 44 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

Figure 4.12. Development environment installed
Figure 4.13. ESP-IDF 4.3 CMD
4.2.3 Setting up ESP-IDF Development Environment on Mac
The process of installing the ESP-IDF development environment on a Mac system is the same as that on a Linux system. The commands for downloading the repository code and installing the tool chain are exactly the same. Only the commands for installing dependency packages are slightly different. 1. Install dependency packages Open a terminal, and install pip, the Python package management tool, by running the following command:
% sudo easy install pip
Install Homebrew, a package management tool for macOS, by running the following command:
% /bin/bash -c “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/ HEAD/install.sh)”
Install the required dependency packages by running the following command:
% brew python3 install cmake ninja ccache dfu-util
2. Download ESP-IDF repository code Follow the instructions provided in section 4.2.1 to download the ESP-IDF repository code. The steps are the same as for downloading on a Linux system.
Chapter 4. Setting Up Development Environment 45

3. Install the ESP-IDF development tool chain
Follow the instructions provided in section 4.2.1 to install the ESP-IDF development tool chain. The steps are the same as for installation on a Linux system.
4.2.4 Installing VS Code
By default, the ESP-IDF SDK does not include a code editing tool (though the latest ESP-IDF installer for Windows offers the option to install ESP-IDF Eclipse). You can use any text editing tool of your choice to edit the code and then compile it using terminal commands.
One popular code editing tool is VS Code (Visual Studio Code), which is a free and featurerich code editor with a user-friendly interface. It offers various plugins that provide functionalities such as code navigation, syntax highlighting, Git version control, and terminal integration. Additionally, Espressif has developed a dedicated plugin called Espressif IDF for VS Code, which simplifies project configuration and debugging.
You can use the code command in the terminal to quickly open the current folder in VS Code. Alternatively, you can use the shortcut Ctrl+ to open the system’s default terminal console within VS Code.
TIPS It is recommended to use VS Code for ESP32-C3 code development. Download and install the latest version of VS Code at https://code.visualstudio.com/.
4.2.5 Introduction to Third-Party Development Environments
In addition to the official ESP-IDF development environment, which primarily uses the C language, ESP32-C3 also supports other mainstream programming languages and a wide range of third-party development environments. Some notable options include:
Arduino: an open-source platform for both hardware and software, supporting various microcontrollers, including ESP32-C3.
It uses the C++ language and offers a simplified and standardised API, commonly referred to as the Arduino language. Arduino is widely used in prototype development and educational contexts. It provides an extensible software package and an IDE that allows for easy compilation and flashing.
MicroPython: a Python 3 language interpreter designed to run on embedded microcontroller platforms.
With a simple script language, it can directly access ESP32-C3’s peripheral resources (such as UART, SPI, and I2C) and communication functions (such as Wi-Fi and Bluetooth LE).
46 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

This simplifies hardware interaction. MicroPython, combined with Python’s extensive mathematical operation library, enables the implementation of complex algorithms on ESP32-C3, facilitating the development of AI-related applications. As a script language, there is no need for repeated compilation; modifications can be made and scripts can be executed directly.
NodeMCU: an LUA language interpreter developed for ESP series chips.
It supports almost all peripheral functions of ESP chips and is lighter than MicroPython. Similar to MicroPython, NodeMCU uses a script language, eliminating the need for repeated compilation.
Furthermore, ESP32-C3 also supports the NuttX and Zephyr operating systems. NuttX is a real-time operating system that provides POSIX-compatible interfaces, enhancing application portability. Zephyr is a small real-time operating system specifically designed for IoT applications. It includes numerous software libraries required in IoT development, gradually evolving into a comprehensive software ecosystem.
This book does not provide detailed installation instructions for the aforementioned development environments. You can install a development environment based on your requirements by following the respective documentation and instructions.
4.3 ESP-IDF Compilation System
4.3.1 Basic Concepts of Compilation System
An ESP-IDF project is a collection of a main program with an entry function and multiple independent functional components. For example, a project that controls LED switches mainly consists of an entry program main and a driver component that controls GPIO. If you want to realise the LED remote control, you also need to add Wi-Fi, TCP/IP protocol stack, etc.
The compilation system can compile, link, and generate executable files (.bin) for the code through a set of building rules. The compilation system of ESP- IDF v4.0 and above versions is based on CMake by default, and the compilation script CMakeLists.txt can be used to control the compilation behavior of the code. In addition to supporting the basic syntax of CMake, the ESP-IDF compilation system also defines a set of default compilation rules and CMake functions, and you can write the compilation script with simple statements.
4.3.2 Project File Structure
A project is a folder that contains an entry program main, user-defined components, and files required to build executable applications, such as compilation scripts, configuration
Chapter 4. Setting Up Development Environment 47

files, partition tables, etc. Projects can be copied and passed on, and the same executable file can be compiled and generated in machines with the same version of ESP-IDF development environment. A typical ESP-IDF project file structure is shown in Figure 4.14.
Figure 4.14. Typical ESP-IDF project file structure Since ESP-IDF supports multiple IoT chips from Espressif, including ESP32, ESP32-S series, ESP32-C series, ESP32-H series, etc., a target needs to be determined before compiling the code. The target is both the hardware device that runs the application program and the build target of the compilation system. Depending on your needs, you can specify one or more targets for your project. For example, through command idf.py set-target esp32c3, you can set the compilation target to ESP32-C3, during which the default parameters and compilation tool chain path for ESP32C3 will be loaded. After compilation, an executable program can be generated for ESP32C3. You can also run the command set-target again to set a different target, and the compilation system will automatically clean up and reconfigure. Components
Components in ESP-IDF are modular and independent code units managed within the compilation system. They are organised as folders, with the folder name representing the component name by default. Each component has its own compilation script that 48 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

specifies its compilation parameters and dependencies. During the compilation process, components are compiled into separate static libraries (.a files) and eventually combined with other components to form the application program.
ESP-IDF provides essential functions, such as the operating system, peripheral drivers, and network protocol stack, in the form of components. These components are stored in the components directory located within the ESP-IDF root directory. Developers do not need to copy these components to the components directory of myProject. Instead, they only need to specify the dependency relationships of these components in the project’s CMakeLists.txt file using the REQUIRES or PRIV_REQUIRES directives. The compilation system will automatically locate and compile the required components.
Therefore, the components directory under myProject is not necessary. It is only used to include some custom components of the project, which can be third-party libraries or user-defined code. Additionally, components can be sourced from any directory other than ESP-IDF or the current project, such as from an open-source project saved in another directory. In this case, you only need to add the path of the component by setting the EXTRA_COMPONENT_DIRS variable in the CMakeLists.txt under the root directory. This directory will override any ESP-IDF component with the same name, ensuring the correct component is used.
Entry program main The main directory within the project follows the same file structure as other components (e.g., component1). However, it holds a special significance as it is a mandatory component that must exist in every project. The main directory contains the project’s source code and the user program’s entry point, typically named app_main. By default, the execution of the user program starts from this entry point. The main component also differs in that it automatically depends on all components within the search path. Therefore, there is no need to explicitly indicate dependencies using the REQUIRES or PRIV_REQUIRES directives in the CMakeLists.txt file.
Configuration file The root directory of the project contains a configuration file called sdkconfig, which contains the configuration parameters for all the components within the project. The sdkconfig file is automatically generated by the compilation system and can be modified and regenerated by the command idf.py menuconfig. The menuconfig options mainly originate from the Kconfig.projbuild of the project and the Kconfig of the components. Component developers generally add configuration items in Kconfig to make the component flexible and configurable.
Build directory By default, the build directory within the project stores intermediate files and the fi-
Chapter 4. Setting Up Development Environment 49

nal executable programs generated by the idf.py build command. In general, it is not necessary to directly access the contents of the build directory. ESP- IDF provides predefined commands to interact with the directory, such as using the idf.py flash command to automatically locate the compiled binary file and flash it to the specified flash address, or using the idf.py fullclean command to clean the entire build directory.
Partition table (partitions.csv) Each project requires a partition table to divide the space of flash and specify the size and starting address of the executable program and user data space. Command idf.py flash or OTA upgrade program will flash the firmware to the corresponding address according to this table. ESP-IDF provides several default partition tables in components/ partition_table, such as partitions_singleapp.csv and partitionstwo ota.csv, which can be selected in menuconfig.
If the default partition table of the system cannot meet the requirements of the project, a custom partitions.csv can be added to the project directory and be selected in menuconfig.
4.3.3 Default Build Rules of the Compilation System
Rules for overriding components with the same name During the component search process, the compilation system follows a specific order. It first searches for internal components of ESP-IDF, then looks for components of the user project, and finally searches for components in EXTRA_COMPONENT_DIRS. In cases where multiple directories contain components with the same name, the component found in the last directory will override any previous components with the same name. This rule allows for the customisation of ESP-IDF components within the user project, while keeping the original ESP-IDF code intact.
Rules for including common components by default As mentioned in section 4.3.2, components need to explicitly specify their dependencies on other components in the CMakeLists.txt. However, common components such as freertos are automatically included in the build system by default, even if their dependency relationships are not explicitly defined in the compilation script. ESP-IDF common components include freertos, Newlib, heap, log, soc, esp_rom, esp_common, xtensa/riscv, and cxx. Using these common components avoids repetitive work when writing CMakeLists.txt and make it more concise.
Rules for overriding configuration items Developers can add default configuration parameters by adding a default configuration file named sdkconfig.defaults to the project. For example, adding CONFIGLOG
50 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

DEFAULT_LEVEL_NONE = y can configure the UART interface to not print log data by default. Furthermore, if specific parameters need to be set for a particular target, a configuration file named sdkconfig.defaults.TARGET_NAME can be added, where TARGET_NAME can be esp32s2, esp32c3, and so on. These configuration files are imported into the sdkconfig during compilation, with the general default configuration file sdkconfig.defaults being imported first, followed by the target-specific configuration file, such as sdkconfig.defaults.esp32c3. In cases where there are configuration items with the same name, the latter configuration file will override the former.
4.3.4 Introduction to the Compilation Script
When developing a project using ESP-IDF, developers not only need to write source code but also need to write CMakeLists.txt for the project and components. CMakeLists.txt is a text file, also known as a compilation script, which defines a series of compilation objects, compilation configuration items, and commands to guide the compilation process of the source code. The compilation system of ESP-IDF v4.3.2 is based on CMake. In addition to supporting native CMake functions and commands, it also defines a series of custom functions, making it much easier to write compilation scripts.
The compilation scripts in ESP-IDF mainly include the project compilation script and the component compilation scripts. The CMakeLists.txt in the root directory of the project is called the project compilation script, which guides the compilation process of the entire project. A basic project compilation script typically includes the following three lines:
1. cmake_minimum_required(VERSION 3.5) 2. include($ENV{IDF_PATH}/tools/cmake/project.cmake) 3. project(myProject)
Among them, the cmake_minimum_required (VERSION 3.5) must be placed on the first line, which is used to indicate the minimum CMake version number required by the project. Newer versions of CMake are generally backward compatible with older versions, so adjust the version number accordingly when using newer CMake commands to ensure compatibility.
include($ENV {IDF_PATH}/tools/cmake/project.cmake) imports pre-defined configuration items and commands of ESP-IDF compilation system, including the default build rules of the compilation system described in Section 4.3.3. project(myProject) creates the project itself and specifies its name. This name will be used as the final output binary file name, i.e., myProject.elf and myProject.bin.
A project can have multiple components, including the main component. The top- level directory of each component contains a CMakeLists.txt file, which is called the component compilation script. Component compilation scripts are mainly used to specify component dependencies, configuration parameters, source code files, and included header files for
Chapter 4. Setting Up Development Environment 51

compilation. With ESP-IDF’s custom function idf_component_register, the minimum required code for a component compilation script is as follows:

1. idf_component_register(SRCS “src1.c”

INCLUDE_DIRS “include”

REQUIRES component1)

The SRCS parameter provides a list of source files in the component, separated by spaces if there are multiple files. The INCLUDE_DIRS parameter provides a list of public header file directories for the component, which will be added to the include search path for other components that depend on the current component. The REQUIRES parameter identifies the public component dependencies for the current component. It is necessary for components to explicitly state which components they depend on, such as component2 depending on component1. However, for the main component, which depends on all components by default, the REQUIRES parameter can be omitted.

In addition, native CMake commands can also be used in the compilation script. For example, use the command set to set variables, such as set(VARIABLE “VALUE”).

4.3.5 Introduction to Common Commands
ESP-IDF uses CMake (project configuration tool), Ninja (project building tool) and esptool (flash tool) in the process of code compilation. Each tool plays a different role in the compilation, building, and flash process, and also supports different operating commands. To facilitate user operation, ESP-IDF adds a unified front-end idf.py that allows the above commands to be called quickly.
Before using idf.py, make sure that:
· The environment variable IDF_PATH of ESP-IDF has been added to the current terminal. · The command execution directory is the root directory of the project, which includes the
project compilation script CMakeLists.txt.
The common commands of idf.py are as follows:
· idf.py –help: displaying a list of commands and their usage instructions. · idf.py set-target : setting the compilation taidf.py fullcleanrget, such
as replacing with esp32c3. · idf.py menuconfig: launching menuconfig, a terminal graphical configuration
tool, which can select or modify configuration options, and the configuration results are saved in the sdkconfig file. · idf.py build: initiating code compilation. The intermediate files and the final executable program generated by the compilation will be saved in the build directory of the project by default. The compilation process is incremental, which means that if only one source file is modified, only the modified file will be compiled next time.

52 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

· idf.py clean: cleaning the intermediate files generated by the project compilation. The entire project will be forced to compile in the next compilation. Note that the CMake configuration and the configuration modifications made by menuconfig will not be deleted during cleanup.
· idf.py fullclean: deleting the entire build directory, including all CMake configuration output files. When building the project again, CMake will configure the project from scratch. Please note that this command will recursively delete all files in the build directory, so use it with caution, and the project configuration file will not be deleted.
· idf.py flash: flashing the executable program binary file generated by build to the target ESP32-C3. The options -p and -b are used to set the device name of the serial port and the baud rate for flashing, respectively. If these two options are not specified, the serial port will be automatically detected and the default baud rate will be used.
· idf.py monitor: displaying the serial port output of the target ESP32-C3. The option -p can be used to specify the device name of the host-side serial port. During serial port printing, press the key combination Ctrl+] to exit the monitor.
The above commands can also be combined as needed. For example, the command idf.py build flash monitor will perform code compilation, flash, and open the serial port monitor in sequence.
You can visit https://bookc3.espressif.com/build-system to know more about ESP-IDF compilation system.
4.4 Practice: Compiling Example Program “Blink”
4.4.1 Example Analysis
This section will take the program Blink as an example to analyse the file structure and coding rules of a real project in detail. The Blink program implements the LED blinking effect, and the project is located in the directory examples/get-started/blink, which contains a source file, configuration files, and several compilation scripts.
The smart light project introduced in this book is based on this example program. Functions will be gradually added in later chapters to finally complete it.
Source code In order to demonstrate the entire development process, the Blink program has been copied to esp32c3-iot-projects/device firmware/1 blink.
The directory structure of the blink project files is shown in Figure 4.15.
The blink project contains only one main directory, which is a special component that
Chapter 4. Setting Up Development Environment 53

Figure 4.15. File directory structure of the blink project

must be included as described in section 4.3.2. The main directory is mainly used to store the implementation of the app_main() function, which is the entry point to the user program.The blink project does not include the components directory, because this example only needs to use the components that come with ESP-IDF and does not require additional components. The CMakeLists.txt included in the blink project is used to guide the compilation process, while Kconfig.projbuild is used to add configuration items for this example program in menuconfig. Other unnecessary files will not affect the compilation of the code, so they will not be discussed here. A detailed introduction to the blink project files is as follows.

1. /blink.c includes the following header files/

2. #include <stdio.h>

//Standard C library header file

3. #include “freertos/freeRTOS.h” //FreeRTOS main header file

4. #include “freertos/task.h”

//FreeRTOS Task header file

5. #include “sdkconfig.h”

//Configuration header file generated by kconfig

6. #include “driver/gpio.h”

//GPIO driver header file

The source file blink.c contains a series of header files corresponding to function declara-

tions. ESP-IDF generally follows the order of including standard library header files, FreeR-

TOS header files, driver header files, other component header files, and project header files.

The order in which header files are included may affect the final compilation result, so try to

follow the default rules. It should be noted that sdkconfig.h is automatically generated

by kconfig and can only be configured through the command idf.py menuconfig.

Direct modification of this header file will be overwritten.

1. /*You can select the GPIO corresponding to the LED in idf.py menuconfig, and the modification result of menuconfig is that the value of CONFIG_BLINK

_GPIO will be changed. You can also directly modify the macro definition

here, and change CONFIG_BLINK_GPIO to a fixed value.*/ 2. #define BLINK_GPIO CONFIG_BLINK_GPIO

3. void app_main(void)

4. {

/*Configure IO as the GPIO default function, enable pull-up mode, and

disable input and output modes*/

gpio_reset_pin(BLINK_GPIO);

54 ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT

8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. }

/Set GPIO to output mode/ gpio_set_direction(BLINK_GPIO, GPIO_MODE_OUTPUT); while(1) {
/Print log/ printf(“Turning off the LEDn”); /Turn off the LED (output low level)/ gpio_set_level(BLINK_GPIO, 0); /Delay (1000 ms)/ vTaskDelay(1000 / portTICK_PERIOD_MS); printf(“Turning on the LEDn”); /Turn on the LED (output high level)/ gpio_set_level(BLINK_GPIO, 1); vTaskDelay(1000 / portTICK_PERIOD_MS); }

The app_main() function in the Blink example program serves as the entry point for user programs. It is a simple function with no parameters and no return value. This function is called after the system has completed initialisation, which includes tasks such as initialising the log serial port, configuring single/dual core, and configuring the watchdog.

The app_main() function runs in the context of a task named main. The stack size and priority of this task can be adjusted in menuconfig Componentconfig Common ESP-related.

For simple tasks like blinking an LED, all the necessary code can be implemented directly in the app_main() function. This typically involves initialising the GPIO corresponding to the LED and using a while(1) loop to toggle the LED on and off. Alternatively, you can use FreeRTOS API to create a new task that handles the LED blinking. Once the new task is successfully created, you can exit the app_main() function.

The content of main/CMakeLists.txt file, which guides the compilation process for the main component, is as follows:

1. idf_component_register(SRCS “blink.c” INCLUDE_DIRS “.” )

Among them, main/CMakeLists.txt only calls one compilation system function, that is idf_component_register. Similar to the CMakeLists.txt for most other components, blink.c is added to SRCS, and the source files added to SRCS will be compiled. At the same time, “.”, which represents the path where CMakeLists.txt is located, should be added to INCLUDE_DIRS as the search directories for header files. The content of CMakeLists.txt is as follows:
1. #Specify v3.5 as the oldest CMake version supported by the current project 2. #Versions lower than v3.5 must be upgraded before compilation continues 3. cmake_minimum_required(VERSION 3.5) 4. #Include the default CMake configuration of the ESP-IDF compilation system

Chapter 4. Setting Up Development Environment 55

5. include($ENV{IDFPATH}/tools/cmake/project.cmake) 6. #Create a project named “blink” 7. project(myProject)
Among them, the CMakeLists.txt in the root directory mainly includes $ENV{IDF PATH}/tools/cmake/project.cmake, which is the main CMake configuration file provided by ESP-IDF. It is used to con

References

• ESP Insights Dashboard
• ESP RainMaker Dashboard
• esp.device.tv
• ESP32 Forum - Index page
• GitHub - espressif/idf-installer: ESP IDF Windows Installer
• ESP Product Selector
• proguard-rules.pro
• Swagger UI
• Swagger UI
• Swagger UI
• Swagger UI
• Swagger UI
• Swagger UI
• Swagger UI
• Bootloader - ESP32 - — ESP-IDF Programming Guide latest documentation
• Build System - ESP32-C3 - — ESP-IDF Programming Guide v4.3.2 documentation
• 快速入门 - ESP32-C3 - — ESP-IDF 编程指南 v4.3.2 文档
• mDNS Service - ESP32 - — ESP-IDF Programming Guide v4.3.2 documentation
• Non-volatile storage library - ESP32-C3 - — ESP-IDF Programming Guide v4.3.2 documentation
• Unified Provisioning - ESP32 - — ESP-IDF 编程指南 v4.3.2 文档
• C API Reference — ESP RainMaker Programming Guide documentation
• Python API Reference — ESP RainMaker Programming Guide documentation
• Secure Boot V2 - ESP32-C3 - — ESP-IDF Programming Guide latest documentation
• USB Serial/JTAG Controller Console - ESP32-C3 - — ESP-IDF Programming Guide latest documentation
• Visual Studio Code - Code Editing. Redefined
• dl.espressif.com/dl/esp-idf/
• ESP-IDF Programming Guide - ESP32 - — ESP-IDF Programming Guide latest documentation
• espsecure.py - ESP32 - — esptool.py latest documentation
• GitHub - espressif/book-esp32c3-iot-projects: 《ESP32-C3 物联网工程开发实战》配套代码
• GitHub - espressif/esp-idf: Espressif IoT Development Framework. Official development framework for Espressif SoCs.
• GitHub - espressif/esp-idf: Espressif IoT Development Framework. Official development framework for Espressif SoCs.
• GitHub - espressif/esp-idf at v4.3.2
• GitHub - espressif/esp-insights: ESP Insights: A remote diagnostics/observability framework for connected devices
• GitHub - espressif/esp-insights: ESP Insights: A remote diagnostics/observability framework for connected devices
• GitHub - espressif/esp-rainmaker: ESP RainMaker Agent for firmware development

Read User Manual Online (PDF format)

Download This Manual (PDF format)

Download this manual  >>