Connecting to an MQTT broker

In this exercise, we will establish bidirectional communication between your board and another MQTT client which enables you to control/monitor the board remotely from any other MQTT client, running on a PC, tablet or smartphone.

When pressing button 1 on the board, the message “Hi from the nRF91 Series device” is published to CONFIG_MQTT_PUB_TOPIC, and any client subscribed to this topic will receive this message.

On the other hand, when the “LED1ON” message is sent from another client to the CONFIG_MQTT_SUB_TOPIC topic, the LED on the board (LED1 on the nRF91 Series DK, red LED on the Thingy:91) is turned ON. When the “LED1OFF” message is sent from another client to the CONFIG_MQTT_SUB_TOPIC topic, the LED is turned OFF.

We will practice using the MQTT library in nRF Connect SDK to:

• Configure the board as an MQTT client
• Connect to a public MQTT broker (mqtt.nordicsemi.academy) . The source code for the server is hosted on Nordic Playground GitHub Repo. Note that you can use the same code to connect to a different MQTT server for example ( broker.hivemq.com or test.mosquitto.org )
• Publish and subscribe to MQTT topics, set by the Kconfig symbols: CONFIG_MQTT_PUB_TOPIC and CONFIG_MQTT_SUB_TOPIC
• Control an LED on the board from another MQTT client running on a PC, tablet, or a smartphone
• Read the status of a button on the board from another MQTT client running on a PC, tablet, or a smartphone

Exercise Steps

1. In the GitHub repository for this course, go to the base code for this exercise, found in lesson4/cellfund_less4_exer1, of whichever version directory you are using (v2.2.0-v2.3.0 or v2.4.0-v2.x.x).

Notice that the source code is divided into two files: main.c and mqtt_connection.c.

• main.c: Establishes an LTE connection, configures the LED and the button on the board, and enters an infinite loop to keep the MQTT connection alive.
• mqtt_connection.c: Configures the MQTT client and its socket, defines the MQTT handler callback function which is called on different MQTT events and defines functions to manage subscribing and publishing to topics.

2. Enable and configure the MQTT library.

2.1. Enable and configure the MQTT library in your application by enabling the following Kconfig symbols in the prj.conf file.

CONFIG_MQTT_LIB=y
CONFIG_MQTT_CLEAN_SESSION=y

CONFIG_MQTT_CLEAN_SESSION is to disable/enable persistent sessions. Setting this flag to y disables a persistent MQTT session.

2.2 Configure the topic name for publishing CONFIG_MQTT_PUB_TOPIC, the topic name for subscribing CONFIG_MQTT_SUB_TOPIC, and the message to be sent when the button is pressed CONFIG_BUTTON_EVENT_PUBLISH_MSG, as well as the MQTT broker name and TCP port.

CONFIG_MQTT_PUB_TOPIC="devacademy/publish/topic"
CONFIG_MQTT_SUB_TOPIC="devacademy/subscribe/topic"
CONFIG_BUTTON_EVENT_PUBLISH_MSG="Hi from the nRF91 Series device"
CONFIG_MQTT_BROKER_HOSTNAME="mqtt.nordicsemi.academy"
CONFIG_MQTT_BROKER_PORT=1883

2.3  Include the header file for the MQTT Library in main.c.

#include <zephyr/net/mqtt.h>

3. Define the function client_init() to initialize the MQTT client instance.

In this function, we:

A. Initialize the client instance with mqtt_client_init().

B. Call the function broker_init() to resolve the hostname and get the IP address, which we will use to populate the MQTT broker structure.

C. Populate the mqtt_client struct, which includes:

• Passing the pointer to the broker structure client->broker = &broker.
• Registering the MQTT event handler client->evt_cb = mqtt_evt_handler.
• Assigning an MQTT client ID for the board. The client ID is generated using the AT command +CGSN, which returns the board’s IMEI. This is because the MQTT client ID must be unique across all clients connected to a certain broker.
• The broker does not require a password or a user name, therefore we set these two to NULL in client->password = NULL; and client->user_name = NULL;

D. Assign the receive and transmit buffers. Note that rx_buffer and tx_buffer are already defined in mqtt_connection.c .

E. Set the transport type of the MQTT client to non-secure (MQTT_TRANSPORT_NON_SECURE), since we are using non-secure TCP transport for MQTT connection.

In mqtt_connection.c, add the definition for the client_init() function.

int client_init(struct mqtt_client *client)
{
	int err;
	/* initializes the client instance. */
	mqtt_client_init(client);

	/* Resolves the configured hostname and initializes the MQTT broker structure */
	err = broker_init();
	if (err) {
		LOG_ERR("Failed to initialize broker connection");
		return err;
	}

	/* MQTT client configuration */
	client->broker = &broker;
	client->evt_cb = mqtt_evt_handler;
	client->client_id.utf8 = client_id_get();
	client->client_id.size = strlen(client->client_id.utf8);
	client->password = NULL;
	client->user_name = NULL;
	client->protocol_version = MQTT_VERSION_3_1_1;

	/* MQTT buffers configuration */
	client->rx_buf = rx_buffer;
	client->rx_buf_size = sizeof(rx_buffer);
	client->tx_buf = tx_buffer;
	client->tx_buf_size = sizeof(tx_buffer);

	/* We are not using TLS in Exercise 1 */
	client->transport.type = MQTT_TRANSPORT_NON_SECURE;


	return err;
}

4. Define the function subscribe() to subscribe to a specific topic.

We can subscribe to as many MQTT topics as we want.

For each topic of interest, declare a variable of type mqtt_topic. This variable needs to contain the topic name (in UTF-8 format), the length of the topic name, and the quality of service requested for the subscription.

This is done in the code below where we have created one variable subscribe_topic of type mqtt_topic, to subscribe to CONFIG_MQTT_SUB_TOPIC with QoS1

Once we have declared the topic(s) of interest, we need to create a subscription list variable of type mqtt_subscription_list. In the initialization of the list, we must provide a pointer to the topic or a pointer to the array of topics (if subscribed to more than one topic). In addition, we specify the number of topics and a message id, which can be a random number, and is used to identify the subscription request.

Once we have the list variable initialized, we can call the MQTT library function mqtt_subscribe(), which takes two parameters, the mqtt_client and the mqtt_subscription_list.

In mqtt_connection.c, add the definition of the subscribe() function.

static int subscribe(struct mqtt_client *const c)
{
	struct mqtt_topic subscribe_topic = {
		.topic = {
			.utf8 = CONFIG_MQTT_SUB_TOPIC,
			.size = strlen(CONFIG_MQTT_SUB_TOPIC)
		},
		.qos = MQTT_QOS_1_AT_LEAST_ONCE
	};

	const struct mqtt_subscription_list subscription_list = {
		.list = &subscribe_topic,
		.list_count = 1,
		.message_id = 1234
	};

	LOG_INF("Subscribing to: %s len %u", CONFIG_MQTT_SUB_TOPIC,
		(unsigned int)strlen(CONFIG_MQTT_SUB_TOPIC));

	return mqtt_subscribe(c, &subscription_list);
}

5. Upon a successful connection to a broker (MQTT_EVT_CONNACK), call subscribe() to subscribe to topics.

First, investigate evt->result for a successful connection or not, then call subscribe() to subscribe to the topic CONFIG_MQTT_SUB_TOPIC.

Add the following lines under the MQTT_EVT_CONNACK case.

if (evt->result != 0) {
	LOG_ERR("MQTT connect failed: %d", evt->result);
	break;
}

LOG_INF("MQTT client connected");
subscribe(c);
break;

6. On event MQTT_EVT_PUBLISH, listen to published messages received from the broker and extract the message.

6.1 First we extract the payload using get_received_payload(). Then we examine the QoS of the received message and if it is 1 (at least once), we send an acknowledgment with mqtt_publish_qos1_ack().

The function to extract the published payload, get_received_payload(), is defined in mqtt_connection.c and uses the MQTT API functions mqtt_read_publish_payload_blocking() and mqtt_readall_publish_payload().

const struct mqtt_publish_param *p = &evt->param.publish;
//Print the length of the recived message 
LOG_INF("MQTT PUBLISH result=%d len=%d", evt->result, p->message.payload.len);

//Extract the data of the recived message 
err = get_received_payload(c, p->message.payload.len);
		
//Send acknowledgment to the broker on receiving QoS1 publish message 
if (p->message.topic.qos == MQTT_QOS_1_AT_LEAST_ONCE) {
	const struct mqtt_puback_param ack = {
		.message_id = p->message_id
	};

	/* Send acknowledgment. */
	mqtt_publish_qos1_ack(c, &ack);
}

6.2 If the extraction of the received message was successful, examine the message and use strncmp() to compare it to the LED ON and LED OFF commands and turn the LED on or off accordingly.

//On successful extraction of data 
if (err >= 0) {
	data_print("Received: ", payload_buf, p->message.payload.len);
	// Control the LED 
	if (strncmp(payload_buf, CONFIG_TURN_LED_ON_CMD, sizeof(CONFIG_TURN_LED_ON_CMD) - 1) == 0) {
		dk_set_led_on(LED_CONTROL_OVER_MQTT);
	} else if (strncmp(payload_buf, CONFIG_TURN_LED_OFF_CMD, sizeof(CONFIG_TURN_LED_OFF_CMD) - 1) == 0) {
		dk_set_led_off(LED_CONTROL_OVER_MQTT);
	}
}

6.3 If the extraction was unsuccessful, examine the error code from get_received_payload().

If the error is the payload buffer being too small, print an error message. If the extraction failed for any other reason, disconnect from the MQTT broker.

// On failed extraction of data - Payload buffer is smaller than the recived data . Increase 
else if (err == -EMSGSIZE) {
	LOG_ERR("Received payload (%d bytes) is larger than the payload buffer size (%d bytes).",
		p->message.payload.len, sizeof(payload_buf));
//On failed extraction of data - Failed to extract data, disconnect 
} else {
	LOG_ERR("get_received_payload failed: %d", err);
	LOG_INF("Disconnecting MQTT client...");

	err = mqtt_disconnect(c);
	if (err) {
		LOG_ERR("Could not disconnect: %d", err);
	}
}

7. Publish data to the broker.

In order to publish to a broker (send a message to a topic), we need to use the MQTT library function mqtt_publish(), which takes two parameters: a pointer to the client instance (mqtt_client) and a pointer to a variable of type mqtt_publish_param which encapsulates the message to be sent.

7.1 Define the function data_publish() to publish data.

The function should take as input the client instance pointer, the quality of service requested, the data to be sent, and the length of the data. The function needs to populate the members of the mqtt_publish_param struct and call mqtt_publish() to publish the message to the broker.

In mqtt_connection.c, add the following definition of data_publish().

int data_publish(struct mqtt_client *c, enum mqtt_qos qos,
	uint8_t *data, size_t len)
{
	struct mqtt_publish_param param;

	param.message.topic.qos = qos;
	param.message.topic.topic.utf8 = CONFIG_MQTT_PUB_TOPIC;
	param.message.topic.topic.size = strlen(CONFIG_MQTT_PUB_TOPIC);
	param.message.payload.data = data;
	param.message.payload.len = len;
	param.message_id = sys_rand32_get();
	param.dup_flag = 0;
	param.retain_flag = 0;

	data_print("Publishing: ", data, len);
	LOG_INF("to topic: %s len: %u",
		CONFIG_MQTT_PUB_TOPIC,
		(unsigned int)strlen(CONFIG_MQTT_PUB_TOPIC));

	return mqtt_publish(c, &param);
}

A few things to note:

• We are setting the topic the device will publish to in the line param.message.topic.topic.utf8 = CONFIG_MQTT_PUB_TOPIC.
• The data to be sent is passed to the function, uint8_t *data, and is set in the line param.message.payload.data = data.
• The message ID is set to be a random number by calling the function sys_rand32_get().
• Since we are not using duplication or a persistent MQTT session, the flags dup_flag and retain_flag are set to 0.

7.2 When button 1 is pressed, call data_publish() to publish a message.

In button_handler() in main.c, if button 1 has been pressed call data_publish() with the four parameters:

• Client instance pointer &client which is defined in main.c and initialized through the function client_init() in step 3.
• The quality of service requested (MQTT_QOS_0_AT_MOST_ONCE, MQTT_QOS_1_AT_LEAST_ONCE, or MQTT_QOS_2_EXACTLY_ONCE). In this exercise, we are using MQTT_QOS_1_AT_LEAST_ONCE both for subscription and publishing.
• Pointer to the message to be sent CONFIG_BUTTON_EVENT_PUBLISH_MSG defined in prj.conf.
• The size of the message to be sent. Note that we are not sending the null terminator of the string here and hence the -1 at the end.

if (button_state & DK_BTN1_MSK){	
	int err = data_publish(&client, MQTT_QOS_1_AT_LEAST_ONCE,
		   CONFIG_BUTTON_EVENT_PUBLISH_MSG, sizeof(CONFIG_BUTTON_EVENT_PUBLISH_MSG)-1);
	if (err) {
		LOG_INF("Failed to send message, %d", err);
		return;	
	}
}

8. Build the exercise and flash it on your board.

9. Examine the log output

On a successful connection to the LTE network and the MQTT broker you should see an output similar to the one below. In addition, the LTE connection LED (LED2 on the nRF91 Series DK, green LED on the Thingy:91) should be on.

*** Booting nRF Connect SDK 2.6.1-3758bcbfa5cd ***
[00:00:00.312,052] <inf> Lesson4_Exercise1: Initializing modem library
[00:00:00.519,042] <inf> Lesson4_Exercise1: Connecting to LTE network
[00:00:03.930,969] <inf> Lesson4_Exercise1: RRC mode: Connected
[00:00:05.255,859] <inf> Lesson4_Exercise1: Network registration status: Connected - roaming
[00:00:05.256,042] <inf> Lesson4_Exercise1: Connected to LTE network
[00:00:05.442,962] <inf> Lesson4_Exercise1: IPv4 Address found 20.82.16.164
[00:00:05.649,902] <inf> Lesson4_Exercise1: MQTT client connected
[00:00:05.649,932] <inf> Lesson4_Exercise1: Subscribing to: devacademy/subscribe/topic len 26
[00:00:05.786,956] <inf> Lesson4_Exercise1: SUBACK packet id: 1234
[00:00:10.661,132] <inf> Lesson4_Exercise1: RRC mode: Idle

Testing

9. To test the application, let’s set up a MQTT client to communicate with our device.

You will need an MQTT client running on your PC, smartphone or tablet. We will use MQTT Explorer, which enables us to create an MQTT client on our PC. You can use any MQTT client of your preference, there are also available MQTT clients for Android and iOS (for example Mqtt Dashboard for Android)

10. Connect to the MQTT broker.

In MQTT Explorer, add a connection to the same broker the board is connected to, by providing the broker name, hostname and its port as shown in the illustration below. Make sure to switch off TLS and certificates. Then click on Connect.

11. Subscribe to the topic that the board is publishing to.

In the search bar at the top of the window, input the topic name that the board is publishing to (set by CONFIG_MQTT_PUB_TOPIC defined in prj.conf), to subscribe to it.

The default value is devacademy/publish/topic.

12. Push button 1 on your board. You should see a message show up, and if you expand it to reveal the payload, you should see the message “Hi from the nRF91 Series device”.

13. Publish to the topic that the board is subscribed to.

In the panel to the right, scroll down to find the “Publish” window. Enter the topic name that the board is subscribed to (set by CONFIG_MQTT_SUB_TOPIC defined in prj.conf). The default value is devacademy/subscribe/topic.

Then select “raw” as the message type, and QoS is 1.

Now type in the command for turning on the LED

LED1ON

Click Publish.

Observe that either LED1 on the nRF91 Series DK or the red LED on the Thingy:91 turns on.

The red on the Thingy:91 will be mixed with the connection green LED, so you should see a yellow color.

Now send the command for turning off the LED

LED1OFF

The solution for this exercise can be found in lesson4/cellfund_less4_exer1_solution, of whichever version directory you are using (v2.2.0-v2.3.0 or v2.4.0-v2.x.x).