Adding TLS to the MQTT connection

Now we want to encrypt the MQTT connection that we set up in the previous exercise by implementing TLS. We will also cover how to verify the broker’s authenticity.

Recall from lesson 3 that TLS can provide :

• Confidentiality (communication cannot be read by third parties)
• Integrity (communication cannot be altered by third parties)
• Authenticity (verifies the identity of the client and server).

You can read more about how TLS provides authentication, confidentiality, and integrity here.

In this exercise, we will learn how to encrypt communication, first between your device and the MQTT broker and then between the MQTT broker and the other MQTT client. Then we will use the x.509 certificate of a server (or MQTT broker) and convert it into a C header file, and use it to verify the brokers’ authenticity.

Exercise Steps

1. In the GitHub repository for this course, go to the base code for this exercise, found in l4/l4_e2 of whichever version directory you are using.

2. Enable TLS support for socket MQTT Library and change the port number

2.1 Enable TLS for the MQTT library by enabling CONFIG_MQTT_LIB_TLS in prj.conf.

CONFIG_MQTT_LIB_TLS=y

2.2 Change the MQTT broker port by changing the value to 8883. The 8883 is the standard TLS port for MQTT.

CONFIG_MQTT_BROKER_PORT=8883

2.3 Enable the modem key management library in order to store the server certificate in the modem and use it for verification.

CONFIG_MODEM_KEY_MGMT=y

2.4 In mqtt_connection.c, include the header file for modem key management library.

#include <modem/modem_key_mgmt.h>

3. Obtain the certificate from the MQTT broker and convert it into a C header file, certificate.h.

3.1 Using a browser, visit the MQTT broker main page, in this case: https://mqtt.nordicsemi.academy/ and download the mqtt.nordicsemi.academy.chain file in PEM format.

Store the certificate in the exercise root directory of the application (cellfund_less4_exer2) and make sure to name it server_certificate.crt

3.2 Convert the server certificate into a header file in C.

In this exercise, we will use a simple Python script that takes care of that for us. It is provided with the exercise and it will convert the .crt file into a header file certificate.h that will contain the converted certificate as a C string defined as a C macro called CA_CERTIFICATE.

Open a new terminal, navigate to the script folder (lesson4/cellfund_less4_exer2/script), and then run the Python script crt_to_header.py by running the following command

python.exe .\crt_to_header.py

This will generate the converted certificate.h in the /src subdirectory

Note that the script assumes that the root certificate is called server_certificate.crt and is placed in the root directory of your exercise.

3.3 Include the newly generated certificate.h file in mqtt_connection.c

#include "certificate.h"

4. Provision of the server certificate to the modem

4.2 Add the function certificate_provision() that will store the certificate to the modem.

In this function, we write the certificate to the modem, using modem_key_mgmt_write(), which has the following signature

• sec_tag – The security tag associated with the certificate, in our case CONFIG_MQTT_TLS_SEC_TAG.
• cred_type – The type of credential we are storing in the modem. To authenticate the server, we use MODEM_KEY_MGMT_CRED_TYPE_CA_CHAIN, which is the root certificate of the server issues by the Certificate Authorities.
• buf – The actual certificate, defined in the header file certificate.h as the macro CA_CERTIFICATE.
• len – Length of the certificate.

It is important not to rewrite the same certificate every time we boot up the application, to avoid wearing out the flash. Therefore, we are checking if a certificate exists, using modem_key_mgmt_exists(), and if so, checking if it is identical to the one we want to write, using modem_key_mgmt_cmp(). If this is the case, we don’t need to write the certificate to the modem.

int certificate_provision(void)
{
	int err = 0;
	bool exists;

	err = modem_key_mgmt_exists(CONFIG_MQTT_TLS_SEC_TAG,
				    MODEM_KEY_MGMT_CRED_TYPE_CA_CHAIN,
				    &exists);
	if (err) {
		LOG_ERR("Failed to check for certificates err %d\n", err);
		return err;
	}

	if (exists) {
		/* Let's compare the existing credential */
		err = modem_key_mgmt_cmp(CONFIG_MQTT_TLS_SEC_TAG,
					 MODEM_KEY_MGMT_CRED_TYPE_CA_CHAIN,
					 CA_CERTIFICATE, 
					 strlen(CA_CERTIFICATE));
		LOG_INF("%s\n", err ? "mismatch" : "match");
		if (!err) {
			return 0;
		}
	}
	LOG_INF("Provisioning certificates");
	err = modem_key_mgmt_write(CONFIG_MQTT_TLS_SEC_TAG,
				   MODEM_KEY_MGMT_CRED_TYPE_CA_CHAIN,
				   CA_CERTIFICATE,
				   strlen(CA_CERTIFICATE));
	if (err) {
		LOG_ERR("Failed to provision CA certificate: %d", err);
		return err;
	}
	return err;
}

4.3 Store the certificate in the modem while the modem is in offline mode

To use any cryptographic functions in the modem, the application must provision the security credentials to the modem. To be able to provision credentials, the modem must be in offline mode.

This is why it is important to call certificate_provision() in modem_configure(), right after initializing the nRF Modem library when the modem is still in offline mode.

Add the following code snippet in the main.c file.

err = certificate_provision();
if (err != 0) {
	LOG_ERR("Failed to provision certificates");
	return err;
}

5. Modify the client client_init() function to use secure TCP transport instead of non-secure TCP transport.

In mqtt_connection.c, replace the line below

client->transport.type = MQTT_TRANSPORT_NON_SECURE;

With the following

struct mqtt_sec_config *tls_cfg = &(client->transport).tls.config;
static sec_tag_t sec_tag_list[] = { CONFIG_MQTT_TLS_SEC_TAG };

LOG_INF("TLS enabled");
client->transport.type = MQTT_TRANSPORT_SECURE;

tls_cfg->peer_verify = CONFIG_MQTT_TLS_PEER_VERIFY;
tls_cfg->cipher_list = NULL;
tls_cfg->cipher_count = 0;
tls_cfg->sec_tag_count = ARRAY_SIZE(sec_tag_list);
tls_cfg->sec_tag_list = sec_tag_list;
tls_cfg->hostname = CONFIG_MQTT_BROKER_HOSTNAME;

tls_cfg->session_cache = IS_ENABLED(CONFIG_MQTT_TLS_SESSION_CACHING) ?
				    TLS_SESSION_CACHE_ENABLED :
				    TLS_SESSION_CACHE_DISABLED;

In the above code, we are populating the mqtt_sec_config member of the MQTT client instance. The sec_tag_list array holds the security tag associated with the server certificate, that the MQTT library should use for authentication. We enforce peer certificate verification by setting the peer_verify field. We do not specify cipher_list, to allow the use of all cipher suites available in the system. We set the hostname field to the broker hostname, which is required for server authentication.

6. Update the file descriptor for the socket to use the TLS socket instead of a plain TCP socket.

In mqtt_connection.c, change the following line

return -ENOTSUP;

To this line

fds->fd = c->transport.tls.sock;

7. Build the exercise and flash it on your board.

8. Examine the log output.

On a successful connection to an LTE network and connection to the MQTT broker, you should see an output similar to the one below.

*** Booting nRF Connect SDK 2.6.1-3758bcbfa5cd ***
[00:00:00.322,170] <inf> Lesson4_Exercise2: Initializing modem library
[00:00:00.498,870] <inf> Lesson4_Exercise2: Provisioning certificates
[00:00:00.702,545] <inf> Lesson4_Exercise2: Connecting to LTE network
[00:00:10.253,997] <inf> Lesson4_Exercise2: Network registration status: Connected - roaming
[00:00:10.254,150] <inf> Lesson4_Exercise2: Connected to LTE network
[00:00:10.380,371] <inf> Lesson4_Exercise2: IPv4 Address found 137.135.83.217
[00:00:10.380,859] <inf> Lesson4_Exercise2: TLS enabled
[00:00:12.465,270] <inf> Lesson4_Exercise2: MQTT client connected
[00:00:12.465,332] <inf> Lesson4_Exercise2: Subscribing to: devacademy/subscribe/topic len 26
[00:00:12.619,323] <inf> Lesson4_Exercise2: SUBACK packet id: 1234

Testing

We will basically follow the same test procedure we used in Exercise 1, except this time we will have to configure the client to use TLS and configure it to use the server root certificate issued by the certificate authority.

9. In MQTT Explorer, add a new connection and set it up as shown in the illustration below. Note that the port number changed to 8883:

10. Add the server (MQTT broker) root certificate (server_certificate.crt) by clicking on the advanced button.

Click on the CERTIFICATES,

then select SERVER CERTIFICATE (CA) and select the .crt file we downloaded in step 3.1.

Click back twice, and make sure to save the connection settings.

11. Press Connect to establish a connection with the MQTT broker. Repeat steps 11 and 12 of the previous exercise. The functionality of this exercise is identical to the previous one, except for the fact that communication now is over TLS so it ensures confidentiality, integrity of the data sent over the network, and authentication of the server.

The solution for this exercise can be found in l4/l4_e2_sol of whichever version directory you are using.