An introduction to Z-Wave for the end user

In this introduction to Z-Wave, I describe the core principles of the Z-Wave protocol. It is intended for beginners: you won't find detailed technical specifications but general information (i.e. the basics) about the protocol and the way it works. I wrote this article to get you started and give you the tools to continue to learn about Z-Wave on your own. In particular you will learn:

• the different components in a Z-Wave network
• the usual way of setting up a Z-Wave network
• how the Z-Wave components talk to each other
• the different flavours of the Z-Wave specification

What is Z-Wave ?

Have you heard about home automation? There are many devices that give you the ability to control your home with other devices. For instance, with Philips Hue lights, you can control your lights with your smartphone or tablet. There are also motion sensors, temperature sensors, smart locks, etc.

There are many ways your sensor can provide its information: Hue lights use the ZigBee protocol to communicate with the Hue hub, other devices directly use Wi-Fi. Z-Wave is another wireless communications protocol used for home automation. This protocol has three main advantages:

• it uses low-energy radio waves, so the devices can be powered by batteries for a year or two.
• it is a mesh network: every device that is A/C powered is used to relay the network frames. The more, the better!
• it has a range of 90 meter outdoor and 24+ meter indoor that can be extended with the mesh relays.

There are many devices on the market using this technology, and at the time of writing this article, most of the home automation devices can use Z-Wave. It is also worth noting the the standard is open source since 2016.

Z-Wave Gateway

Even though Z-Wave is a mesh network, it needs a gateway, also called primary controller or even hub. This gateway is the brain of the Z-Wave network, when you need to add or remove a device to the Z-Wave network, you talk to the gateway. In practice this is either a USB stick that you plug to your PC or a fully fledge home automation center.

The home centers are usually tied to a specific brand, for example there is the Fibaro home center. The advantage of these hubs is that almost everything is pre-set up for you: once you have plugged your hub, you can manage new Z-Wave devices and create automation scenarios very easily. The disadvantage is that you will most likely be limited in terms of available features. In summary the home center combines the hardware capable of talking with Z-Wave devices and the software allowing you to automate, control and monitor the devices.

On the other end of the spectrum we have the USB stick, like the one Aeotec sells. With this stick alone, you won't be able to do much: you will have to install and configure a home automation center software like Home Assitant or OpenHab to use the Z-Wave gateway. This solution will be harder to implement that the "plug-and-play" home center but it provides three advantages you can't disregard:

1. First, these solutions are completely free, run it on a Raspberry Pi or an old laptop, it's your choice.
2. Second, they are highly interoperable with other protocols. Not only does is work with Z-Wave, but also ZigBee, Wi-Fi, proprietary solutions like Philips Hue, Alexa, etc. It's one control center to rule all of your smart devices. And obviously, you can add any Z-Wave device, you won't be tied to a specific brand.
3. Third, these solutions are highly customisable. For instance, if you start a movie on your Kodi device, you can program a rule to close your blinds (controlled by Z-Wave) and dim the lights (controlled by Philips Hue). This level of integration might be unachievable on a proprietary home center.

Network ID, Node ID, controllers and slaves

Each Z-Wave device is called a node. Each node has two identifiers:

• a Network ID (also called Home ID): this is the identifier of the Z-Wave network. This ID will be the same for every device in the network.
• a Node ID: this is the identifier of the node in the network. This ID will be unique for each device in the network.

There are two kind of nodes:

• controllers: this is your gateway. The controller can control other Z-Wave devices. It is factory programmed with a Network ID and a Node ID. The controller knows every other node in the network and is responsible for assigning Node IDs.
• slaves: this is all of your other devices. The slaves are controlled by other Z-Wave devices. They are not factory programmed with a Network ID or a Node ID.

You can have multiple controllers in one network; however, there will always be exactly one primary controller (the gateway), capable of including new devices and assigning Node IDs.

Z-Wave radio communication

The Z-Wave nodes operate on the 800-900MHz radio frequency range. Your Wi-Fi signal operates mainly on the 2.4GHz frequency (but also others). Since they are different, the signals won't interfere with each other, which is great!

When a sensor has a report to send to the gateway (motion detected, temperature change, button pressed, etc.), there are two ways to do it.

Polling

The first and antiquated solution is polling: the gateway regularly asks each node if there is something new. It starts at node 1, then goes on to node 2, etc. When it reaches the last node it starts back at node 1. The bigger the network, the longer the poll time. On most hubs, you can adjust this poll interval (don't do it though).

Instant status

The second and more sensible solution is called instant status. Here the Z-Wave sensor initiates the status report to the controller by itself (without a poll). So why would we even use polling? One of the main reason is the patent US 5905442 by Lutron Electronics. And Lutron is known to defend its patent with teeth and nails. The good news is that this patent expired in 2016, and while there might be another Lutron patent that could block the use of this particular approach, manufacturers already found other ways to do instant status without infringing it.

Mesh network

When a node needs to do a report, it sends the data to the gateway, that being said, the node might not be in direct line of sight with the controller. That's where the mesh network shines! Even though some nodes cannot reach the gateway directly, they can ask other nodes to transfer the message for them.

In the diagram below you can see that the gateway can only reach nodes 1, 2 and 3. But the nodes 5 and 6 can use Node 3 as an intermediary to talk with the gateway. Each straight line is called a hop, so when Node 6 uses Node 3 to talk to the gateway, there are two hops. The dotted line circles indicate a battery powered device while the solid line circles indicate an always on (A/C powered) device.

The nodes in direct wireless range of another node are called neighbours: the nodes 3 and 5 are neighbours of the node 6.

Mesh networking is actually to make RF communication more reliable, the range extension is just a secondary effect. Think of the times where a node should have been in range but an obstacle or an interference blocks the communication: this is where the redundancy is useful.

Routing

All the intelligence of the network is gathered in one place: the controller. It is in charge of knowing the place of each node, the neighbours of each node and the available routes to reach them.

The network layer specification is not open source. We don't know exactly how any of this happens, how the controller chooses one route over the other, but we have some information. There are some people working on understanding it.

There are two routing options I want to explain: explorer frames and source routing. For the sake of brevity the explanation about explorer frames and source routing is dumbed down. An entire blog post could be dedicated to Z-Wave routing.

Explorer frames (flooding)

When you perform the inclusion process to add a new device to the Z-Wave network, the new device will flood the network with Explorer Frames. Whenever a device sees an explorer frame, it re-broadcasts it, while adding its node ID in the frame. At some point, some explorer frames will reach the controller who will have the list of nodes visited by the explorer frames. This list of nodes constitutes the new route.

This process puts a load on the network since it will be flooded by explorer frames. That's why the network uses source routing whenever possible.

Source routing

With source routing, a Z-Wave node has a list of routes it can use to reach the controller. This list of routes was given by the controller itself. For instance, node 7 in the diagram below has the routes 7 > 2 > Gateway and 7 > 3 > Gateway configured.

When a node wants to send a report, it chooses a route and inserts the route in the packet. Then when a node receives a packet with a route, it re-broadcasts the packet if its node ID is in the route, and discards the packet otherwise.

Z-Wave flavours: Plus, S2, SmartStart

Since its creation, the Z-Wave specification hasn't stopped evolving. The latest addition to the specification is Z-Wave Long Range, described in the next section. Currently on the market you will find Z-Wave (also called Z-Wave Classic), Z-Wave Plus, and Z-Wave Plus S2 devices. What are the differences between these labels?

Z-Wave Plus

Z-Wave Plus, also known as 500 Series (or Gen5) brings several new features to the network:

• a better battery life (+50%)
• a better wireless range (+67%)
• higher bandwidth (+250%)
• over the air (OTA) updates

That said, the most interesting features are Network Wide Inclusion (NWI) and explorer frames. Explorer frames were described earlier in the routing section, these frames allow to find available routes between a device and the gateway.

Network Wide Inclusion is the ability to add a new device in the network even if it is not in direct range of the gateway (NWI relies on explorer frames). Before that, you had to bring your device next to your controller, perform the inclusion and then place your device at its definitive spot. This means that the routing table had to be reconfigured after you installed your device (which is a burden on the network).

Z-Wave Plus also introduces optional security improvements, namely the S2 security framework, Elliptic-curve Diffie Hellman, SmartStart and 128-bit AES encryption.

S2

Z-Wave S2 is a new Security framework. Here is an excerpt from the z-wave alliance announcement:

Z-Wave’s S2 framework was developed in conjunction with cybersecurity hacking experts, giving the already secure Z-Wave devices, new levels of impenetrability.

I can feel the difficulty of selling advanced security without saying that the specification had security holes before, but this is a bit too much for my taste and borderline false advertising. Especially when a few lines further they state that they are "removing vulnerability".

The new security framework brings secure inclusion to the table. The former inclusion process was:

1. Put the gateway on inclusion mode. The gateway is listening for inclusion requests.
2. Put the device in inclusion mode.
3. The gateway sees a new device and automatically includes it.

This was actually a security vulnerability (just take look at WPS which is very similar) since any rogue device could insert itself in the network at step 2. With the S2 security framework a PIN or QR code (called Device Specific Key) is printed on the device itself and the steps look like this:

1. Put the gateway on inclusion mode. The gateway is listening for inclusion requests.
2. Put the device in inclusion mode.
3. The gateway sees that a new device is attempting to join the network and asks the Device Specific Key (DSK) to authenticate it (this step acts as the user's consent).

The new framework also specifies the use of Elliptic Curve Diffie-Hellman for secure key exchanges. This brings a good security even with small keys, which allows to reduce the frame sizes.

Smartstart

Smartstart is the new easy way of including devices in your network. With a smartstart device, you don't have anything to do beside powering on the device to include it in your network.

How does it work? With a smartstart device, the inclusion is initiated automatically on power-ON and repeated at dynamic intervals for as long as the device is not included into a Z-Wave network. Then the gateway can initiate the inclusion process in the background (without the need for user interaction).

That doesn't sound very secure at first, but the new protocol is built on S2. So when you include a S2 device, you will still have to enter the PIN code (called DSK) in the user interface. Here is a diagram showing how it works with and without smartstart, directly extracted from the Silicon Labs official docs:

Z-Wave Plus v2

Were you excited about Z-Wave Plus? Then get ready for Z-Wave Plus version 2.

This version of Z-Wave is also known as 700 Series or Gen 7. With this new version, the optional security features introduced in Z-Wave Plus are now mandatory (S2, SmartStart, ECDH, etc.); however, there are also improvements in the usual areas. In particular, v2 offers:

• better wireless range
• better battery life (up to 10 years!)
• better performance with the 700 Series chips

Z-Wave Long Range

In September 2020, the Z-Wave alliance announced Z-Wave Long Range. In a nutshell, this new specification should provide a network with:

• 4 times the wireless range
• 10 times the node scalability
• 10-year battery life for devices
• backward compatibility and interoperability

Z-Wave LR operates on a star network topology, just like your home Wi-Fi, that also means routing is not necessary anymore. The Long Range network will be able to support up to 4000 devices, while the current Z-Wave specifications only supports 232. The frequency range stays around 900MHZ but the max output power will go up to 30dBm to be able to reach those far away places (currently -1dBm max).

Security

I scattered info about security in the many sections of this blog post. Basically, the first Z-Wave generation (S0) had little to no security. The option to use AES encryption was deeply flawed. Sensepost showed how litte secure the protocol was in a security evaluation (which probably was the first one).

S0 exchanges the network key by encrypting it with a fixed key of 0000000000000000.

The S2 security framework solves the initial problem with Diffie-Hellman and adds secure inclusion. Since the protocol is used in a niche market and was closed-source for a long time, few security researchers decided to spend time on it. Nonetheless, PenTest Partners discovered Z-Shave, a downgrade attack on the S2 security framework.

This attack exploits an integrity flaw in the inclusion process where an attacker can compromise the security class of a new device to make it appear as a S0 device instead of a S2 device. In a nutshell, the attacker has to be near your home when you include the device to perform the attack. Once the attack is successful, the attacker can use your device.

Apart from this, no other vulnerabilities have been found in the protocol yet.

Conclusion

Z-Wave is a great protocol for your home automation system. There are a ton of devices you can use, from door locks, to temperature and motion sensors. And most of the open source home automation systems (Home Assistant, OpenHab, Jeedom, HomeSeer, Domoticz, ...) are capable of using Z-Wave. The only thing I deplore is the lack of security in existing devices: even though the S2 framework exists, there are very few slave devices and even fewer controllers implementing it. So while in theory the protocol has a good (but not super good) security, in practice you'll have to wait for a few years before the market adapts and the new devices use the S2 framework.

Resources