Thursday, August 9, 2012

Forget Equality

The general concept of peer-to-peer networks is extremely attractive. It appeals to my philosophical leanings and to my sense of engineering elegance. The prospect of billions of devices seamlessly interacting with one another seems to allow the Internet of Things to escape the limitations of centralized command and control, instead taking full advantage of Metcalfe's Law to create more value through more interconnections.

But true peer-to-peer communication isn't perfect democracy – it's senseless cacophony. In the IoT, devices at the edge of the network have no need to be connected with other devices at the edge of the network – there is zero value in the information. These devices have simple needs to speak and hear: sharing a few bytes of data per hour on bearing temperature and fuel supply for a diesel generator, perhaps. Therefore, burdening them with protocol stacks, processing, and memory to allow true peer-to-peer networking is a complete waste of resources and creates more risk of failures, management and configuration errors, and hacking.

Having said that, there is obviously a need to transport the data destined to or originating from these edge devices. The desired breakthrough for a truly universal IoT is using increasing degrees of intelligence and networking capability to mange that transportation of data.

Conceptually, a very simple three-level model will suffice. At the edge of the network are simple Devices. They transmit or receive their small amounts of data in a variety of ways: wirelessly over any number of protocols, via power line networking, or by being directly connected to a higher level device. These edge devices simply "chirp" their bits of data or listen for chirps directed toward them (and how are these addressed, you might ask – we'll get there in a later blog post).

Note that I've said nothing about error-checking, routing, higher-level addressing or anything of the sort. That's because none of these are needed. Edge devices (Level I, if you will) are fairly mindless "worker bees" existing on a minimum of data flow. This will suffice for the overwhelming majority of devices connected to the IoT*.

The emphasis in the last sentence above is a key point. Much of what has been written about the IoT assumes an IP stack in every refrigerator, parking meter, and fluid valve. Why? It's obvious that these devices won't need the decades of built-up network protocol detritus encoded in TCP/IP. We all must free our thinking from our personal experience of the networking of computers, Smartphones and human users to address the much simpler needs of the myriad devices at the edge of the IoT.

So if the end devices aren't capable of protocol intelligence, it must reside somewhere. And the major elements of that somewhere are the Level II Propagator nodes. These are technologically a bit more like the networking equipment with which we are all familiar, but they operate in a different way. Propagators listen for data "chirping" from any device. Based on a simple set of rules regarding the "arrow" of transmission (toward devices or away from devices), propagator nodes decide how to broadcast these chirps to other propagator nodes or to the higher-level Integrator device I'll discuss in a moment.

In order to scale to the immense size of the Internet of Things, these propagator nodes must be capable of a great deal of discovery and self-organization. They will recognize other propagator nodes within range, set up simple routing tables of adjacencies, and discover likely paths to the appropriate integrators. I've solved this sort of problem before with wireless mesh networking and although the topology algorithms are complex, the amount of data exchange needed is small.

One of the important capabilities of propagator nodes will be their ability to prune and optimize broadcasts. Chirps passing from-and-to end devices may be combined with other traffic and forwarded in the general direction of their transmission "arrow". In my view of the IoT, propagators are the closest thing to the traditional idea of peer-to-peer networking, but they are providing this networking on behalf of devices and integrators at levels "above" and "below" themselves. Any of the standard networking protocols may be used, and propagator nodes will perform important translation functions between different networks (power line or Bluetooth to ZigBee or WiFi, for example).

Integrator functions are where the chirps from hundreds to millions of devices are analyzed and acted upon. Integrator functions also send their own chirps to get information or set values at devices – of course these chirps' transmission arrow is pointed toward devices. Integrator functions may also incorporate a variety of inputs, from big data to social networking trends and "Likes" to weather reports.

Integrator functions are the human interface to the IoT. As such, they will be built to reduce the unfathomably large amounts of data collected over a period of time to a simple set of alarms, exceptions, and other reports for consumption by humans. In the other direction, they will be used to manage the IoT by biasing devices to operate within certain desired parameters.

Using simple concepts such as "cluster" and "avoid", integrated scheduling and decision-making processes within the integrator functions will allow much of the IoT to operate transparently and without human intervention. One integrator function might be needed for an average household, operating on a Smartphone, computer, or home entertainment device. Or the integrator function could be scaled up to a huge global enterprise, tracking and managing energy usage across a corporation, for example.

When it comes to actually packaging and delivering products, some physical devices will certainly be combinations of functions. Propagator nodes combined with one or more end devices certainly make sense, as will other combinations. But the important concept here is to replace the idea of peer-to-peer for everything with a graduated amount of networking delivered as needed and where needed. In the Internet of Things, we need a division of labor (like ant and bee colonies) so that devices with not much to say or hear receive only the amount of networking they need and no more.

Next time, we'll talk about the communications architecture for the IoT in more detail and why the needs of the crowd are different from the needs of the cloud.

*Yes, there will be a relatively small number (still billions and billions) of more-sophisticated devices connected to the IoT. But these will connect with good ol' IPv6, as the investment of a protocol stack is worth it.


  1. I like your concept of devices on the edge of the network. I can draw a human parallel by saying that I live in a rural area, and don't have much use for much of what goes on inside of the local town. I get more information that I can use from the local township residents. A good post!

  2. Thanks for your comment, Joe.

    That's very much what I am thinking. In the IoT, the "neighborhoods" aren't only (or even primarily) geographic, but will be formed by the Integrator functions as communities of interest or zones of control.

    In a future post, I'll share what I've been thinking about "local" relevance as an aspect of naming devices in the IoT.