Friday, February 28, 2014

Writing the Book on the IoT

I’d like to announce that I have written a book proposing a new architecture for the coming explosion of devices in the Internet of Things (IoT). I hasten to add that I didn't set out to write a book – rather Intel Corporation approached me on the basis of the blog posts below and asked me to expand my thoughts into a more detailed treatment. You can click the link to learn more about Rethinking the Internet of Things: A Scalable Approach to Connecting Everything.

I did not originally intend to develop a new architecture for the IoT. Rather, I was thinking about the implications of control and scheduling within machine social networks in the context of Metcalfe's Law. The coming tsunami of machine-to-machine interconnections could yield tremendous flows of information – and knowledge. The book builds on the principles outlined in the blog posts below, but it also expands on a number of new directions that I will touch on in later blog posts.

Once we free the machine social network of the drag of human interaction, there is tremendous potential for creating autonomous communities of machines that require only occasional interaction with or reporting to humans. As the Internet of Things expands exponentially over the coming years, it will be expected to connect to devices that are cheaper, dumber, and more diverse. Traditional networking thinking will fail for multiple reasons.

Fundamentally, traditional IP-based peer-to-peer relationships lock out much of the potential richness of the Internet of Things. There will be vast streams of data flowing, many of which are unknown or unplanned. Only a publish/subscribe architecture allows us to tap into this knowledge by discovering interesting data flows and relationships. And only a publish/subscribe network can scale to the tremendous size of the coming Internet of Things. So appliances, sensors, and actuators must use self-classified traffic schemes to allow for discovery and creation of information “neighborhoods”.

Note that the data needs of the IoT are completely different from the traditional global Internet. Most of the communications will be terse machine-to-machine interchanges that are largely asymmetrical, with much more data flowing in one direction (sensor to server, for example) than in the other. And in most cases, losing an individual message to an intermittent or noisy connection will be no big deal. Unlike the traditional Internet, which is primarily human-oriented (and thus averse to data loss), much of the Internet of Things traffic will be analyzed over time, not acted upon immediately. Most of the end devices will be essentially autonomous, operating independently whether anyone is “listening” or not.

Although IPv6 can potentially provide addresses for myriad devices, the largest population of these appliances, sensors, and actuators will lack the horsepower in terms of processors, memory, and bandwidth to run the bloated IP protocol stack. It simply does not make financial sense to burden a simple sensor with all of the protocol overhead needed for host-to-host communications.

Additionally, the conventional implementation of IP protocols implies networking knowledge on the part of device manufacturers: without centrally authorized MAC IDs and end-to-end management, IP falls flat. Many of the hundreds of thousands of manufacturers of all sizes worldwide building moisture sensors, streetlights, and toasters lack the technical expertise to implement legacy network technology in traditional ways.

There is another subtle reason why a new architecture is needed, and it relates to a favorite theme of mine: control loops. When there are real-time sensing and response loops needed in the Internet of Things, traditional network architectures with their round-trip control loops will be problematic. Instead, a way would be needed to engender independent local control loops managing the “business” of appliances, sensors, and actuators while still permitting occasional “advise and consent” communications with central servers.

The only systems on earth that have ever scaled to the size and scope of the Internet of Things are natural systems: pollen distribution, ant colonies, redwoods, and so on. From examining these natural systems, I developed the concept of the three-tiered IoT architecture described in the book: simple end devices, networking specialist propagator nodes, and information-seeking integrator functions.

I hope that you’ll read my explanation of why terse, self-classified messages, networking overhead isolated to a specialized tier of devices, and publish/subscribe relationships are the only way to fully distill the power of the coming Internet of Things. And I hope especially for your feedback, pro and con.