Wednesday, August 15, 2012

Not a Stack, a Crowd

What we all used to know as simple server-based computing architecture has been replaced by the glossy and over-hyped marketing term "the cloud". But from an architecture perspective, the cloud is pretty much the same IP-based networking we've been using for decades. Because the Internet backbone is (at least today) still over-provisioned, an Internet and IP-based cloud can work well, even for important transactions. (Whether that holds true for the decades ahead is a different matter, of course.)

But the Internet of Things shares only a need of wide connectivity with "the cloud". In most other important ways, it’s completely different: crowds of billions of end devices that connect intermittently at very low speeds to other machines, not to humans. In my developing picture of the IoT, this makes traditional protocol stacks irrelevant – or at the very least, overkill.

While the traditional protocols may make sense for connecting Propagator nodes and Integrator functions (see the previous blog post), the vast numerical majority of connections will be to relatively low-data-need devices such as HVAC units, air quality sensors, and street lights. This is the segment of the communications architecture that must be re-thought from the ground up, in my opinion.

Rather than treat intermittent connections, data loss, and low data rates as problems (as they would be in IP), we must embrace these as facts of life in the Internet of Things. It’s a lossy world on the IoT frontier, and that's OK – if we engineer the architecture with that in mind. Most of these end devices won't need constant check-ins with a central site to function. They'll simply keep running, functioning with or without network updates. If an update comes, fine, but there's no immediate response required.

Turning to nature, birdsong and pollen give us another picture of how the IoT devices will treat communications. Many birds sing without expecting (or waiting for) an answer. They sing "blindly" to mark territory, advertise mating availability, or signal danger – and trust in the universe to deliver the message to hearers who may act upon the message. Similarly, trees and other flowering plants broadcast pollen extremely broadly (hence, allergy season) without any feedback on whether the "message" is received. Propagated by winds, pollen may be carried hundreds or thousands of miles away from the originating source.

All of this leads to my heretical view of the very edge of the Internet of Things: it just isn't reliable when viewed from the perspective of a single message. The devices may be switched off at various times, propagation paths may be lost, etc. Yet by sending the same small data chirps over and over, eventually there is a good chance that some or a few will get through. This will mean over-transmitting on a massive scale. But because each data chirp is so small, there is virtually no net cost involved in this over-provisioning.

I believe that this is one of the key things about the Internet of Things that is completely different from the "Big I" Internet: the very small amount of data in each transmission and the lack of criticality of any single transmission. As the traditional Internet becomes clogged with ever larger real-time data streams such as those generated by video and multiplayer gaming, the IoT's growth will be at the fringes of the network with billions of low-duty-cycle, low-data-rate devices.

I believe we'll need a new architecture at the edges of the Internet of Things. In place of the traditional IP protocol stack with hierarchical layers of routing topologies, there will instead be a gigantic crowd of devices speaking and listening – each unconcerned with what's happening anywhere else in the network. Instead of rigid routing paths there will be transient clumps and aggregations of unrelated devices sharing propagation facilities. It's a truly best-effort world, and as I have said before – that will be good enough.

Conceptually, I am breaking this new architecture into the elements of Naming, Communication, and Propagation. Next blog, we'll start with the most challenging aspect of this architecture: naming those billions of IoT end points in the crowd.

7 comments:

  1. I find your concepts interesting. With the numbers of devices you are contemplating, a little chirp of data x a very large number is still a lot of data. There would have to be some kind of aggregation going on at each level before it hit the Big I on the way to some cloud service.

    ReplyDelete
  2. Thanks for your comments, Chris.

    I believe that we're in agreement. That's why I view what I call the Propagator node's role as important in packaging and pruning traffic. I touched on that role in the "Forget Equality" post:
    http://net-of-things.blogspot.com/2012/08/forget-equality.html

    I think I'll also expand on that thought in a future post.

    ReplyDelete
  3. Yes.. maybe it has to be ant net of things..Ants have better visioneering of life. The grasshoppers move at high speed without colliding..Sustainable models can be derived if we look at nature closely.
    Sreenivas Parasa
    www.identium.in

    ReplyDelete
    Replies
    1. Yes, I agree, we can learn much from nature.

      Delete
  4. This comment has been removed by the author.

    ReplyDelete
  5. I also wonder how are we going to tackle the issue of malafide communications in the context of resource constrained devices. In the bird analogy, when a bird listens to a talk or song by another bird, it's inherently been designed by nature to recognize its own kind and (most likely) recognize and reject calls from birds of a different type. At least, it'll be able to tell that the song has been created by one of its own species or not.
    Once standardized protocols are in-place, we'll need to tackle those nodes who are working against the interest of the crowd. Dealing with this issue, however unpleasant, is going to be a necessary evil.

    ReplyDelete
  6. Yes, unfortunately it's always necessary to plan for bad actors. I think you are on the right track in noting that the _receiver_ is doing the sorting between chirps from different "species" -- acting on its "own" chirps and ignoring the rest. I have been thinking about security in a chirp world and will write about it here in the future.

    ReplyDelete