From Oil Pipelines to the IoT: A Brief History of MQTT

If it feels like the Internet of Things (IoT) has kind of gone from zero to ubiquitous hype in the space of no time at all, you're not wrong: it's only been around in its current form since the late 90s. Now I know, many might argue that there was a "smart" toaster in the 70s or a monitored coffee pot in the 80s, but the IoT as we now know it only really took shape at the turn of the century. And while it showed some potential right from the early days, it needed a few developments before it could truly become useful.

In the late 20th century and the first decade of the 21st century, those developments happened; there was a convergence of several factors that suddenly made the IoT viable. And then the IoT exploded. So what were those factors? There were many, like the advent of RFID technology, the introduction of IPv6, which provides approximately 340 undecillion (that's a real counting word) addresses and drastically increases the amount of connected devices we can have, and faster network connections.

However, one factor that is arguably among the most important factors for the state of the IoT today is Message Queuing Telemetry Transport, or MQTT.

MQTT as the Backbone

It doesn't matter how sophisticated IoT devices are: they are just about useless if they cannot communicate with other devices, services, or applications. As with most environments with devices and systems that need to send messages to each other, the IoT needs a protocol to connect its "Things". There are many options available for this. HTTP might seem like the obvious choice, but its not too well-suited to IoT requirements (more on that in the next article). Another option is to utilize an enterprise messaging system like AMQP. However, there is one protocol that is so widely used that it's become an ISO standard, and can be found in IoT applications from production floors to smart homes: MQTT. But how did it get so popular, and why is it so ubiquitous in the IoT? I'll try answer that second question in a future blog post. For now, I'll look at how MQTT rose to fame by considering its history.  

SCADA to the Future

Back in the late nineties, Supervisory Control and Data Acquisition systems were used to manage large industrial infrastructure, such as oil pipelines or power grids (in fact, SCADA is still very much used today). Essentially, SCADA systems allowed operators to control various aspects of the infrastructure remotely, such as switching a generator on or off in a power grid. Incidentally, because SCADA systems could operate machinery and devices by connecting to them remotely, they are largely considered forerunners to today's IoT systems.

The means of connecting to remote devices and controlling them was referred to as "telemetry". The problem with telemetry in those days was that there were various protocols, usually developed by the manufacturers of the devices or machinery, and thus proprietary. This resulted in all kinds of compatibility issues. Two engineers from IBM decided they would try solve this by developing a single open-source protocol to rule them all. These engineers were Andy Stanford-Clark and Arlen Nipper (two kind-of unsung heroes of the modern IoT age, if you ask me), and together they began work on something they dubbed, "Message Queuing Telemetry Transport", or MQTT.

Stanford-Clark and Nipper had a philosophy for the protocol they were developing: it had to be compact, easy to understand, and easy to implement (these tenets are still upheld to this day, as Stanford-Clark, unofficially the patron saint of MQTT, tries to ensure that any updates to the protocol adhere to those three principles). They released the first version into the wild in 1998.

Although it was initially only intended for managing components of oil pipelines remotely via satellite, it soon became obvious that MQTT might have applications outside of SCADA systems, too.

The IoT Adopts MQTT

"I really only foresaw that it would be useful in the SCADA world, but then we saw uses for it in other industries like vehicle telematics, and agriculture, and healthcare. A whole lot of different industries." (Andy Stanford-Clark on IBM's Internet of Things Blog)

As the IoT was burgeoning in the first decade of the 21st Century, engineers and manufacturers needed a way to connect their new-fangled devices. The simple sophistication and utility of MQTT suited their purposes nicely, and it slowly became more widespread. Then, around 2008, came a tipping point (as Stanford-Clark calls it) when the open-source MQTT broker Mosquitto was released (nowadays called Eclipse Mosquitto). Suddenly, MQTT was more accessible, and its adoption rate drastically increased.

By 2014, MQTT was an OASIS standard, and in 2016 it became an ISO standard. By this point, it was already connecting millions of devices worldwide in all kinds of applications and industries. IBM, Amazon, and Microsoft are just some examples of big players who have adopted MQTT as their core protocol.

But Why MQTT?

The simple answer lies in the three goals that the founders of MQTT aimed for: that it be compact, easy-to-use, and easy-to-implement. To a large extent, they achieved these goals. In fact, MQTT was so well designed that there were very few changes made to the protocol during its first ten years of existence. But why would someone choose MQTT over HTTP as a communication protocol? And why MQTT over enterprise messaging systems like AMQP?

To answer these questions, it helps to first consider how MQTT works, and its advantages and disadvantages. And we'll do just that...in a future blog post. We'll also consider some recent problems that MQTT has been faced with (spoiler: it's got to do with security, obviously), as well as what the future for MQTT as the backbone of IoT is.