MQTT is a publish/subscribe protocol that is popular for home automation and
small devices. Programs never talk to each other directly. A broker sits in
the middle: a client publishes a message to a topic such as
home/kitchen/temperature, and the broker forwards it to every client that has
subscribed to a matching topic filter such as home/+/temperature.
Dodo has everything for both sides, much like the web packages:
| You want to | Use | Start with |
|---|---|---|
| Run a broker on your machine or network | std/mqtt/broker |
broker.new().run(b"0.0.0.0:1883") |
| React to messages, publish on a timer, report status | std/mqtt/app |
app.new(b"my-app").on(filter, handler).run(url) |
| Control every step yourself | std/mqtt/hosted |
hosted.Client.connect(url, client_id, &mut workspace) |
| Encode packets for your own transport | std/mqtt |
mqtt.encode_publish, mqtt.decode |
std/mqtt also holds the types every layer shares: mqtt.Message,
mqtt.Outbox, mqtt.QoS, and mqtt.Error. The broker, app and hosted
run on Linux GNU, macOS, and Windows x64; std/mqtt also builds for
WebAssembly and bare-metal targets.
Quickstart
You need two programs: a broker and an app. Save this as broker.dodo:
package main
import "std/mqtt/broker"
fn main() -> i32 {
return broker.new()
.on(b"#", broker.print())
.run(b"127.0.0.1:1883")
}
Start it with dodo run broker.dodo. It accepts clients on port 1883, the
standard MQTT port, and prints every message it sees. Ctrl+C stops it.
In a second terminal, save this app as main.dodo:
package main
import "std/mqtt"
import "std/mqtt/app"
fn main() -> i32 {
return app.new(b"dodo-pinger")
.on(b"ping", mqtt.reply(b"pong", b"hello from Dodo"))
.run(b"mqtt://127.0.0.1")
}
Start it with dodo run. The app connects, subscribes to ping, and answers
every message on ping with a message on pong. Any MQTT client can talk to
it. With the Mosquitto command-line tools installed,
send a ping:
mosquitto_pub -h 127.0.0.1 -t ping -m hi
The broker terminal now shows ping hi and then pong hello from Dodo.
run returns an exit code for main: 0 after a clean stop, 1 if starting
failed (it prints why, for example a wrong address or a refused login), and 2
if any handler returned an error along the way. run keeps going until the
process stops. The app reconnects by itself if the broker restarts.
Handlers
mqtt.reply is a ready-made handler. Your own handlers are structs with a
handle method. The app calls it for every message whose topic matches the
filter you registered it with:
package lamp
import "core/bytes"
import "std/mqtt"
import "std/mqtt/app"
pub struct Lamp {
on: bool
pub fn handle(&mut self, message: &mqtt.Message, outbox: &mut mqtt.Outbox) -> void!mqtt.Error {
if bytes.equal(message.payload, b"on") {
self.on = true
} else if bytes.equal(message.payload, b"off") {
self.on = false
} else {
return err(mqtt.failure(mqtt.ErrorKind.InvalidInput, 0))
}
return outbox.publish_with(b"home/lamp/state", message.payload, mqtt.QoS.AtLeastOnce, true)
}
}
fn main() -> i32 {
return app.new(b"dodo-lamp")
.on(b"home/lamp/set", Lamp { on: false })
.on(b"home/#", app.print())
.run(b"mqtt://127.0.0.1")
}
Handlers are ordinary values, so they keep state between messages, like
Lamp.on here. The handler receives:
message, withtopicandpayloadbytes, itsqos,retain(a stored value the broker sent on subscription), andduplicate(possibly sent before).message.level(1)returns one level of the topic: forhome/kitchen/lightit iskitchen.message.matches(filter)tests another filter.outbox, for messages to publish in response.outbox.publish(topic, payload)sends at QoS 0;outbox.publish_with(topic, payload, qos, retain)chooses the quality of service and whether the broker keeps the message as the topic’s current value. The app sends them right after the handler returns.
Every handler whose filter matches runs, in the order you registered them.
Above, a message on home/lamp/set goes to Lamp and is printed. If a
handler returns an error, the app prints it to stderr, counts it, and carries
on with the next message.
| Ready-made handler | Effect |
|---|---|
mqtt.reply(topic, payload) |
Publishes a fixed payload to a fixed topic. |
mqtt.forward(topic) |
Republishes each payload to another topic. |
app.print(), broker.print() |
Prints topic payload lines to stdout. |
Be careful when a handler publishes to a topic the same app subscribes to: the broker sends that message back, which can loop forever.
Timers and settings
.every(milliseconds, task) runs a task right after connecting and then
repeatedly. A task is a struct with a tick method that publishes through its
outbox, for example a sensor reading:
pub struct Thermometer {
pub fn tick(&mut self, outbox: &mut mqtt.Outbox) -> void!mqtt.Error {
return outbox.publish(b"home/kitchen/temperature", b"21.5")
}
}
fn main() -> i32 {
return app.new(b"kitchen-sensor")
.every(10000, Thermometer {})
.status(b"home/kitchen/status")
.run(b"mqtt://127.0.0.1")
}
To publish a number you compute, format it into a byte buffer first, as shown in format into caller-owned bytes.
| Builder method | Effect |
|---|---|
.on(filter, handler) |
Subscribe and call handler for matching messages. |
.every(milliseconds, task) |
Call task.tick(outbox) after connecting and then periodically. |
.status(topic) |
Keep a retained online or offline on topic, even after a crash. |
.login(username, password) |
Credentials for brokers that require them. |
.will(topic, payload) |
Message the broker publishes if the app disappears without disconnecting. |
.qos(qos) |
Subscription quality of service; mqtt.QoS.AtLeastOnce by default. |
.keep_alive(seconds) |
How often an idle app pings the broker; 60 by default. |
.max_messages(count) |
Disconnect after this many messages. Useful for tests and scripts. |
.build() |
Check the setup and get an App for tests. |
.run(url) |
Connect and run, with printed errors and an exit code. |
.serve(url), .serve_until(url, cancel) |
Run and return a Report, or stop when cancel.cancelled() returns true. |
An invalid filter, such as home/#/light, is reported by build and run
before connecting:
mqtt route #2 (home/#/light): InvalidFilter; use levels separated by /, + for one whole level, or # as the last level
serve returns an app.Report with the received, published, and failed
counts, the number of reconnects, and the last_error. It fails only if the
first connection fails or the broker refuses the login or a subscription. After
that, a lost connection is printed and retried, waiting 1 second at first and
up to 30 seconds between attempts.
Test without a broker
build() returns the configured app without connecting. deliver(topic, payload) runs the matching handlers in process, and tick() runs every
task once. Both return an mqtt.Delivery that records what was published:
package lamp_test
import "core/bytes"
import "std/mqtt"
import "std/mqtt/app"
pub struct Lamp {
pub fn handle(&mut self, message: &mqtt.Message, outbox: &mut mqtt.Outbox) -> void!mqtt.Error {
if !bytes.equal(message.payload, b"on") && !bytes.equal(message.payload, b"off") {
return err(mqtt.failure(mqtt.ErrorKind.InvalidInput, 0))
}
return outbox.publish(b"home/lamp/state", message.payload)
}
}
pub struct Heartbeat {
pub fn tick(&mut self, outbox: &mut mqtt.Outbox) -> void!mqtt.Error {
return outbox.publish(b"home/lamp/heartbeat", b"alive")
}
}
@test
fn lamp_reports_its_state() {
lamp := app.new(b"lamp")
.on(b"home/lamp/set", Lamp {})
.every(10000, Heartbeat {})
.build()!
lamp.deliver(b"home/lamp/set", b"on")
.expect_ok()
.expect_handled(1)
.expect_publish(b"home/lamp/state", b"on")
lamp.deliver(b"home/lamp/set", b"purple").expect_failure().expect_published(0)
lamp.deliver(b"home/garage/set", b"on").expect_handled(0)
lamp.tick().expect_publish(b"home/lamp/heartbeat", b"alive")
}
Run dodo test. The expect_* methods assert and return the delivery, so
they chain. expect_handled(n) counts matching handlers, expect_published(n)
counts published messages, and expect_publish(topic, payload) looks for one.
To inspect results yourself, use handled, failure, len(), published(index),
or contains(topic, payload). A delivery keeps copies of up to
mqtt.DELIVERY_BYTES of published messages.
The broker
broker.new().run(address) is a complete broker for a home network, a lab, or
tests. Use b"127.0.0.1:1883" to accept only programs on the same computer,
or b"0.0.0.0:1883" to accept other devices on your network too.
The broker supports what MQTT 3.1.1 clients expect: QoS 0, 1, and 2,
wildcard subscriptions, retained messages, wills, keep-alive, and a new
connection with the same client ID taking over the old one. Any MQTT 3.1.1
client works with it, including std/mqtt/app, Mosquitto’s tools, and phone
apps.
Broker handlers work like app handlers. They see every message a client publishes to a matching topic, and what they put in the outbox goes to subscribers like any other message:
package doorbell
import "std/mqtt"
import "std/mqtt/broker"
pub struct Doorbell {
rings: u32
pub fn handle(&mut self, message: &mqtt.Message, outbox: &mut mqtt.Outbox) -> void!mqtt.Error {
self.rings += 1
return outbox.publish(b"home/announce", b"someone is at the door")
}
}
fn main() -> i32 {
return broker.new()
.on(b"home/doorbell", Doorbell { rings: 0 })
.password(b"family", b"secret")
.run(b"0.0.0.0:1883")
}
Messages published by handlers are not passed to other broker handlers, so
broker.print() does not show them.
| Builder method | Effect |
|---|---|
.on(filter, handler) |
Call handler for messages clients publish to matching topics. |
.password(username, password) |
Accept only clients that log in with these credentials. |
.authenticate(auth) |
Decide yourself: auth.allow(client_id, username, password) -> bool. |
.max_connections(count) |
Stop once this many connections have come and gone. |
.build() |
Check the setup and get a Broker for tests. |
.run(address), .serve(address), .serve_until(address, cancel) |
As for apps. |
Refused clients receive CONNACK return code 5 (not authorized). Test handlers and authenticators without a network, too:
package broker_test
import "std/mqtt"
import "std/mqtt/broker"
@test
fn doorbell_announces_visitors() {
server := broker.new()
.on(b"home/doorbell", mqtt.reply(b"home/announce", b"visitor"))
.password(b"family", b"secret")
.build()!
server.publish(b"home/doorbell", b"ring").expect_handled(1).expect_publish(b"home/announce", b"visitor")
assert(server.allows(b"phone", b"family", b"secret"))
assert(!server.allows(b"phone", b"family", b"guess"))
}
The broker is one thread that keeps everything in fixed memory, so it has limits. The defaults suit a household of devices:
| Limit | Default | What happens beyond it |
|---|---|---|
| Simultaneous clients | broker.CLIENTS (16) |
New connections wait until one leaves. |
| Packet size | broker.PACKET_BYTES (4096) |
The client is disconnected. |
| Subscriptions per client | broker.FILTER_BYTES (1024 bytes of filters) |
SUBACK refuses the filter (128). |
| Client ID length | broker.ID_BYTES (128) |
CONNACK refuses the client (2). |
| Will topic plus payload | broker.WILL_BYTES (512) |
CONNACK refuses the client (3). |
| All retained messages | broker.RETAINED_BYTES (16384) |
The new message is delivered but not retained. |
Everything lives in memory: subscriptions, retained messages, and sessions
are lost when the broker stops, and a client asking for a persistent session
always starts a new one. A client that does not read for
config.write_timeout_ms (5 seconds) is disconnected so it cannot stall
everyone else. serve returns a broker.Report with counts of accepted,
connected, and refused connections, messages received, copies
delivered, handler failed, and dropped connections.
The step-by-step client
std/mqtt/hosted is the layer under app: you call connect, publish,
subscribe, and receive yourself, in your own loop. Use it for short
scripts, or when a program decides itself when to talk to the broker. This
program subscribes to lights/#, publishes to lights/hall, and prints the
message the broker sends back:
package lights
import "std/console"
import "std/mqtt"
import "std/mqtt/hosted"
fn main() {
workspace := [0u8; hosted.WORKSPACE_BYTES]
client := hosted.Client.connect(b"mqtt://localhost", b"dodo-lights", &mut workspace)!
client.subscribe(b"lights/#", mqtt.QoS.AtLeastOnce)!
client.publish(b"lights/hall", b"on")!
message := client.receive()!
out := console.stdout()
out.write(message.topic)!
out.write(b" = ")!
out.write(message.payload)!
out.write(b"\n")!
client.disconnect()!
}
It prints lights/hall = on.
URLs have the form mqtt://host[:port]; the port defaults to 1883 and IPv6
literals use brackets (mqtt://[::1]:1883). The client ID names the session on
the broker. Two clients connected with the same ID disconnect each other.
Storage
The client keeps no heap memory. Every packet lives in the workspace you pass to
connect, which the client borrows until it is dropped:
- The first quarter stages outgoing packets. A larger payload is sent straight from your slice after the packet header.
- The remaining three quarters buffer incoming packets. The largest message
you can receive is the size of this buffer, minus a few header bytes. A
larger one fails with
PacketTooLargeand ends the connection.
hosted.WORKSPACE_BYTES (16 KiB) receives messages up to about 12 KiB. Pass a
larger array for bigger messages. The minimum is hosted.MIN_WORKSPACE_BYTES.
Publish
publish(topic, payload) sends a QoS 0 message. publish_with(topic, payload, qos, retain) chooses the quality of service. It returns when the broker has
done what that QoS promises:
| QoS | mqtt.QoS |
Delivery | publish_with returns after |
|---|---|---|---|
| 0 | AtMostOnce |
Fire and forget; may be lost | the packet is written |
| 1 | AtLeastOnce |
Retried until acknowledged; may repeat | PUBACK |
| 2 | ExactlyOnce |
Four-step handshake | PUBREC, PUBREL, PUBCOMP |
With retain set, the broker stores the message as the topic’s last known
value and gives it to future subscribers immediately. Publishing an empty
retained payload clears it.
Topic names cannot be empty or contain the + or # wildcards. An invalid
topic fails with InvalidTopic before anything is sent, and the client stays
usable.
Subscribe and receive
subscribe(filter, qos) returns the QoS the broker granted, which can be lower
than the one you asked for. Messages on a subscription arrive at no more than
the granted QoS. A broker that refuses the filter makes subscribe fail with
Rejected. The connection stays open. unsubscribe(filter) removes a
subscription.
receive() blocks until the next message arrives. poll(timeout_ms) waits at
most timeout_ms and returns none if nothing arrived; poll(0) only takes a
message that is already buffered.
fn drain(client: &mut hosted.Client) -> void!mqtt.Error {
for true {
match client.poll(1000)? {
some(message) => {
handle(message.topic, message.payload)
}
none => {
return ok()
}
}
}
return ok()
}
An mqtt.Message has topic, payload, qos, retain (a stored value sent on
subscription), and duplicate (the broker may have sent it before). Its views
borrow the client’s workspace, so copy anything you need to keep before calling
the client again.
Acknowledgement happens on your next call. A QoS 1 or 2 message is
acknowledged when you next call receive, poll, publish, subscribe, or
another client method. If your program stops while handling a message, the
broker still considers it undelivered.
Messages that arrive while publish_with, subscribe or unsubscribe waits
for its acknowledgement are kept in order for the next receive. They count
against the incoming buffer. If too many pile up, the call fails with
BufferFull.
Options and keep-alive
hosted.Options.new(client_id) gives the defaults; change fields and pass the
value to Client.connect_with:
options := hosted.Options.new(b"thermostat-1")
options.username = b"thermostat"
options.password = password
options.keep_alive_s = 30
options.will_topic = b"devices/thermostat-1/status"
options.will_payload = b"offline"
options.will_retain = true
client := hosted.Client.connect_with(b"mqtt://broker.local", &options, &mut workspace)?
| Field | Default | Meaning |
|---|---|---|
username, password |
empty (absent) | Credentials. A password needs a username. |
keep_alive_s |
60 | Idle seconds before the client pings the broker. 0 disables pings. |
clean_session |
true |
false asks the broker to keep subscriptions and queued messages for this client ID between connections. session_present() reports whether it did. |
will_topic, will_payload, will_qos, will_retain |
no will | Message the broker publishes if the connection ends without disconnect(). |
connect_timeout_ms |
10000 | Resolution, TCP connect, and CONNACK together. |
timeout_ms |
10000 | Each write, and each wait for an acknowledgement. |
The client sends keep-alive pings itself, but only while one of its methods is
running. While waiting for messages, call poll or receive at least once per
keep-alive interval. If the broker does not answer a ping within the interval,
the call fails with TimedOut.
disconnect() ends the session cleanly, so the broker discards the will
message. Dropping the client closes the socket without DISCONNECT, which
publishes the will.
Errors
Every operation returns mqtt.Error, with a kind and a numeric code:
kind |
Cause | code |
|---|---|---|
InvalidInput, InvalidTopic, InvalidFilter |
Rejected before sending; the client stays usable. | 0 |
Refused |
The broker refused CONNECT. | CONNACK return code, such as mqtt.BAD_CREDENTIALS (4) or mqtt.NOT_AUTHORIZED (5) |
Rejected |
The broker refused a subscription; the client stays usable. | mqtt.SUBSCRIBE_FAILURE (128) |
Resolve, Connect, Network |
DNS, TCP connection, or socket failure. | Operating-system code |
TimedOut |
A deadline passed, or the broker stopped answering pings. | 0 or system code |
Closed |
The broker closed the connection, or the client was disconnected. | 0 or system code |
Malformed, Protocol, PacketTooLarge, BufferFull |
The broker sent invalid or unexpected data, or more than the workspace holds. | 0 |
Apart from the cases marked usable, an error ends the session, and later calls
fail with Closed. The client does not reconnect by itself. To recover, drop it
and connect again with the same workspace.
Topics
Topic levels are separated by /. In filters, + matches exactly one level,
and #, which must be the last level, matches that level and everything below
it. Wildcards in the first level never match topics starting with $, which
brokers use for their own statistics.
package topic_example
import "std/mqtt"
@test
fn filters_select_topics() {
assert(mqtt.matches(b"sensors/+/temperature", b"sensors/kitchen/temperature"))
assert(mqtt.matches(b"sensors/#", b"sensors"))
assert(!mqtt.matches(b"sensors/+", b"sensors/kitchen/temperature"))
assert(!mqtt.matches(b"#", b"$SYS/broker/uptime"))
assert(mqtt.valid_filter(b"sensors/+/temperature"))
assert(!mqtt.valid_filter(b"sensors/#/temperature"))
assert(!mqtt.valid_topic(b"sensors/+"))
}
valid_topic, valid_filter, and valid_string check the MQTT rules: UTF-8
without U+0000, at most 65535 bytes, and wildcards only where allowed.
Packets for other transports
On targets without the hosted client, use std/mqtt directly over any byte
stream. Each encode_* function writes one complete packet into your buffer
and returns its length: encode_connect (from an mqtt.Connect value),
encode_publish, encode_publish_head (everything except the payload),
encode_subscribe, encode_unsubscribe, encode_ack (PUBACK, PUBREC, PUBREL,
PUBCOMP, UNSUBACK), encode_pingreq, and encode_disconnect. The server side has encode_connack, encode_suback,
and encode_pingresp, plus decode_connect, subscribe_entry, and
unsubscribe_entry for reading client requests.
For input, frame_length(bytes) returns none until the fixed header is
complete, then the total packet length. Once that many bytes have arrived,
decode returns a Packet whose topic and payload borrow the buffer.
package codec_example
import "core/bytes"
import "std/mqtt"
@test
fn round_trips_a_publish() {
frame := [0u8; 64]
length := mqtt.encode_publish(&mut frame, b"lights/hall", b"on", mqtt.QoS.AtLeastOnce, false, false, 1)!
packet := mqtt.decode(&frame[..length])!
assert(packet.kind == mqtt.PacketKind.Publish)
assert_eq(packet.length, length)
assert_eq(packet.packet_id, 1u16)
assert(bytes.equal(packet.topic, b"lights/hall"))
assert(bytes.equal(packet.payload, b"on"))
}
Limits
- MQTT 3.1.1 only, for clients and the broker. MQTT 5 properties and reason codes are not supported; every common broker and client also speaks 3.1.1.
- Plain TCP only.
mqtts://(TLS) and WebSocket transports are not built in yet, so keep the broker on a trusted network. hosted.Clienthas one acknowledged operation in flight at a time and does not reconnect;appreconnects for you.- In-flight QoS 1 and 2 state is kept only for the current connection. After a reconnect a message may arrive again, so make handlers safe to repeat.