Skip to main content
Version: 4.0

Inter-Protocol Property Conversions

Overview

RabbitMQ is a multi-protocol broker supporting:

Messages can readily be published by one protocol and consumed by others. This necessitates data format conversions between different protocol formats. Prior to RabbitMQ 3.13 all messages were converted into an internal format based on the AMQP 0.9.1 protocol when published to RabbitMQ. This approach often resulted in unnecessary conversions and/or data fidelity issues.

In current RabbitMQ versions, the approach has changed for the AMQP, AMQP 0.9.1 and MQTT protocols in that messages will always be stored in their original format and only converted if consumed by a protocol different from the originating protocol. This has the advantage that no protocol specific information is lost when the originating and the consuming protocols are the same.

The only exception to this rule are streams: No matter which protocol published, streams internally store their messages as AMQP encoded data. Hence, when for example publishing using AMQP 0.9.1 a conversion to AMQP takes place.

All conversions have been re-written and formalised. This document aims to capture the conversions rules that are implemented so that application developers can look back at this to understand how their messages will be transformed in their multi-protocol messaging applications.

Conventions

shortstr: (condition) a string that is smaller than 256 bytes and contains only valid UTF-8 encoded data with no NUL (binary zero octet) characters.

*: is read as "any" (field/type etc).

Conditions can be read to be evaluated in the order they are written. For example an AMQP message_id of type utf8 with more than 255 bytes will fail the first shortstr condition and then fall through to the next line. If there is no matching line then the data will not be converted (and therefore not be available to the consuming application).

AMQP 1.0 -> AMQP 0.9.1

A conversion from AMQP to AMQP 0.9.1 takes place if the message is consumed with AMQP 0.9.1 and the message either

  • was published with AMQP 1.0, or
  • is consumed from a stream.
AMQP sectionAMQP fieldAMQP typeConditionAMQP 0.9.1 sectionAMQP 0.9.1 FieldAMQP 0.9.1 TypeComment
headerdurablebooleanpropertiesdelivery_modeubyte2 = durable, 1 = transient
headerpriorityubytepropertiespriorityubyte
headerttlmilliseconds (uint)propertiesexpirationshortstrmilliseconds converted to string value
propertiesmessage_idutf8shortstrpropertiesmessage_idshortstr
propertiesmessage_idutf8> 256 bytespropertiesheaders, Key: x-message-idlongstr
propertiesmessage_iduuid (16 byte binary)propertiesmessage_idshortstrconverted to text e.g. "urn:uuid:550e8400-e29b-41d4-a716-446655440000"
propertiesmessage_idulongpropertiesmessage_idshortstrconverted to text
propertiesmessage_idbinarypropertiesheaders, Key: x-message-idbinary
propertiesuser_idbinaryshortstrpropertiesuser_idshortstr
propertiestoaddress
propertiessubjectutf8
propertiesreply_toaddressshortstrpropertiesreply_toshortstr
propertiescorrelation_idutf8shortstrpropertiescorrelation_idshortstr
propertiescorrelation_idutf8> 256 bytespropertiesheaders, Key: x-correlation-idlongstr
propertiescorrelation_iduuid (16 byte binary)propertiescorrelation_idshortstrconverted to text representation, eg. "urn:uuid:550e8400-e29b-41d4-a716-446655440000"
propertiescorrelation_idulongpropertiescorrelation_idshortstrConverted to text representation of number
propertiescorrelation_idbinarypropertiesheaders, Key: x-correlation-idbinary
propertiescontent_typesymbolpropertiescontent_typeshortstr
propertiescontent_encodingsymbolpropertiescontent_encodingshortstr
propertiesabsolute_expiry_timetimestamp
propertiescreation_timetimestamppropertiestimestamptimestampconverted to seconds
propertiesgroup_idutf8shortstrpropertiesapp_idshortstr
propertiesgroup_sequencesequence-no
propertiesreply_to_group_idutf8
application properties** (see type conversions)Key: shortstrpropertiesheaders*
message annotations* (symbol - x-cc*)Key: x-ccpropertiesheaders, Key: "CC"longstr
message annotations* (symbol - x-*)* (see type conversions)Key: shortstr & x-*propertiesheaders*Typically this means x- headers
datadatasingle data sectionpayloadbinaryplain binary extracted from data section
data (multiple)**multiple data sectionspayloadAMQP 1.0 encoded binaryproperties.type will be set to "amqp-1.0"
amqp.value**payloadAMQP 1.0 encoded binaryproperties.type will be set to "amqp-1.0"
amqp.sequence**payloadAMQP 1.0 encoded binaryproperties.type will be set to "amqp-1.0"

Type Conversions

AMQP 1.0 typeConditionAMQP 0.9.1 typeComment
utf8 (string)longstrRabbitMQ does not support shortstr header values so all utf8 inputs are converted to longstr unless it is a field in the basic properties
binarybinary
longlong
ulonglongrisk of overflow
ubyteunsignedbyte
shortshort
ushortusignedshort
uintunsignedint
intsignedint
doubledouble
floatfloat
booleanbool
timestamptimestamp (seconds)value divided by 1000
bytebyte
nullvoid
listarray
maptable
symbollongstr

AMQP 0.9.1 -> AMQP 1.0

AMQP 0.9.1 sectionAMQP 0.9.1 fieldAMQP 0.9.1 typeConditionAMQP 1.0 sectionAMQP 1.0 FieldAMQP 1.0 TypeComment
basic propertiesmessage_idshortstrvalid urn uuidpropertiesmessage_iduuid
basic propertiesmessage_idshortstrpropertiesmessage_idutf8
basic propertiescorrelation_idshortstrvalid urn uuidpropertiescorrelation_iduuid
basic propertiescorrelation_idshortstrpropertiescorrelation_idutf8
basic propertiesuser_idshortstrpropertiesuser_idbinary
basic propertiesexpirationshortstrif convertible to numeric typeheaderttluint
basic propertiestypeshortstrmessage annotationsx-basic-typeutf8
basic propertiesreply_toshortstrpropertiesreply_toutf8
basic propertiesapp_idshortstrpropertiesgroup-idutf8
basic propertiestimestamptimestamp (seconds)propertiescreation_timetimestampconverted to ms
basic propertiescontent-typeshortstrpropertiescontent-typesymbol
basic propertiescontent-encodingshortstrpropertiescontent-encodingsymbol
basic propertiesdelivery-modeoctetheaderdurablebooleanpreviously in x-basic-delivery-mode
basic propertiespriorityoctetheaderpriorityubytepreviously in x-basic-priority
basic.propertiesheadersKey=x-amqp-1.0-propertiesproperties--legacy 1.0 plugin headers
basic.propertiesheadersKey=x-amqp-1.0-application-propertiesapplication properties--legacy 1.0 plugin headers
basic.propertiesheadersKey=x-amqp-1.0-message-annotationsmessage annotations--legacy 1.0 plugin headers
basic.propertiesheadersKey=x-reply-to-topicpropertiesreply_toutf8"/topic/" RK
basic propertiesheaders*Key begins with "x-"message annotationsKey*see type conversion rules
basic propertiesheaderstableValue not array or tableapplication propertiesKey*see type conversion rules
basic propertiesheaderstableValue is array or table---not converted
payload-binarydata-data

Type Conversions

AMQP 0.9.1 typeConditionAMQP 1.0 typeComment
longstrshortstrutf8Performance / accuracy trade-off
longstrbinary
longlong
ubyteubyte
shortshort
ushortushort
uintuint
intint
doubledouble
floatfloat
boolboolean
binarybinary
timestamptimestampConverted from seconds to milliseconds
bytebyte
voidnull
arraylist
tablemap
table.keyx-* headersymbol
table.keyutf8
table.value*converted according to this table

MQTT 5.0 -> AMQP 1.0

MQTT 5.0 SectionMQTT FieldMQTT 5.0 TypeConditionAMQP 1.0 SectionAMQP 1.0 FieldAMQP 1.0 TypeComment
Fixed HeaderDupBitsSetting to header first-acquirer does not make sense because the DUP flag only applies from client to server (consumption from server to client is determined by the Redelivered flag sent from the queues)
Fixed HeaderQoSBitsheaderdurablebooleandurable is true if QoS > 0
Fixed HeaderRetainBits
Variable HeaderPayload Format IndicatorBitssee row "Payload" with condition "Payload Format Indicator set"
Variable HeaderMessage Expiry Intervaluintheaderttlmillisecondsseconds to milliseconds
Variable HeaderTopic Aliasushort
Variable HeaderResponse Topicutf8propertiesreply_toutf8Translate MQTT topic to AMQP routing key (RK). Set reply-to address to "/exchange/" X "/" RK.
Variable HeaderCorrelation Databinaryurn:uuidpropertiescorrelation_iduuid
Variable HeaderCorrelation Databinarypropertiescorrelation_idbinary
Variable HeaderUser Propertyutf8Key begins with "x-" and key is ASCIImessage annotationvalue: utf8Key is of type symbol
Key does not begin with "x-"application propertiesvalue: utf8Key is of type utf8
Variable HeaderSubscription Identifieruint
Variable HeaderContent Typeutf8valid ASCIIpropertiescontent_typesymbolMQTT content type is UTF-8 whereas AMQP 1.0 content type is only ASCII
PayloadPayload Format Indicator unsetdatadata
PayloadPayload Format Indicator setamqp.valueutf8If the Payload Format Indicator is set, convert the MQTT payload to a string (i.e. single AMQP value section) because an AMQP string is UTF-8 encoded

Type Conversions

MQTT 5.0 TypeConditionAMQP 1.0 TypeComment
Bitsbooleanonly selected flags are converted
ushortushort
uintuint
utf8utf8
binary datautf8utf8
binary databinary
utf8 string pairsmapde-duplicate keys, but maintains order

MQTT 5.0 -> AMQP 0.9.1

MQTT 5.0 SectionMQTT 5.0 FieldMQTT 5.0 TypeConditionAMQP 0.9.1 SectionAMQP 0.9.1 FieldAMQP 0.9.1 TypeComment
Fixed HeaderDupBitsSetting to basic.deliver field redelivered does not make sense because the DUP flag only applies from client to server (consumption from server to client is determined by the Redelivered flag sent from the queues)
Fixed HeaderQoSBitspropertiesdelivery_modeQoS 0 maps to delivery_mode 1 QoS 1 maps to delivery_mode 2
Fixed HeaderRetainBits
Variable HeaderPayload Format IndicatorBitsAMQP 0.9.1 content_encoding refers to a MIME content encoding whose valid values are listed in https://www.iana.org/assignments/http-parameters/http-parameters.xml Therefore AMQP 0.9.1 content_encoding is something different. It would best map to text/<subtype>; charset="utf-8 of content_type, but the <subtype> is not defined. Furthermore MQTT 5.0 property Content Type is already defined.
Variable HeaderMessage Expiry Intervaluintpropertiesexpirationshortstrseconds to milliseconds
Variable HeaderTopic Aliasushort
Variable HeaderResponse Topicutf8propertiesheaders, Key: x-reply-to-topiclongstrAMQP 0.9.1 property reply_to refers to a queue name, not to a topic name
Variable HeaderCorrelation Databinaryshortstrpropertiescorrelation_idshortstr
propertiesheaders, Key: x-correlation-idlongstr
Variable HeaderUser Propertyutf8 string pairname is shortstrpropertiesheaderslongstrRabbitMQ cannot have shortstr as e.g. header values.
Variable HeaderSubscription Identifiervariable byte integer
Variable HeaderContent Typeutf8shortstrpropertiescontent_typeshortstr
Payloadpayload

Type Conversions

MQTT 5.0 TypeConditionAMQP 0.9.1 TypeComment
BitsBits
ushortushort
uintuint
utf8< 256 bytes AND target is not headershortstrRabbitMQ cannot have shortstr as e.g. header values.
utf8>= 256 byteslongstr
binary< 256 bytes of UTF-8 encoded datashortstr
binaryno valid UTF-8 or >= 256 byteslongstr
utf8 pairstable"Duplicate fields are illegal" in AMQP 0.9.1 Field Tables. RabbitMQ sorts the fields by key.

AMQP 1.0 -> MQTT 5.0

AMQP 1.0 sectionAMQP 1.0 fieldAMQP 1.0 typeConditionMQTT 5.0 SectionMQTT 5.0 FieldMQTT 5.0 TypeComment
headerdurablebooleanFixed HeaderQoSBitsdurable=true maps to QoS 1 durable=false maps to QoS 0
propertiesmessage_id*
propertiesuser_idbinary
propertiestoaddress
propertiessubjectutf8
propertiesreply_toaddress"/exchange/" X "/" RK where X matches mqtt.exchangeVariable HeaderResponse Topicutf8Translate AMQP topic to MQTT topic
propertiescorrelation_id*Variable HeaderCorrelation Databinaryconvert to binary
propertiescontent_typesymbolVariable HeaderContent Typeutf8
propertiescontent_encodingsymbol
propertiesabsolute_expiry_timetimestamp
propertiescreation_timetimestamp
propertiesgroup_idutf8
propertiesgroup_sequencesequence-no
propertiesreply_to_group_idutf8
application properties**not array, list, map or binaryUser Propertiesutf8Values converted to string representation
message annotations**x- headers only, excluding those with array, list, map or binary valuesUser Propertiesutf8
datadataone or more data sectionsPayloadbinary
data**single AMQP binary value sectionPayloadbinary
dataamqp.value(utf8)single AMQP value section convertible to UTF-8PayloadbinaryPayload-Format-Indicator is set to 1 such that MQTT 5.0 clients can parse the text representation of the message and therefore understand the message.
data*other single AMQP value section or amqp-sequence sectionsPayloadbinaryEncode with the AMQP type system and include a Content Type message/vnd.rabbitmq.amqp. This Content Type is not registered.

AMQP 0.9.1 -> MQTT 5.0

AMQP 0.9.1 sectionAMQP 0.9.1 fieldAMQP 0.9.1 typeConditionMQTT 5.0 SectionMQTT 5.0 FieldMQTT 5.0 TypeComment
basic propertiesmessage_idshortstr
basic propertiescorrelation_idshortstrVariable HeaderCorrelation Databinary
basic propertiesuser_idshortstr
basic propertiesexpirationshortstrVariable HeaderMessage Expiry Intervaluintconvert from milliseconds to seconds
basic propertiestypeshortstr
basic propertiesreply_toshortstr
basic propertiesapp_idshortstr
basic propertiestimestamptimestamp (seconds)
basic propertiescontent_typeshortstrVariable HeaderContent Typeutf8
basic propertiescontent_encodingshortstr
basic propertiesdelivery_modeoctetFixed HeaderQoSBitsdelivery-mode 2 maps to QoS 1 delivery-mode 1 maps to QoS 0
basic propertiespriorityoctet
basic.propertiesheaderslongstrKey=x-reply-to-topicVariable HeaderResponse Topicutf8Translate AMQP topic to MQTT topic
basic.propertiesheaderslongstrKey=x-correlation-idVariable HeaderCorrelation Databinary
basic.propertiesheaders*not array, tableUser propertiesutf8Simple values are converted into their string representation
basic propertiesheaderstable
payloadbinaryPayload