Event Format

CloudEvents is an upcoming CNCF standard for describing events in a consistent and portable way. We believe that CloudEvents brings benefits at a very low adoption cost. After all, any custom-developed event envelope-format likely contains the same properties anyway; CloudEvents mostly just make sure they are named the same way.

We aim at following the CloudEvents specification as closely as possible. This includes the event format itself, but also the way events are serialized on different transport, the so-called transport bindings.

Example event

{
  "specversion": "0.2",
  "type": "com.github.pull.create",
  "source": "https://github.com/cloudevents/spec/pull/123",
  "id": "A234-1234-1234",
  "time": "2018-04-05T17:31:00Z",
  "comexampleextension1": "value",
  "comexampleextension2": {
    "othervalue": 5
  },
  "contenttype": "text/xml",
  "data": "<much wow=\"xml\"/>"
}

Encoding

In addition to JSON encoding, RIG also supports publishing CloudEvents using the Apache Avro format. With Confluent recommending the use of Avro in combination with their Kafka distribution, Avro sees much adoption lately, especially when used alongside the Confluent Schema Registry (open source under the Confluent Community License). Consequently, RIG primarily supports the "registry model", where the message schema is not sent alongside every message; instead, each message only contains a schema ID and the schema itself is looked up (and cached) at runtime, using the Schema Registry.

To enable Avro & Schema Registry support, checkout the Operator's Guide and look for the environment variable KAFKA_SCHEMA_REGISTRY_HOST.

For further details and examples, checkout the dedicated Section on Avro.

Transport Bindings

When sending or receiving an event, you have a couple of options:

  • Send the event as-is, or send only the "data" (= payload) in the "body" and the rest (the so-called "context attributes") in headers. The former is called structured transport mode, while the latter is known as binary transport mode.
  • When using structured transport mode, the event may be encoded using JSON or Avro.
  • Regardless of transport mode, the event's payload ("data" field or body) may be encoded in a format different from the event's encoding. For example, you might want to send CloudEvents in JSON format, but have their "data" field encoded using Protobuf.

Next we look at the HTTP and Kafka transport bindings in more detail.

HTTP Transport Binding

RIG implements HTTP Transport Binding for CloudEvents v0.2, with the exception of batched mode. The two supported modes of operation, structured and binary, are described below.

Structured

In structured mode the event is encoded in full and sent in the request body. The content-type HTTP header is expected to be application/cloudevents+json (application/cloudevents+avro might be supported in future versions of RIG).

While the specification defines that only the content type should be used to determine the transport mode, RIG also accepts messages with content type application/json as structured if, and only if, there is no ce-specversion HTTP header present in the request.

Example HTTP request with event in structured mode

HTTP header that announces a JSON-encoded CloudEvent:

Content-Type: application/cloudevents+json; charset=UTF-8

Request body:

{
  "specversion": "0.2",
  "type": "com.example.someevent",
  "source": "example",
  "id": "80dc037c-fb24-43e9-9759-94f91f310a4b1",
  "data": {
    "this is": "the payload"
  }
}

Binary

In binary mode the request body only contains the data value of the corresponding CloudEvent. The context attributes - i.e., all other fields - are moved into the HTTP header (this means also extensions). The data/body encoding is determined by the content-type header. At the time of writing there are two content types supported: application/json and avro/binary.

Same example event, sent using HTTP request in binary mode

In binary mode the HTTP header contains all context attributes. It also announces the body encoding:

ce-specversion: 0.2
ce-type: com.example.someevent
ce-source: example
ce-id: 80dc037c-fb24-43e9-9759-94f91f310a4b1
Content-Type: application/json; charset=UTF-8

Request body:

{
  "this is": "the payload"
}

Kafka Transport Binding

Implemented using Kafka Transport Binding for CloudEvents v1.0. We utilize Kafka headers that have been introduced in Kafka version 0.11. In order to support older Kafka versions as well, RIG defaults to structured mode and does not require any headers at all (see below).

Like with the HTTP transport binding, we define two modes of operation: structured and binary.

Structured

In structured mode the event is encoded in full and sent as the message body. Structured mode is determined by parsing the content-type header, which defaults to application/cloudevents+json.

The default value means that the body contains a CloudEvents-formatted event in JSON encoding. The related content type for Avro encoding is application/cloudevents+avro.

Example event in structured mode

Message header that announces a JSON-encoded CloudEvent:

Content-Type: application/cloudevents+json; charset=UTF-8

Message body:

{
  "specversion": "0.2",
  "type": "com.example.someevent",
  "source": "example",
  "id": "80dc037c-fb24-43e9-9759-94f91f310a4b1",
  "data": {
    "this is": "the payload"
  }
}

Binary

In binary mode the message body only contains the data value of the corresponding CloudEvent. The context attributes - i.e., all other fields - are moved into the message header (this means also extensions). The data/body encoding is determined by the content-type header. In this mode there is no default for content-type and RIG rejects messages that come without it. At the time of writing there are two content types supported: application/json and avro/binary.

Same example in binary mode

In binary mode the message header contains all context attributes. It also announces the body encoding:

ce_specversion: 0.2
ce_type: com.example.someevent
ce_source: example
ce_id: 80dc037c-fb24-43e9-9759-94f91f310a4b1
Content-Type: application/json; charset=UTF-8

Message body:

{
  "this is": "the payload"
}

← FeaturesPublishing Events →