Forwarding Requests
RIG includes a configurable, distributed HTTP reverse proxy. Depending on the configuration, RIG forwards incoming HTTP requests to backend services, to a Kafka topic or to a Kinesis stream, then waits for the reply and forwards that reply to the original caller.
API Endpoint Configuration
The configuration should be passed at startup. Additionally, RIG provides an API to add, change or remove routes at runtime. These changes are replicated throughout the cluster, but they are not persisted; that is, if all RIG nodes are shut down, any changes to the proxy configuration are lost. Check out the API Gateway Synchronization to learn more.
To pass the configuration at startup, RIG uses an environment variable called PROXY_CONFIG_FILE
. This variable can be used to either pass the path to an existing JSON file, or to directly pass the configuration as a JSON string. Let's configure a simple endpoint to show how this works.
The configuration JSON (file) holds a list of API definitions. Refer to the API Gateway Management for details. You can also utilize small playground in examples
We define an endpoint configuration like this:
[
{
"id": "my-service",
"version_data": {
"default": {
"endpoints": [
{
"id": "my-endpoint",
"method": "GET",
"path_regex": "/"
}
]
}
},
"proxy": {
"use_env": true,
"target_url": "API_HOST",
"port": 3000
}
}
]
This defines a single service called "my-service". The URL is read from an given environment variable in this case (use_env: true
). Because we want to run RIG inside a Docker container, we cannot use localhost
. Instead, we can use host.docker.internal
within the container to refer to the Docker host. This way, the service URL is resolved to http://host.docker.internal:3000
. The service has one endpoint called "my-endpoint" at path /
, which forwards GET
requests to the same path (http://host.docker.internal:3000/
).
As a demo service, we use a small Node.js script:
const http = require("http");
const port = 3000;
const handler = (_req, res) => res.end("Hi, I'm a demo service!\n");
const server = http.createServer(handler);
server.listen(port, (err) => {
if (err) {
return console.error(err);
}
console.log(`server is listening on ${port}`);
});
Using Docker, our configuration can be put into a file and mounted into the container. Also, we set API_HOST
to the Docker host URL as mentioned above. On Linux or Mac, this looks like this:
$ cat <<EOF >config.json
<paste the configuration from above>
EOF
$ docker run -d \
-e API_HOST=http://host.docker.internal \
-v "$(pwd)"/config.json:/config.json \
-e PROXY_CONFIG_FILE=/config.json \
-p 4000:4000 -p 4010:4010 \
accenture/reactive-interaction-gateway
After that we should be able to reach our small demo service through RIG:
$ curl \
-H 'traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01' \
-H 'tracestate: hello=tracing' \
localhost:4000
Hi, I'm a demo service!
Alternatively, instead of using a file we can also pass the configuration directly:
$ config="$(cat config.json)"
$ docker run \
-e API_HOST=http://host.docker.internal \
-e PROXY_CONFIG_FILE="$config" \
-p 4000:4000 -p 4010:4010 \
accenture/reactive-interaction-gateway
Note that this way we don't need a Docker volume, which might work better in your environment. Again, we should be able to reach the demo service:
$ curl \
-H 'traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01' \
-H 'tracestate: hello=tracing' \
localhost:4000
Hi, I'm a demo service!
Dynamic URL parameters
It's a common case that you want to fetch detail for some entity e.g. /books/123
. To make sure the dynamic value 123
is correctly matched and forwarded API endpoint can be configured as a regular expression:
[{
"id": "my-service",
"version_data": {
"default": {
"endpoints": [{
"id": "my-detail-endpoint",
"method": "GET",
"path_regex": "/books/(.+)"
}]
}
},
...
}]
Publishing to event streams
Instead of forwarding an HTTP request to an internal HTTP endpoint, RIG can also produce an event to a Kafka topic (or Kinesis stream). What looks like a standard HTTP call to the frontend, actually produces an event for backend services to consume.
Depending on the use case, the request may either return immediately or after a response has been produced to another Kafka topic (or Kinesis stream), as described below.
For fire-and-forget style requests, the endpoint configuration looks like this:
[{
...
"version_data": {
"default": {
"endpoints": [{
"id": "my-endpoint",
"method": "POST",
"path_regex": "/",
"target": "kafka",
"topic": "my-topic",
"schema": "my-avro-schema"
}]
}
},
...
}]
Note that the target
field is set to kafka
(for Kinesis use kinesis
). The topic
field is mandatory, but the schema
field is optional. Note that the topic
and schema
fields are just about publishing to event stream and have nothing to do with events consumption.
The endpoint expects the following request format:
{
"id": "069711bf-3946-4661-984f-c667657b8d85",
"type": "com.example",
"time": "2018-04-05T17:31:00Z",
"specversion": "0.2",
"source": "/cli",
"contenttype": "application/json",
"rig": {
"target_partition": "the-partition-key"
},
"data": {
"foo": "bar"
}
}
target_partition
is optional, if not set -- RIG produces event to random Kafka/Kinesis partition.
Wait for response
Sometimes it makes sense to provide a simple request-response API to something that runs asynchronously on the backend. For example, let's say there's a ticket reservation process that takes 10 seconds in total and involves three different services that communicate via message passing. For an external client, it may be simpler to wait 10 seconds for the response instead of polling for a response every other second.
A behavior like this can be configured using an endpoints' response_from
property. When set to kafka
, the response to the request is not necessarily taken from the target
, e.g., for target
= http
this means the backend's HTTP response might be ignored - it's the responsibility of the backend service where to read the response from: If the backend returns with the HTTP code 202 Accepted
, the response will be read from a Kafka topic. If the backend returns a different HTTP code (can be 200
or 400
or whatever makes sense), only then the response will be read synchronously from the http target directly (this allows the backend to return cached responses). In order to enable RIG to correlate the response from the kafka topic with the original request, RIG adds a correlation ID to the request (using a query parameter in case of target
= http
, or baked into the produced CloudEvent otherwise). Backend services that work with the request need to include that correlation ID in their response; otherwise, RIG won't be able to forward it to the client (and times out).
Configuration of such API endpoint might look like this:
[{
...
"version_data": {
"default": {
"endpoints": [{
"id": "my-endpoint",
"method": "POST",
"path_regex": "/",
"target": "kafka",
"topic": "my-topic",
"response_from": "kafka"
}]
}
},
...
}]
Note the presence of
response_from
field. This tells RIG to wait for different event with the same correlation ID.
target
-> response_from
)
Supported combinations (- HTTP ->
kafka
/http_async
/kinesis
- Kafka ->
kafka
- Kinesis -> not supported
- Nats ->
nats
http_async
means that correlated response has to be sent to internal :4010/v3/responses
POST
endpoint.
Supported formats
All response_from
options support only binary mode.
Message headers:
rig-correlation: "correlation_id_sent_by_rig"
rig-response-code: "201"
content-type: "application/json"
All headers are required.
Message body:
{
"foo": "bar"
}
Auth
RIG can do simple auth check for endpoints. Currently supports JWT.
API configuration is following:
[{
"id": "my-service",
"auth_type": "jwt",
"auth": {
"use_header": true,
"header_name": "Authorization",
"use_query": false,
"query_name": ""
},
"version_data": {
"default": {
"endpoints": [{
"id": "my-unsecured-endpoint",
"method": "GET",
"path_regex": "/unsecured"
},{
"id": "my-secured-endpoint",
"method": "GET",
"path_regex": "/secured",
"secured": true
}]
}
},
...
}]
Important blocks are auth_type
and auth
. auth_type
sets which auth mechanism to use -- currently jwt
or none
. auth
sets where to find token that should be used. It's possible to send token in 2 places -- HTTP headers (use_header
) and as URL query parameter (use_query
). header_name
and query_name
define lookup key in headers/query. You can use headers and query at the same time.
Once you set how to use auth, you can simply define which API endpoint should be secured via secured
property. Auth check is by default disabled and secured
field set to false
.
Make sure to use
Bearer ...
form as a value for auth header.
Headers transformations
Headers transformations are supported in a very simple way. Assume following API configuration:
[{
"id": "my-service",
"version_data": {
"default": {
"transform_request_headers": {
"add_headers": {
"host": "my-very-different-host.com",
"custom-header": "custom-value"
}
},
"endpoints": [{
"id": "my-endpoint",
"method": "GET",
"path_regex": "/"
},{
"id": "my-transformed-endpoint",
"method": "GET",
"path_regex": "/transformed",
"transform_request_headers": true
}]
}
},
...
}]
Via transform_request_headers
you can set which headers should be overridden or added. In this case RIG would override host
header and add completely new header custom-header
. Same as with auth, you can define per endpoint if you want to transform headers using transform_request_headers
property. Headers transformation is by default disabled and transform_request_headers
field set to false
.
URL rewriting
With URL rewriting you can set how the incoming and outgoing request urls should look like.
[{
"id": "my-service",
"version_data": {
"default": {
"endpoints": [{
"id": "my-transformed-endpoint",
"method": "GET",
"path_regex": "/foo/([^/]+)/bar/([^/]+)",
"path_replacement": "/bar/\\1/foo/\\2"
}]
}
},
...
}]
As you send GET request to /foo/1/bar/2
RIG will forward it to GET /bar/1/foo/2
.
CORS
Quite often you need to deal with cross origin requests. CORS itself is configured via CORS
environment variable, which defaults to *
. In addition RIG requires to configure OPTIONS pre-flight endpoints:
[{
"id": "my-service",
"version_data": {
"default": {
"endpoints": [{
"id": "my-endpoint",
"method": "GET",
"path_regex": "/"
},{
"id": "my-endpoint-preflight",
"method": "OPTIONS",
"path_regex": "/"
}]
}
},
...
}]
Request logger
Every request going through reverse proxy can be tracked by loggers -- console
or/and kafka
. To enable such logging, set REQUEST_LOG
to one or both of them (comma separated).
In case of Kafka, you can also set which Avro schema to use via KAFKA_LOG_SCHEMA
.