September 3, 2013
updated September 3, 2013

Tags

Webhook payload formats

Webhooks are callbacks on the web, and they are cool . When some event occurs on server 1, it notifies server 2 by simply sending a HTTP request to it.

But simply notifying that an event happened is often not enough; most times you want to know what happened. This information is transferred in the payload.

Payload formats​

There are several ways to transfer the additional data:

application/x-www-form-urlencoded​

That's the format browsers send when you submit a simple form on the web. It was officially defined in HTML 4.01 and is supported by probably every HTTP client and server.

When receiving such a request, the Content-Type header is set to application/x-www-form-urlencoded.

Data in this format are written down like the URL parameters in GET requests:

Special characters like = and & have to be percent-encoded, increasing the size of the POST content a bit.

Another issue with this format is that the size increases strongly when transmitting deeply nested data, because for each attribute, the full path needs to be submitted again and again:

In this example, addressbook.entries is repeated, and the data path is actually longer than the values.

You also have to take into consideration that - since the Content-Type header is always set to application/x-www-form-urlencoded - there is no standard way of determining which data you get, except by inspecting the data itself. So it's not possible to use the web hook endpoint for more than one type of web hooks.

Examples in the wild​

JSON​

JSON, defined in RFC 4627, is a JavaScript-native readable data serialization format. It's ideal as light-weight data exchange format between different programming languages, and especially useful when used in JavaScript web applications.

Compared to x-www-form-urlencoded, its size only increases linear for complex nested data structures.

The POST request's Content-Type header can either be set to application/json or the MIME type of the data that gets transferred in the payload, e.g. application/vnd.foo.commithook+json. By utilizing the Content-Type header, endpoints can reject unknown and support different payload formats easily.

Validation is a bit problematic, since JSON-schema is not finished nor stable (as of 2013-09). JSON validation tools are only beginning to emerge.

Examples in the wild​

XML​

XML has the same payload features like JSON, but is much more easily expandable thanks to namespaces.

The size of XML-serialized payload data is slightly larger than that of an equal JSON serialization.

XML can be validated against DTDs or XML Schema, both mature solutions. Tool support for validation is strong.

Examples in the wild​

application/x-www-form-urlencoded + JSON​

Both Github and Gitorious have a curious mix of form-encoded and JSON data: The payload is encoded as JSON, which in turn is form-encoded as "payload" parameter.

This is nonsense; the payload size increases through double encoding, and you don't even have the benefit of data schema identification via the Content-Type header.

Examples in the wild​

Best format for payloads?​

application/x-www-form-urlencoded is only for simple data, and shouldn't be used if you want to validate incoming data or want to provide a handler for multiple hooks.

XML and JSON are on par, with XML having better validation support. JSON is often easier to load. Use XML if you want easy extensibility, JSON otherwise.

Receiving webhook calls​

Implementing a webhook handler is easy. It differs slightly from format to format.

application/x-www-form-urlencoded​

Easy peasy: The data are already in $_POST: 

<?php
var_dump($_POST);
?>

JSON​

The data are in the POST body, which you need to read from php://input: 

<?php
$fullPostData = file_get_contents('php://input');
$data = json_decode($fullPostData);
var_dump($data);
?>

You may want to make sure that it's a POST request, and that the content type is correct: 

<?php
if ($_SERVER['REQUEST_METHOD'] != 'POST') {
    header('HTTP/1.0 405 Method Not Allowed');
    echo "Only POST requests allowed\n";
    exit();
}
 
if ($_SERVER['CONTENT_TYPE'] != 'application/vnd.example.activation+json') {
    header('HTTP/1.0 415 Unsupported Media Type');
    echo "I only accept application/vnd.example.activation+json data\n";
    exit();
}
 
$fullPostData = file_get_contents('php://input');
if ($fullPostData == '') {
    header('HTTP/1.0 400 Bad request');
    echo "No data submitted\n";
    exit();
}
 
$data = json_decode($fullPostData);
if ($data === null) {
    header('HTTP/1.0 400 Bad request');
    echo "JSON cannot be decoded\n";
    exit();
}
 
//do something with the data
var_dump($data);
?>

In case of an error, status code 405, status code 415 or status code 400 is returned.

Sending webhook calls​

Calling web hooks is easy, too - stock PHP brings everything we need, thanks to the HTTP stream context options : 

<?php
$payload = array(
    'username'  => 'dirty.harry',
    'confirmed' => true,
    'email'     => 'harry@mud.example.org'
);
 
$context = stream_context_create(
    array(
        'http' => array(
            'method'  => 'POST',
            'header'  => 'Content-Type: application/vnd.example.activation+json',
            'content' => json_encode($payload)
        )
    )
);
 
$response = file_get_contents('http://example.org/handler.php', false, $context);
//in most cases we can ignore the response data
echo $response;
?>

Written by Christian Weiske.

Comments? Please send an e-mail.