blob: 5fb255c36e0ff79a7fb7cc8a0449b8fba241392b [file] [log] [blame]
/*
* Copyright (c) Facebook, Inc. and its affiliates.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
{namespace buck.http_cache_api}
/***/
{template .soyweb}
{call buck.page}
{param title: 'HTTP Cache API' /}
{param navid: 'concept_http_cache_api' /}
{param description}
How Buck communicates with an HTTP caching server.
{/param}
{param content}
<p>
The buck client has two distinct protocols to communicate with the buck HTTP cache servers:
</p>
<ul>
<li><a href='#thrift_http'>Thrift Over HTTP Protocol</a></li>
<li><a href='#binary_http'>Binary HTTP Protocol</a></li>
</ul>
<h2 id="thrift_http">Thrift Over HTTP Protocol <a class="inline-link" href="#thrift_http">#</a></h2>
<p>
The goal behind this protocol is to make it easier to extend in the future.
Binary protocols with custom serialization/deserialization always result in
backwards compatibility problems. We overcome this by using thrift
serialization to represent our requests and responses.
</p>
<h3>The HTTP Headers</h3>
<p>
Each request and response should set the following HTTP headers:
</p>
<table>
<thead><tr><th>HTTP Header</th><th>Possible Values (comma separated)</th></tr></thead>
<tr><td>Content-Type</td><td>application/x-hybrid-thrift-binary</td></tr>
<tr><td>X-Thrift-Protocol</td><td>compact, binary, json, simple_json</td></tr>
</table>
<p>
The <i>X-Thrift-Protocol</i> is intended to specify the thrift protocol used to serialize the Thrift Metadata Section of the body.
</p>
<h3>The HTTP Request/Response Format</h3>
The body of each Request and Response is always composed of:
<ol>
<li>
<strong>Metadata Size Bytes Section: </strong>
4 bytes representing an int32 in big-endian byte-order that corresponds to the number of bytes N of the <i>Thrift Metadata Section</i>.
</li>
<li>
<strong>Thrift Metadata Section: </strong>
N bytes representing the <i>Thrift Metadata Section</i>.
</li>
<li>
<strong>Payload Section: </strong>
The rest of the bytes in the body correspond to the <i>Payload Section</i>.
</li>
</ol>
<p>
All HTTP requests made using this protocol should result in a 200 HTTP Status Code.
Information about cache miss, invalid formats or backend problems should all come as part of the <i>BuckCacheResponse</i> thrift type described below.
</p>
<h3>Thrift Metadata Section</h3>
<p>
The Thrift Metadata Section for HTTP Requests is always of thrift
type <i>BuckCacheRequest</i>. In the case of HTTP Responses, the thrift
type of this section will be BuckCacheResponse.
</p>
<p>
All thrift types involved in this protocol can be found in the following source file:
<ul>
<li>
<a href="https://github.com/facebook/buck/blob/master/src/com/facebook/buck/artifact_cache/thrift/buckcache.thrift">buck/src/com/facebook/buck/artifact_cache/thrift/buckcache.thrift</a>
</li>
</ul>
</p>
<h3>Payload Section</h3>
<p>
This sections contains the raw bytes corresponding to the contents of the cache artifact being operated on.
For cache fetch Requests and cache store Responses, this payload will contain zero bytes (ie, it is not used).
</p>
<h2 id="binary_http">Binary HTTP Protocol <a class="inline-link" href="#binary_http">#</a></h2>
<p>
Buck makes two types of requests to the cache. Note that while there is a distinction between
metadata and data, the two are combined in the same way in both types of request, so they can be
stored as a single blob by the server. However, while metadata is unique to each set of keys, data
can be duplicated between keys, so storing them separately allows for deduplication.
</p>
<h3>Fetch an artifact from the cache.</h3>
<p><code>GET /artifacts/key/[key]?target=[buildTarget]</code></p>
<ul>
<li>
<p>If the artifact is cached, the response should be:</p>
<ul>
<li><p>status <code>200</code></p></li>
<li><p>content-type <code>application/octet-stream</code></p></li>
</ul>
<ol>
<li>
<p>
32 bit big endian signed integer denoting the number of length in bytes of the metadata
</p>
</li>
<li><p>(1) bytes of metadata</p></li>
<li><p>The artifact's data</p></li>
</ol>
</li>
<li>
<p>If the artifact is not cached, the response should be:</p>
<ul>
<li><p>status <code>404</code></p></li>
</ul>
</li>
</ul>
<p>The <code>target</code> <code>GET</code> parameter is the fully qualified target name (eg <code>//foo:bar#default,static</code>) associated with the rulekey. The parameter is optional and will only be included in situations where the target is known to Buck.</p>
<h3>Store an artifact in the cache.</h3>
<p><code>PUT /artifacts/key</code></p>
<ul>
<li><p>status <code>202</code></p></li>
<li><p>content-type <code>application/octet-stream</code></p></li>
</ul>
<ol>
<li><p>32 bit big endian signed integer denoting the number of keys</p></li>
<li>
<p>
(1) strings, one for each key. Each string is represented as a 16 bit big endian unsigned
integer followed by the bytes of the string encoded in UTF-8
</p>
</li>
<li>
<p>
32 bit big endian signed integer denoting the number of length in bytes of the metadata
</p>
</li>
<li><p>(3) bytes of metadata</p></li>
<li>The artifact's data</li>
</ol>
{/param}
{/call}
{/template}