configuring varnish to use custom http headers
By default caches uses the field “Cache-Control” to get the TTL for objects they care about. In case of a server side cache the parameter “s-maxage” is responsible for the life time of an object. (see rfc2616). This is fine for almost all cases. The bad thing about this is a bad configured cache proxy also uses this parameter to get it’s caching informations. This causes the backend to have no control to invalidate such an object because it will remain in memory of such a bad configured cache.
So we decided to use a custom field in the http header to tell varnish how long an object should stay in cache. In the backend it was very easy to implement a few lines of code to set the custom field. I called it “Varnish-Control” . The field will contain an integer value which indicates the TTL in seconds. The main part of the work was to configure varnish to use this value instead of s-maxage form the field “Cache-Control”.
In words i did the following:
Once an item was fetched from the backend (so it was not found in the cache) I will set the TTL of the item and stuff it into the cache. Since the field “Varnish-Control” is useless for all request handling nodes on the way to the client i removed the field once the TTL was set.
The first approach was pretty simple:
sub vcl_fetch {
if (obj.http.Varnish-Control) {
set obj.ttl = 7d;
remove obj.http.Varnish-Control;
}
insert;
}
Unfortunately this solution did not care about the value I’ve set into “Varnish-Control” but uses a static value of seven days to cache items. This was not satisfying. So i tried to make it more dynamic. Since it is not possible to assign the value from obj.http.Varnish-Control to obj.ttl i had to use inline C code. This is done by using C{ … }C somewhere in the VCL file.
sub vcl_fetch {
if (obj.http.Varnish-Control) {
C{
char *ttl;
ttl = VRT_GetHdr(sp, HDR_OBJ, "\020Varnish-Control:");
VRT_l_obj_ttl(sp, atoi(ttl));
}C
remove obj.http.Varnish-Control;
}
insert;
}
Finding the right commands to access the header informations and write them into the TTL of the object was PITA. Anyway, I try to explain this two commands:
char *VRT_GetHdr(const struct sess *sp, enum gethdr_e where, const char *n)
- The first parameter is the session pointer…nothing more to say about this.
- The second parameter is a constant indicating from which object will be used to get the header informations. In this case it was “HDR_OBJ” which is equivalent to “obj” in plain vcl configuration scenarios. possible values are: HDR_REQ, HDR_RESP, HDR_OBJ and HDR_BEREQ.
- The third parameter is the name of the field i like to access. NOTE: The field name has to be suffixed with a colon and it has to be prefixed with the length of the field name (including the colon). This value has to be noted octal.
void VRT_l_obj_ttl(const struct sess *, double)
- The first parameter is again the session pointer.
- The second parameter is the TTL as double.
Thats it!
All VRT_x commands will be available in this context. This commands can be found in varnish-cache/include/vrt.h and varnish-cache/include/vrt_obj.h
Lovely Systems is using Amazons Elastic Cloud (EC2), Simple Storage Service (S3) and Simple Queue Service (SQS) for business critical applications. yesterday we experienced a outage of the services.