Apache Content Negotiation

+ +Content Negotiation + +

+ +

Apache's supports content negotiation as described in + the HTTP/1.1 specification. It can choose the best + representation of a resource based on the browser-supplied + preferences for media type, languages, character set and + encoding. It also implements a couple of features to give + more intelligent handling of requests from browsers that send + incomplete negotiation information.

+ +

Content negotiation is provided by the + mod_negotiation module. + which is compiled in by default.

+ +

About Content Negotiation + +

A resource may be available in several different + representations. For example, it might be available in + different languages or different media types, or a combination. + One way of selecting the most appropriate choice is to give the + user an index page, and let them select. However it is often + possible for the server to choose automatically. This works + because browsers can send as part of each request information + about what representations they prefer. For example, a browser + could indicate that it would like to see information in French, + if possible, else English will do. Browsers indicate their + preferences by headers in the request. To request only French + representations, the browser would send

+ +Accept-Language: fr + +

Note that this preference will only be applied when there is + a choice of representations and they vary by language.

+ +

As an example of a more complex request, this browser has + been configured to accept French and English, but prefer + French, and to accept various media types, preferring HTML over + plain text or other text types, and preferring GIF or JPEG over + other media types, but also allowing any other media type as a + last resort:

+ + + Accept-Language: fr; q=1.0, en; q=0.5
+ Accept: text/html; q=1.0, text/*; q=0.8, image/gif; q=0.6, image/jpeg; q=0.6, image/*; q=0.5, */*; q=0.1 + + +

Apache supports 'server driven' content negotiation, as + defined in the HTTP/1.1 specification. It fully supports the + Accept, Accept-Language, Accept-Charset and Accept-Encoding + request headers. Apache also supports 'transparent' + content negotiation, which is an experimental negotiation + protocol defined in RFC 2295 and RFC 2296. It does not offer + support for 'feature negotiation' as defined in these RFCs.

+ +

A resource is a conceptual entity + identified by a URI (RFC 2396). An HTTP server like Apache + provides access to representations of the + resource(s) within its namespace, with each representation in + the form of a sequence of bytes with a defined media type, + character set, encoding, etc. Each resource may be associated + with zero, one, or more than one representation at any given + time. If multiple representations are available, the resource + is referred to as negotiable and each of its + representations is termed a variant. The ways + in which the variants for a negotiable resource vary are called + the dimensions of negotiation.

+ +

Negotiation in Apache + +

In order to negotiate a resource, the server needs to be + given information about each of the variants. This is done in + one of two ways:

+ +

Using a type map (i.e., a *.var + file) which names the files containing the variants + explicitly, or
Using a 'MultiViews' search, where the server does an + implicit filename pattern match and chooses from among the + results.

+ +

Using a type-map file + +

A type map is a document which is associated with the + handler named type-map (or, for + backwards-compatibility with older Apache configurations, the + mime type application/x-type-map). Note that to + use this feature, you must have a handler set in the + configuration that defines a file suffix as + type-map; this is best done with a

+AddHandler type-map .var +

in the server configuration file.

+ +

Type map files should have the same name as the resource + which they are describing, and have an entry for each available + variant; these entries consist of contiguous HTTP-format header + lines. Entries for different variants are separated by blank + lines. Blank lines are illegal within an entry. It is + conventional to begin a map file with an entry for the combined + entity as a whole (although this is not required, and if + present will be ignored). An example map file is shown below. + This file would be named foo.var, as it describes + a resource named foo.

+ + + URI: foo
+
+ URI: foo.en.html
+ Content-type: text/html
+ Content-language: en
+
+ URI: foo.fr.de.html
+ Content-type: text/html;charset=iso-8859-2
+ Content-language: fr, de
+ +

Note also that a typemap file will take precedence over the + filename's extension, even when Multiviews is on. If the + variants have different source qualities, that may be indicated + by the "qs" parameter to the media type, as in this picture + (available as jpeg, gif, or ASCII-art):

+ + + URI: foo
+
+ URI: foo.jpeg
+ Content-type: image/jpeg; qs=0.8
+
+ URI: foo.gif
+ Content-type: image/gif; qs=0.5
+
+ URI: foo.txt
+ Content-type: text/plain; qs=0.01
+ + +

qs values can vary in the range 0.000 to 1.000. Note that + any variant with a qs value of 0.000 will never be chosen. + Variants with no 'qs' parameter value are given a qs factor of + 1.0. The qs parameter indicates the relative 'quality' of this + variant compared to the other available variants, independent + of the client's capabilities. For example, a jpeg file is + usually of higher source quality than an ascii file if it is + attempting to represent a photograph. However, if the resource + being represented is an original ascii art, then an ascii + representation would have a higher source quality than a jpeg + representation. A qs value is therefore specific to a given + variant depending on the nature of the resource it + represents.

+ +

The full list of headers recognized is available in the mod_negotation + typemap documentation.

+ +

Multiviews + +

MultiViews is a per-directory option, meaning it + can be set with an Options + directive within a Directory, Location or Files section in + access.conf, or (if AllowOverride is properly set) in + .htaccess files. Note that Options All + does not set MultiViews; you have to ask for it by + name.

+ +

The effect of MultiViews is as follows: if the + server receives a request for /some/dir/foo, if + /some/dir has MultiViews enabled, and + /some/dir/foo does not exist, then the + server reads the directory looking for files named foo.*, and + effectively fakes up a type map which names all those files, + assigning them the same media types and content-encodings it + would have if the client had asked for one of them by name. It + then chooses the best match to the client's requirements.

+ +

MultiViews may also apply to searches for the file + named by the DirectoryIndex directive, if the + server is trying to index a directory. If the configuration files + specify

+DirectoryIndex index +

then the server will arbitrate between index.html + and index.html3 if both are present. If neither + are present, and index.cgi is there, the server + will run it.

+ +

If one of the files found when reading the directory does not + have an extension recognized by mod_mime to designate + its Charset, Content-Type, Language, or Encoding, then the result + depends on the setting of the MultiViewsMatch directive. This + directive determines whether handlers, filters, and other + extension types can participate in MultiViews negotiation.

+ +

The Negotiation Methods + +

After Apache has obtained a list of the variants for a given + resource, either from a type-map file or from the filenames in + the directory, it invokes one of two methods to decide on the + 'best' variant to return, if any. It is not necessary to know + any of the details of how negotiation actually takes place in + order to use Apache's content negotiation features. However the + rest of this document explains the methods used for those + interested.

+ +

There are two negotiation methods:

+ +

Server driven negotiation with the Apache + algorithm is used in the normal case. The Apache + algorithm is explained in more detail below. When this + algorithm is used, Apache can sometimes 'fiddle' the quality + factor of a particular dimension to achieve a better result. + The ways Apache can fiddle quality factors is explained in + more detail below.
Transparent content negotiation is used + when the browser specifically requests this through the + mechanism defined in RFC 2295. This negotiation method gives + the browser full control over deciding on the 'best' variant, + the result is therefore dependent on the specific algorithms + used by the browser. As part of the transparent negotiation + process, the browser can ask Apache to run the 'remote + variant selection algorithm' defined in RFC 2296.

+ +

Dimensions of Negotiation + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Dimension	Notes
Media Type	Browser indicates preferences with the Accept header + field. Each item can have an associated quality factor. + Variant description can also have a quality factor (the + "qs" parameter).
Language	Browser indicates preferences with the Accept-Language + header field. Each item can have a quality factor. Variants + can be associated with none, one or more than one + language.
Encoding	Browser indicates preference with the Accept-Encoding + header field. Each item can have a quality factor.
Charset	Browser indicates preference with the Accept-Charset + header field. Each item can have a quality factor. Variants + can indicate a charset as a parameter of the media + type.

+ +

Apache Negotiation Algorithm + +

Apache can use the following algorithm to select the 'best' + variant (if any) to return to the browser. This algorithm is + not further configurable. It operates as follows:

+ +

First, for each dimension of the negotiation, check the + appropriate Accept* header field and assign a + quality to each variant. If the Accept* header for + any dimension implies that this variant is not acceptable, + eliminate it. If no variants remain, go to step 4.
+ Select the 'best' variant by a process of elimination. Each + of the following tests is applied in order. Any variants + not selected at each test are eliminated. After each test, + if only one variant remains, select it as the best match + and proceed to step 3. If more than one variant remains, + move on to the next test. + +
1. Multiply the quality factor from the Accept header + with the quality-of-source factor for this variant's + media type, and select the variants with the highest + value.
2. Select the variants with the highest language quality + factor.
3. Select the variants with the best language match, + using either the order of languages in the + Accept-Language header (if present), or else the order of + languages in the LanguagePriority directive + (if present).
4. Select the variants with the highest 'level' media + parameter (used to give the version of text/html media + types).
5. Select variants with the best charset media + parameters, as given on the Accept-Charset header line. + Charset ISO-8859-1 is acceptable unless explicitly + excluded. Variants with a text/* media type + but not explicitly associated with a particular charset + are assumed to be in ISO-8859-1.
6. Select those variants which have associated charset + media parameters that are not ISO-8859-1. If + there are no such variants, select all variants + instead.
7. Select the variants with the best encoding. If there + are variants with an encoding that is acceptable to the + user-agent, select only these variants. Otherwise if + there is a mix of encoded and non-encoded variants, + select only the unencoded variants. If either all + variants are encoded or all variants are not encoded, + select all variants.
8. Select the variants with the smallest content + length.
9. Select the first variant of those remaining. This + will be either the first listed in the type-map file, or + when variants are read from the directory, the one whose + file name comes first when sorted using ASCII code + order.
+
The algorithm has now selected one 'best' variant, so + return it as the response. The HTTP response header Vary is + set to indicate the dimensions of negotiation (browsers and + caches can use this information when caching the resource). + End.
To get here means no variant was selected (because none + are acceptable to the browser). Return a 406 status (meaning + "No acceptable representation") with a response body + consisting of an HTML document listing the available + variants. Also set the HTTP Vary header to indicate the + dimensions of variance.

+ +

Fiddling with Quality + Values + +

Apache sometimes changes the quality values from what would + be expected by a strict interpretation of the Apache + negotiation algorithm above. This is to get a better result + from the algorithm for browsers which do not send full or + accurate information. Some of the most popular browsers send + Accept header information which would otherwise result in the + selection of the wrong variant in many cases. If a browser + sends full and correct information these fiddles will not be + applied.

+ +

Media Types and Wildcards + +

The Accept: request header indicates preferences for media + types. It can also include 'wildcard' media types, such as + "image/*" or "*/*" where the * matches any string. So a request + including:

+ +Accept: image/*, */* + +

would indicate that any type starting "image/" is acceptable, + as is any other type (so the first "image/*" is redundant). + Some browsers routinely send wildcards in addition to explicit + types they can handle. For example:

+ + + Accept: text/html, text/plain, image/gif, image/jpeg, */* + +

The intention of this is to indicate that the explicitly listed + types are preferred, but if a different representation is + available, that is ok too. However under the basic algorithm, + as given above, the */* wildcard has exactly equal preference + to all the other types, so they are not being preferred. The + browser should really have sent a request with a lower quality + (preference) value for *.*, such as:

+ + Accept: text/html, text/plain, image/gif, image/jpeg, */*; q=0.01 + +

The explicit types have no quality factor, so they default to a + preference of 1.0 (the highest). The wildcard */* is given a + low preference of 0.01, so other types will only be returned if + no variant matches an explicitly listed type.

+ +

If the Accept: header contains no q factors at all, + Apache sets the q value of "*/*", if present, to 0.01 to + emulate the desired behavior. It also sets the q value of + wildcards of the format "type/*" to 0.02 (so these are + preferred over matches against "*/*". If any media type on the + Accept: header contains a q factor, these special values are + not applied, so requests from browsers which send the + correct information to start with work as expected.

+ +

Language Negotiation Exceptions + +

New in Apache 2.0, some exceptions have been added to the + negotiation algorithm to allow graceful fallback when language + negotiation fails to find a match.

+ +

When a client requests a page on your server, but the server + cannot find a single page that matches the Accept-language sent by + the browser, the server will return either a "No Acceptable + Variant" or "Multiple Choices" response to the client. To avoid + these error messages, it is possible to configure Apache to ignore + the Accept-language in these cases and provide a document that + does not explictly match the client's request. The ForceLanguagePriority + directive can be used to override one or both of these error + messages and subsitute the servers judgement in the form of the + LanguagePriority + directive.

+ +

The server will also attempt to match language-subsets when no + other match can be found. For example, if a client requests + documents with the language en-GB for British + English, the server is not normally allowed by the HTTP/1.1 + standard to match that against a document that is marked as simply + en. (Note that it is almost surely a configuration + error to include en-GB and not en in the + Accept-Language header, since it is very unlikely that a reader + understands British English, but doesn't understand English in + general. Unfortunately, many current clients have default + configurations that resemble this.) However, if no other language + match is possible and the server is about to return a "No + Acceptable Variants" error or fallback to the LanguagePriority, the server + will ignore the subset specification and match en-GB + against en documents. Implicitly, Apache will add + the parent language to the client's acceptable language list with + a very low quality value. But note that if the client requests + "en-GB; qs=0.9, fr; qs=0.8", and the server has documents + designated "en" and "fr", then the "fr" document will be returned. + This is necessary to maintain compliance with the HTTP/1.1 + specification and to work effectively with properly configured + clients.

+ +

Extensions to Transparent Content +Negotiation + +

Apache extends the transparent content negotiation protocol (RFC +2295) as follows. A new {encoding ..} element is used in +variant lists to label variants which are available with a specific +content-encoding only. The implementation of the RVSA/1.0 algorithm +(RFC 2296) is extended to recognize encoded variants in the list, and +to use them as candidate variants whenever their encodings are +acceptable according to the Accept-Encoding request header. The +RVSA/1.0 implementation does not round computed quality factors to 5 +decimal places before choosing the best variant.

+ +

Note on hyperlinks and naming conventions + +

If you are using language negotiation you can choose between + different naming conventions, because files can have more than + one extension, and the order of the extensions is normally + irrelevant (see the mod_mime documentation + for details).

+ +

A typical file has a MIME-type extension (e.g., + html), maybe an encoding extension (e.g., + gz), and of course a language extension + (e.g., en) when we have different + language variants of this file.

+ +

Examples:

+ +

foo.en.html
foo.html.en
foo.en.html.gz

+ +

Here some more examples of filenames together with valid and + invalid hyperlinks:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Filename	Valid hyperlink	Invalid hyperlink
foo.html.en	foo + foo.html	-
foo.en.html	foo	foo.html
foo.html.en.gz	foo + foo.html	foo.gz + foo.html.gz
foo.en.html.gz	foo	foo.html + foo.html.gz + foo.gz
foo.gz.html.en	foo + foo.gz + foo.gz.html	foo.html
foo.html.gz.en	foo + foo.html + foo.html.gz	foo.gz

+ +

Looking at the table above you will notice that it is always + possible to use the name without any extensions in an hyperlink + (e.g., foo). The advantage is that you + can hide the actual type of a document rsp. file and can change + it later, e.g., from html to + shtml or cgi without changing any + hyperlink references.

+ +

If you want to continue to use a MIME-type in your + hyperlinks (e.g. foo.html) the language + extension (including an encoding extension if there is one) + must be on the right hand side of the MIME-type extension + (e.g., foo.html.en).

+ +

Note on Caching + +

When a cache stores a representation, it associates it with + the request URL. The next time that URL is requested, the cache + can use the stored representation. But, if the resource is + negotiable at the server, this might result in only the first + requested variant being cached and subsequent cache hits might + return the wrong response. To prevent this, Apache normally + marks all responses that are returned after content negotiation + as non-cacheable by HTTP/1.0 clients. Apache also supports the + HTTP/1.1 protocol features to allow caching of negotiated + responses.

+ +

For requests which come from a HTTP/1.0 compliant client + (either a browser or a cache), the directive CacheNegotiatedDocs can be + used to allow caching of responses which were subject to + negotiation. This directive can be given in the server config or + virtual host, and takes no arguments. It has no effect on requests + from HTTP/1.1 clients.

+ +

More Information + +

For more information about content negotiation, see Alan + J. Flavell's Language + Negotiation Notes. But note that this document may not be + updated to include changes in Apache 2.0.

+ +

Content Negotiation

Apache HTTP Server Version 2.0

Content Negotiation

About Content Negotiation

About Content Negotiation

Negotiation in Apache

Negotiation in Apache

Using a type-map file

Using a type-map file

Multiviews

Multiviews

The Negotiation Methods

The Negotiation Methods

Dimensions of Negotiation

Dimensions of Negotiation

Apache Negotiation Algorithm

Apache Negotiation Algorithm

Fiddling with Quality +
Fiddling with Quality Values

Apache sometimes changes the quality values from what would @@ -384,36 +368,37 @@ sends full and correct information these fiddles will not be applied.
-

Fiddling with Quality Values

Media Types and Wildcards

Media Types and Wildcards

Language Negotiation Exceptions

Language Negotiation Exceptions

Extensions to Transparent Content Negotiation

Note on hyperlinks and naming conventions

Extensions to Transparent Content +Negotiation

Note on hyperlinks and naming conventions

Note on Caching

Note on Caching

More Information

More Information

Apache HTTP Server Version 2.0

Content Negotiation

Apache HTTP Server Version 2.0

Content Negotiation

About Content Negotiation

Negotiation in Apache

Using a type-map file

Multiviews

The Negotiation Methods

Dimensions of Negotiation

Apache Negotiation Algorithm

Fiddling with Quality +Fiddling with Quality Values Apache sometimes changes the quality values from what would @@ -384,36 +368,37 @@ sends full and correct information these fiddles will not be applied. -

Media Types and Wildcards

Language Negotiation Exceptions

Extensions to Transparent Content Negotiation

Note on hyperlinks and naming conventions

Note on Caching

More Information

Apache HTTP Server Version 2.0

Fiddling with Quality +
Fiddling with Quality Values

Apache sometimes changes the quality values from what would @@ -384,36 +368,37 @@ sends full and correct information these fiddles will not be applied.
-