From: Jim Jagielski Date: Fri, 24 Mar 2017 13:43:12 +0000 (+0000) Subject: Merge r1780328, r1780329, r1781329, r1782164, r1782166, r1782193, r1778350, r1781329... X-Git-Tag: 2.4.26~231 X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=93eb591823450302a2f7e2a679bb82b2be34099a;p=apache Merge r1780328, r1780329, r1781329, r1782164, r1782166, r1782193, r1778350, r1781329, r1782194, r1782323, r1782418, r1782419, r1782482, r1782532, r1788040 from trunk: Adjust as needed debugging add ProxyFCGISetEnvIf Logging update Follow up to r1782164: fix typo (closing double-quote). Remove trailing whitespace : no functional change PR60576: php-fpm broken w/ per-dir rewrites Attempt to dig out of well-meaning fixes for generic fcgi backends that negatively affected some FPM configs. Adds ProxyFCGIBackendType add ProxyFCGISetEnvIf trailing whitespace Allow final admin-level fine-tuning mod_proxy_fcgi: fix spelling in APLOG_INFO message mod_proxy_fcgi: allow setting empty variables in ProxyFCGISetEnvIf support unsetting vars ProxyFCGISetEnvIf: reject invalid invocations, streamline unsets Neither of the following makes sense: - ProxyFCGISetEnvIf cond !VARIABLE value - ProxyFCGISetEnvIf cond ! Error out in these cases. Also, don't execute the (unused) replacement expression when unsetting. drop the longhand version Submitted by: jim, covener, jim, ylavic, jim, covener, covener, jim, jim, jchampion, jchampion, covener, jchampion, covener Reviewed by: jim, covener, jchampion git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1788445 13f79535-47bb-0310-9956-ffa450edef68 --- diff --git a/CHANGES b/CHANGES index b476176913..ccab19ea51 100644 --- a/CHANGES +++ b/CHANGES @@ -2,6 +2,21 @@ Changes with Apache 2.4.26 + *) mod_proxy_fcgi: Add ProxyFCGISetEnvIf to fixup CGI environment + variables just before invoking the FastCGI. [Eric Covener, + Jacob Champion] + + *) mod_proxy: Allow the per-request environment variable "no-proxy" to + be used as an alternative to ProxyPass /path !. This is primarily + to set exceptions for ProxyPass specified in context. + Use SetEnvIf, not SetEnv. [Eric Covener] + + *) mod_proxy_fcgi: Return to 2.4.20-and-earlier behavior of leaving + a "proxy:fcgi://" prefix in the SCRIPT_FILENAME environment variable by + default. Add ProxyFCGIBackendType to allow the type of backend to be + specified so these kinds of fixups can be restored without impacting + FPM. PR60576 [Eric Covener, Jim Jagielski] + *) mod_ssl: work around leaks on (graceful) restart. [Yann Ylavic] *) mod_ssl: Add support for OpenSSL 1.1.0. [Rainer Jung] diff --git a/STATUS b/STATUS index 82ce4f94dd..015316bdcd 100644 --- a/STATUS +++ b/STATUS @@ -137,31 +137,6 @@ PATCHES ACCEPTED TO BACKPORT FROM TRUNK: 2.4.x patch: http://people.apache.org/~covener/patches/ebcdic-interim.diff +1: covener, wrowe, ylavic (by inspection) - *) PR 60576: 2.4.21 broke PHP-FPM with the patch to strip the bogus "proxy://" - prefix from SCRIPT_FILENAME. We need to revert to the previous behavior - ASAP to avoid any further hurdles with FCGI implementations while we figure - this out. - trunk patch: http://svn.apache.org/r1780328 - http://svn.apache.org/r1780329 - http://svn.apache.org/r1781329 - http://svn.apache.org/r1782164 - http://svn.apache.org/r1782166 - http://svn.apache.org/r1782193 - http://svn.apache.org/r1778350 - http://svn.apache.org/r1781329 - http://svn.apache.org/r1782194 - http://svn.apache.org/r1782323 - http://svn.apache.org/r1782418 - http://svn.apache.org/r1782419 - http://svn.apache.org/r1782482 - http://svn.apache.org/r1782532 - http://svn.apache.org/r1788040 - 2.4.x patch: http://home.apache.org/~jim/patches/mod_proxy_fcgi-v3.patch - + http://svn.apache.org/r1788040 - +1: jim, covener, jchampion - jchampion: holding off on promotion for an answer to jailletc36's ML - question regarding `ProxyFCGIBackendType PHP-FPM` - covener: added http://svn.apache.org/r1788040 PATCHES PROPOSED TO BACKPORT FROM TRUNK: [ New proposals should be added at the end of the list ] diff --git a/docs/manual/mod/mod_proxy_fcgi.xml b/docs/manual/mod/mod_proxy_fcgi.xml index fa5f27da28..9281d48cf1 100644 --- a/docs/manual/mod/mod_proxy_fcgi.xml +++ b/docs/manual/mod/mod_proxy_fcgi.xml @@ -195,4 +195,118 @@ ProxyPass "/myapp/" "balancer://myappcluster/" + +ProxyFCGIBackendType +Specify the type of backend FastCGI application +ProxyFCGIBackendType FPM|GENERIC +ProxyFCGIBackendType FPM +server config +virtual hostdirectory +.htaccess +Available in version 2.4.26 and later + + +

This directive allows the type of backend FastCGI application to be +specified. Some FastCGI servers, such as PHP-FPM, use historical quirks of +environment variables to identify the type of proxy server being used. Set +this directive to "GENERIC" if your non PHP-FPM application has trouble +interpreting environment variables such as SCRIPT_FILENAME or PATH_TRANSLATED +as set by the server.

+ +

One example of values that change based on the setting of this directive is +SCRIPT_FILENAME. When using mod_proxy_fcgi historically, +SCRIPT_FILENAME was prefixed with the string "proxy:fcgi://". This variable is +what some generic FastCGI applications would read as their script input, but +PHP-FPM would strip the prefix then remember it was talking to Apache. In +2.4.21 through 2.4.25, this prefix was automatically stripped by the server, +breaking the ability of PHP-FPM to detect and interoperate with Apache in some +scenarios.

+ + + + +ProxyFCGISetEnvIf +Allow variables sent to FastCGI servers to be fixed up +ProxyFCGISetEnvIf conditional-expression + [!]environment-variable-name + [value-expression] +server config +virtual hostdirectory +.htaccess +Available in version 2.4.26 and later + + +

Just before passing a request to the configured FastCGI server, the core of +the web server sets a number of environment variables based on details of the +current request. FastCGI programs often uses these environment variables +as inputs that determine what underlying scripts they will process, or what +output they directly produce.

+

Examples of noteworthy environment variables are:

+
    +
  • SCRIPT_NAME
    • +
  • SCRIPT_FILENAME
    • +
  • REQUEST_URI
    • +
  • PATH_INFO
    • +
  • PATH_TRANSLATED
    • +
+ +

This directive allows the environment variables above, or any others of +interest, to be overridden. This directive is evaluated after the initial +values for these variables are set, so they can be used as input into both +the condition expressions and value expressions.

+

Parameter syntax:

+
+
conditional-expression
+
Specifies an expression that controls whether the environment variable that + follows will be modified. For information on the expression syntax, see + the examples that follow or the full specification at the + ap_expr documentation. +
+
environment-variable-name
+
Specifies the CGI environment variable to change, + such as PATH_INFO. If preceded by an exclamation point, the variable + will be unset.
+
value-expression
+
Specifies the replacement value for the preceding environment variable. + Backreferences, such as "$1", can be included from regular expression + captures in conditional-expression. If omitted, the variable is + set (or overridden) to an empty string — but see the Note below.
+
+ + + +# A basic, unconditional override +ProxyFCGISetEnvIf "true" PATH_INFO "/example" + +# Use an environment variable in the value +ProxyFCGISetEnvIf "true" PATH_INFO "%{reqenv:SCRIPT_NAME}" + +# Use captures in the conditions and backreferences in the replacement +ProxyFCGISetEnvIf "reqenv('PATH_TRANSLATED') =~ m|(/.*prefix)(\d+)(.*)|" PATH_TRANSLATED "$1$3" + + + +Note: Unset vs. Empty + The following will unset VARIABLE, preventing it from being sent + to the FastCGI server: + + ProxyFCGISetEnvIf true !VARIABLE + + Whereas the following will erase any existing value of + VARIABLE (by setting it to the empty string), but the empty + VARIABLE will still be sent to the server: + + ProxyFCGISetEnvIf true VARIABLE + + The CGI/1.1 specification + does not + distinguish between a variable with an empty value and a variable that + does not exist. However, many CGI and FastCGI implementations distinguish (or + allow scripts to distinguish) between the two. The choice of which to use is + dependent upon your implementation and your reason for modifying the variable. + + + + + diff --git a/modules/mappers/mod_actions.c b/modules/mappers/mod_actions.c index 2a67a2742a..ac9c3b7428 100644 --- a/modules/mappers/mod_actions.c +++ b/modules/mappers/mod_actions.c @@ -186,7 +186,8 @@ static int action_handler(request_rec *r) ap_field_noparam(r->pool, r->content_type); if (action && (t = apr_table_get(conf->action_types, action))) { - if (*t++ == '0' && r->finfo.filetype == APR_NOFILE) { + int virtual = (*t++ == '0' ? 0 : 1); + if (!virtual && r->finfo.filetype == APR_NOFILE) { ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r, APLOGNO(00652) "File does not exist: %s", r->filename); return HTTP_NOT_FOUND; @@ -197,6 +198,9 @@ static int action_handler(request_rec *r) * (will be REDIRECT_HANDLER there) */ apr_table_setn(r->subprocess_env, "HANDLER", action); + if (virtual) { + apr_table_setn(r->notes, "virtual_script", "1"); + } } if (script == NULL) diff --git a/modules/proxy/mod_proxy_fcgi.c b/modules/proxy/mod_proxy_fcgi.c index 3dc74dff93..a268556e30 100644 --- a/modules/proxy/mod_proxy_fcgi.c +++ b/modules/proxy/mod_proxy_fcgi.c @@ -17,13 +17,38 @@ #include "mod_proxy.h" #include "util_fcgi.h" #include "util_script.h" +#include "ap_expr.h" module AP_MODULE_DECLARE_DATA proxy_fcgi_module; +typedef struct { + ap_expr_info_t *cond; + ap_expr_info_t *subst; + const char *envname; +} sei_entry; + typedef struct { int need_dirwalk; } fcgi_req_config_t; +/* We will assume FPM, but still differentiate */ +typedef enum { + BACKEND_DEFAULT_UNKNOWN = 0, + BACKEND_FPM, + BACKEND_GENERIC, +} fcgi_backend_t; + + +#define FCGI_MAY_BE_FPM(dconf) \ + (dconf && \ + ((dconf->backend_type == BACKEND_DEFAULT_UNKNOWN) || \ + (dconf->backend_type == BACKEND_FPM))) + +typedef struct { + fcgi_backend_t backend_type; + apr_array_header_t *env_fixups; +} fcgi_dirconf_t; + /* * Canonicalise http-like URLs. * scheme is the scheme for the URL @@ -39,7 +64,7 @@ static int proxy_fcgi_canon(request_rec *r, char *url) fcgi_req_config_t *rconf = NULL; const char *pathinfo_type = NULL; - if (strncasecmp(url, "fcgi:", 5) == 0) { + if (ap_cstr_casecmpn(url, "fcgi:", 5) == 0) { url += 5; } else { @@ -133,6 +158,49 @@ static int proxy_fcgi_canon(request_rec *r, char *url) return OK; } + +/* + ProxyFCGISetEnvIf "reqenv('PATH_INFO') =~ m#/foo(\d+)\.php$#" COVENV1 "$1" + ProxyFCGISetEnvIf "reqenv('PATH_INFO') =~ m#/foo(\d+)\.php$#" PATH_INFO "/foo.php" + ProxyFCGISetEnvIf "reqenv('PATH_TRANSLATED') =~ m#(/.*foo)(\d+)(.*)#" PATH_TRANSLATED "$1$3" +*/ +static void fix_cgivars(request_rec *r, fcgi_dirconf_t *dconf) +{ + sei_entry *entries; + const char *err, *src; + int i = 0, rc = 0; + ap_regmatch_t regm[AP_MAX_REG_MATCH]; + + entries = (sei_entry *) dconf->env_fixups->elts; + for (i = 0; i < dconf->env_fixups->nelts; i++) { + sei_entry *entry = &entries[i]; + + if (entry->envname[0] == '!') { + apr_table_unset(r->subprocess_env, entry->envname+1); + } + else if (0 < (rc = ap_expr_exec_re(r, entry->cond, AP_MAX_REG_MATCH, regm, &src, &err))) { + const char *val = ap_expr_str_exec_re(r, entry->subst, AP_MAX_REG_MATCH, regm, &src, &err); + if (err) { + ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r, APLOGNO(03514) + "Error evaluating expression for replacement of %s: '%s'", + entry->envname, err); + continue; + } + if (APLOGrtrace4(r)) { + const char *oldval = apr_table_get(r->subprocess_env, entry->envname); + ap_log_rerror(APLOG_MARK, APLOG_TRACE4, 0, r, + "fix_cgivars: override %s from '%s' to '%s'", + entry->envname, oldval, val); + + } + apr_table_setn(r->subprocess_env, entry->envname, val); + } + else { + ap_log_rerror(APLOG_MARK, APLOG_TRACE8, 0, r, "fix_cgivars: Condition returned %d", rc); + } + } +} + /* Wrapper for apr_socket_sendv that handles updating the worker stats. */ static apr_status_t send_data(proxy_conn_rec *conn, struct iovec *vec, @@ -253,7 +321,9 @@ static apr_status_t send_environment(proxy_conn_rec *conn, request_rec *r, apr_status_t rv; apr_size_t avail_len, len, required_len; int next_elem, starting_elem; + int fpm = 0; fcgi_req_config_t *rconf = ap_get_module_config(r->request_config, &proxy_fcgi_module); + fcgi_dirconf_t *dconf = ap_get_module_config(r->per_dir_config, &proxy_fcgi_module); if (rconf) { if (rconf->need_dirwalk) { @@ -268,15 +338,24 @@ static apr_status_t send_environment(proxy_conn_rec *conn, request_rec *r, if (!strncmp(r->filename, "proxy:balancer://", 17)) { newfname = apr_pstrdup(r->pool, r->filename+17); } - else if (!strncmp(r->filename, "proxy:fcgi://", 13)) { - newfname = apr_pstrdup(r->pool, r->filename+13); - } - /* Query string in environment only */ - if (newfname && r->args && *r->args) { - char *qs = strrchr(newfname, '?'); - if (qs && !strcmp(qs+1, r->args)) { - *qs = '\0'; + + if (!FCGI_MAY_BE_FPM(dconf)) { + if (!strncmp(r->filename, "proxy:fcgi://", 13)) { + /* If we strip this under FPM, and any internal redirect occurs + * on PATH_INFO, FPM may use PATH_TRANSLATED instead of + * SCRIPT_FILENAME (a la mod_fastcgi + Action). + */ + newfname = apr_pstrdup(r->pool, r->filename+13); + } + /* Query string in environment only */ + if (newfname && r->args && *r->args) { + char *qs = strrchr(newfname, '?'); + if (qs && !strcmp(qs+1, r->args)) { + *qs = '\0'; + } } + } else { + fpm = 1; } if (newfname) { @@ -285,11 +364,43 @@ static apr_status_t send_environment(proxy_conn_rec *conn, request_rec *r, } } +#if 0 + ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r, APLOGNO(09999) + "r->filename: %s", (r->filename ? r->filename : "nil")); + ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r, APLOGNO(09999) + "r->uri: %s", (r->uri ? r->uri : "nil")); + ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r, APLOGNO(09999) + "r->path_info: %s", (r->path_info ? r->path_info : "nil")); +#endif + ap_add_common_vars(r); ap_add_cgi_vars(r); + if (fpm || apr_table_get(r->notes, "virtual_script")) { + /* + * Adjust SCRIPT_NAME, PATH_INFO and PATH_TRANSLATED for PHP-FPM + * TODO: Right now, PATH_INFO and PATH_TRANSLATED look OK... + */ + const char *pend; + const char *script_name = apr_table_get(r->subprocess_env, "SCRIPT_NAME"); + pend = script_name + strlen(script_name); + if (r->path_info && *r->path_info) { + pend = script_name + ap_find_path_info(script_name, r->path_info) - 1; + } + while (pend != script_name && *pend != '/') { + pend--; + } + apr_table_setn(r->subprocess_env, "SCRIPT_NAME", pend); + ap_log_rerror(APLOG_MARK, APLOG_TRACE4, 0, r, + "fpm:virtual_script: Modified SCRIPT_NAME to: %s", + pend); + } + /* XXX are there any FastCGI specific env vars we need to send? */ + /* Give admins final option to fine-tune env vars */ + fix_cgivars(r, dconf); + /* XXX mod_cgi/mod_cgid use ap_create_environment here, which fills in * the TZ value specially. We could use that, but it would mean * parsing the key/value pairs back OUT of the allocated env array, @@ -915,7 +1026,7 @@ static int proxy_fcgi_handler(request_rec *r, proxy_worker *worker, "url: %s proxyname: %s proxyport: %d", url, proxyname, proxyport); - if (strncasecmp(url, "fcgi:", 5) != 0) { + if (ap_cstr_casecmpn(url, "fcgi:", 5) != 0) { ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r, APLOGNO(01077) "declining URL %s", url); return DECLINED; } @@ -976,18 +1087,120 @@ cleanup: return status; } +static void *fcgi_create_dconf(apr_pool_t *p, char *path) +{ + fcgi_dirconf_t *a; + + a = (fcgi_dirconf_t *)apr_pcalloc(p, sizeof(fcgi_dirconf_t)); + a->backend_type = BACKEND_DEFAULT_UNKNOWN; + a->env_fixups = apr_array_make(p, 20, sizeof(sei_entry)); + + return a; +} + +static void *fcgi_merge_dconf(apr_pool_t *p, void *basev, void *overridesv) +{ + fcgi_dirconf_t *a, *base, *over; + + a = (fcgi_dirconf_t *)apr_pcalloc(p, sizeof(fcgi_dirconf_t)); + base = (fcgi_dirconf_t *)basev; + over = (fcgi_dirconf_t *)overridesv; + + a->backend_type = (over->backend_type != BACKEND_DEFAULT_UNKNOWN) + ? over->backend_type + : base->backend_type; + a->env_fixups = apr_array_append(p, base->env_fixups, over->env_fixups); + return a; +} + +static const char *cmd_servertype(cmd_parms *cmd, void *in_dconf, + const char *val) +{ + fcgi_dirconf_t *dconf = in_dconf; + + if (!strcasecmp(val, "GENERIC")) { + dconf->backend_type = BACKEND_GENERIC; + } + else if (!strcasecmp(val, "FPM")) { + dconf->backend_type = BACKEND_FPM; + } + else { + return "ProxyFCGIBackendType requires one of the following arguments: " + "'GENERIC', 'FPM'"; + } + + return NULL; +} + + +static const char *cmd_setenv(cmd_parms *cmd, void *in_dconf, + const char *arg1, const char *arg2, + const char *arg3) +{ + fcgi_dirconf_t *dconf = in_dconf; + const char *err; + sei_entry *new; + const char *envvar = arg2; + + new = apr_array_push(dconf->env_fixups); + new->cond = ap_expr_parse_cmd(cmd, arg1, 0, &err, NULL); + if (err) { + return apr_psprintf(cmd->pool, "Could not parse expression \"%s\": %s", + arg1, err); + } + + if (envvar[0] == '!') { + /* Unset mode. */ + if (arg3) { + return apr_psprintf(cmd->pool, "Third argument (\"%s\") is not " + "allowed when using ProxyFCGISetEnvIf's unset " + "mode (%s)", arg3, envvar); + } + else if (!envvar[1]) { + /* i.e. someone tried to give us a name of just "!" */ + return "ProxyFCGISetEnvIf: \"!\" is not a valid variable name"; + } + + new->subst = NULL; + } + else { + /* Set mode. */ + if (!arg3) { + /* A missing expr-value should be treated as empty. */ + arg3 = ""; + } + + new->subst = ap_expr_parse_cmd(cmd, arg3, AP_EXPR_FLAG_STRING_RESULT, &err, NULL); + if (err) { + return apr_psprintf(cmd->pool, "Could not parse expression \"%s\": %s", + arg3, err); + } + } + + new->envname = envvar; + + return NULL; +} static void register_hooks(apr_pool_t *p) { proxy_hook_scheme_handler(proxy_fcgi_handler, NULL, NULL, APR_HOOK_FIRST); proxy_hook_canon_handler(proxy_fcgi_canon, NULL, NULL, APR_HOOK_FIRST); } +static const command_rec command_table[] = { + AP_INIT_TAKE1("ProxyFCGIBackendType", cmd_servertype, NULL, OR_FILEINFO, + "Specify the type of FastCGI server: 'Generic', 'FPM'"), + AP_INIT_TAKE23("ProxyFCGISetEnvIf", cmd_setenv, NULL, OR_FILEINFO, + "expr-condition env-name expr-value"), + { NULL } +}; + AP_DECLARE_MODULE(proxy_fcgi) = { STANDARD20_MODULE_STUFF, - NULL, /* create per-directory config structure */ - NULL, /* merge per-directory config structures */ + fcgi_create_dconf, /* create per-directory config structure */ + fcgi_merge_dconf, /* merge per-directory config structures */ NULL, /* create per-server config structure */ NULL, /* merge per-server config structures */ - NULL, /* command apr_table_t */ + command_table, /* command apr_table_t */ register_hooks /* register hooks */ };