From: Jeff Trawick Date: Mon, 21 Apr 2014 21:14:21 +0000 (+0000) Subject: Add module mod_ssl_ct, which provides an implementation of Certificate X-Git-Tag: 2.5.0-alpha~4272 X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=c51e4d9cbc1e6185aab143c2670410d3b94eb1e8;p=apache Add module mod_ssl_ct, which provides an implementation of Certificate Transparency (RFC 6962) for httpd. mod_ssl_ct requires OpenSSL 1.0.2 (in beta) and must be explicitly enabled via configure. Note that support/ctauditscts is purposefully not installed; it does not properly function due to a dependency on a certificate-transparency open source project tool which itself is not sufficiently complete at this time. git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/trunk@1588987 13f79535-47bb-0310-9956-ffa450edef68 --- diff --git a/CHANGES b/CHANGES index b8fde68cde..ff880bacc8 100644 --- a/CHANGES +++ b/CHANGES @@ -1,6 +1,9 @@ -*- coding: utf-8 -*- Changes with Apache 2.5.0 + *) Add module mod_ssl_ct, which provides an implementation of Certificate + Transparency (RFC 6962) for httpd. [Jeff Trawick] + *) mod_cache: Fix AH00784 errors on Windows when the the CacheLock directive is enabled. [Eric Covener] diff --git a/docs/manual/mod/mod_ssl_ct.xml b/docs/manual/mod/mod_ssl_ct.xml new file mode 100644 index 0000000000..12d489e22c --- /dev/null +++ b/docs/manual/mod/mod_ssl_ct.xml @@ -0,0 +1,299 @@ + + + + + + + + + +mod_ssl_ct +Implementation of Certificate Transparency (RFC 6962) + +Extension +mod_ssl_ct.c +ssl_ct_module + + +

This module provides an implementation of Certificate Transparency, in +conjunction with mod_ssl and command-line tools from the +certificate-transparency +open source project. The goal of Certificate Transparency is to expose the +use of server certificates which are trusted by browsers but were mistakenly +or maliciously issued. More information about Certificate Transparency is +available at +http://www.certificate-transparency.org/.

+ +

This implementation for Apache httpd provides these features for TLS +servers and proxies:

+ +
    +
  • Signed Certificate Timestamps (SCTs) can be obtained from logs + automatically and, in conjunction with any statically configured SCTs, sent + to aware clients in the ServerHello (during the handshake).
    • +
  • SCTs can be received by the proxy from backend servers in the ServerHello, + in a certificate extension, and/or within stapled OCSP responses; any SCTs + received can be partially validated on-line and optionally queued for off-line + audit.
    • +
  • The proxy can be configured to disallow communication with a backend + which does not provide an SCT which passes on-line validation.
    • +
+ +

Configuration information about logs can be defined statically in the web +server configuration or maintained in a Sqlite3 database. In the latter case, +mod_ssl_ct will reload the database periodically, so any +site-specific infrastructure for maintaining and propagating log configuration +information does not have to also restart httpd to make it take effect.

+ + + +CTAuditStorage +Existing directory where data for off-line audit will be stored +CTAuditStorage directory +none +server config + + +

The CTAuditStorage directive sets the name of a + directory where data will be stored for off-line audit. If directory + is not absolute then it is assumed to be relative to + DefaultRuntimeDir.

+ +

If this directive is not specified, data will not be stored for off-line + audit.

+ +

The directory will contain files named PID.tmp for + active child processes and files named PID.out for exited + child processes. These .out files are ready for off-line audit. + The experimental command ctauditscts (in the httpd source tree, not + currently installed) interfaces with certificate-transparency tools to + perform the audit.

+ + + + +CTLogClient +Location of certificate-transparency log client tool +CTLogClient executable +none +server config + + + +

executable is the full path to the log client tool, which is + normally file src/client/ct within the source tree of the + + certificate-transparency open source project.

+ +

An alternative implementation could be used to retrieve SCTs for a + server certificate as long as the command-line interface is equivalent.

+ + + + +CTLogConfigDB +Log configuration database supporting dynamic updates +CTLogConfigDB filename +none +server config + + +

The CTLogConfigDB directive sets the name of a database + containing configuration about known logs. If filename is not absolute + then it is assumed to be relative to + ServerRoot.

+ +

Refer to the documentation for the ctlogconfig program, + which manages the database.

+ + + + +CTMaxSCTAge +Maximum age of SCT obtained from a log, before it will be +refreshed +CTMaxSCTAge num-seconds +1 day +server config + + +

Server certificates with SCTs which are older than this maximum age will + be resubmitted to configured logs. Generally the log will return the same SCT + as before, but that is subject to log operation. SCTs will be refreshed as + necessary during normal server operation, with new SCTs returned to clients + as they become available.

+ + + + +CTProxyAwareness +Level of CT awareness and enforcement for a proxy + +CTProxyAwareness oblivious|aware|require +aware +server config +virtual host + + +

This directive controls awareness and checks for valid SCTs for a + proxy. Several options are available:

+ +
+
oblivious
+
The proxy will neither ask for nor examine SCTs. Certificate + Transparency processing for the proxy is completely disabled.
+ +
aware
+
The proxy will perform all appropriate Certificate Transparency + processing, such as asking for and examining SCTs. However, the + proxy will not disallow communication if the backend server does + not provide any valid SCTs.
+ +
require
+
The proxy will abort communication with the backend server if it + does not provide at least one SCT which passes on-line validation.
+
+ + + + + +CTSCTStorage +Existing directory where SCTs are managed +CTSCTStorage directory +none +server config + + + +

The CTSCTStorage directive sets the name of a + directory where SCTs and SCT lists will will be stored. If directory + is not absolute then it is assumed to be relative to + DefaultRuntimeDir.

+ +

A subdirectory for each server certificate contains information relative + to that certificate; the name of the subdirectory is the SHA-256 hash of the + certificate.

+ +

The certificate-specific directory contains SCTs retrieved from configured + logs, SCT lists prepared from statically configured SCTs and retrieved SCTs, + and other information used for managing SCTs.

+ + + + +CTServerHelloSCTLimit +Limit on number of SCTs that can be returned in +ServerHello +CTServerHelloSCTLimit limit +100 +server config + + + +

This directive can be used to limit the number of SCTs which can be + returned by a TLS server in ServerHello, in case the number of configured + logs and statically-defined SCTs is relatively high.

+ +

Typically only a few SCTs would be available, so this directive is only + needed in special circumstances.

+ +

The directive does not take into account SCTs which may be provided in + certificate extensions or in stapled OCSP responses.

+ + + + +CTStaticLogConfig +Static configuration of information about a log +CTStaticLogConfig log-id|- public-key-file|- +1|0|- min-timestamp|- max-timestamp|- +log-URL|- +none +server config + + + +

This directive is used to configure information about a particular log. + This directive is appropriate when configuration information changes rarely. + If dynamic configuration updates must be supported, refer to the + CTLogConfigDB directive.

+ +

Each of the six fields must be specified, but usually only a small + amount of information must be configured for each log; use - when no + information is available for the field. The fields are defined as follows:

+ +
+
log-id
+
This is the id of the log. The id is the SHA-256 hash of the log's + public key. In some cases it is appropriate and convenient to identify + the log by the id (hash), such as when configuring information regarding + the log's validity.
+ +
public-key-file
+
This is the name of a file containing the PEM encoding of the log's + public key. If the name is not absolute, then it is assumed to be relative + to ServerRoot. The public key is + required in order to check the signature of SCTs received by the proxy.
+ +
trust
+
This is a generic trust flag. Set this field to 0 to + distrust this log.
+ +
min-timestamp
+
SCTs received from this log by the proxy are invalid if the timestamp + is older than this value.
+ +
max-timestamp
+
SCTs received from this log by the proxy are invalid if the timestamp + is newer than this value.
+ +
log-URL
+
This is the URL of the log, for use in submitting server certificates + and in turn obtaining an SCT to be sent to clients. Each server certificate + will be submitted to all logs for which log-URL is configured.
+
+ + + + +CTStaticSCTs +Static configuration of one or more SCTs for a server certificate + +CTStaticSCTs certificate-pem-file sct-directory +none +server config + + + +

This directive is used to statically define one or more SCTs corresponding + to a server certificate. This mechanism can be used instead of or in + addition to dynamically obtaining SCTs from configured logs.

+ +

certificate-pem-file refers to the server certificate in PEM + format. If the name is not absolute, then it is assumed to be relative to + ServerRoot.

+ +

sct-directory must contain one or more files with extension + .sct, representing one or more SCTs corresponding to the + server certificate. If sct-directory is not absolute, then it is + assumed to be relative to ServerRoot.

+ + + + diff --git a/docs/manual/programs/ctlogconfig.xml b/docs/manual/programs/ctlogconfig.xml new file mode 100644 index 0000000000..c3ffa1886c --- /dev/null +++ b/docs/manual/programs/ctlogconfig.xml @@ -0,0 +1,165 @@ + + + + + + + + +Programs + +ctlogconfig - Certificate Transparency log configuration tool + + +

ctlogconfig is a tool for maintaining a log configuration + database, for use with mod_ssl_ct.

+ +

Refer to the examples below for typical use.

+ + +mod_ssl_ct + +
+ Synopsis +

+ ctlogconfig db-path dump +

+ +

+ ctlogconfig db-path configure-public-key + [ log-id|record-id ] + /path/to/public-key.pem +

+ +

+ ctlogconfig db-path configure-url + [ log-id|record-id ] + log-URL +

+ +

+ ctlogconfig db-path valid-time-range + log-id|record-id + min-timestamp max-timestamp +

+ +

+ ctlogconfig db-path trust + log-id|record-id +

+ +

+ ctlogconfig db-path distrust + log-id|record-id +

+ +

+ ctlogconfig db-path forget + log-id|record-id +

+ +
+ +
+ Sub-commands +
+
dump
+
Display configuration database contents. The record id shown in + the output of this sub-command can be used to identify the affected + record in other sub-commands.
+ +
configure-public-key
+
Add a log's public key to the database or set the public key for an + existing entry. The log's public key is needed to validate the signature + of SCTs received by a proxy from a backend server.
+ +
configure-url
+
Add a log's URL to the database or set the URL for an existing entry. + The log's URL is used when submitting server certificates to logs in + order to obtain SCTs to send to clients.
+ +
valid-time-range
+
Set the minimum valid time and/or the maximum valid time for a log. + SCTs from the log with timestamps outside of the valid range will not be + accepted. Use - for a time that is not being configured.
+ +
trust
+
Mark a log as trusted, which is the default setting. This sub-command + is used to reverse a distrust setting.
+ +
distrust
+
Mark a log as distrusted.
+ +
forget
+
Remove information about a log from the database.
+
+
+ +
+ Examples + +

Consider an Apache httpd instance which serves as a TLS server and a proxy. + The TLS server needs to obtain SCTs from a couple of known logs in order to + pass those to clients, and the proxy needs to be able to validate the signature + of SCTs received from backend servers.

+ +

First we'll configure the URLs for logs where server certificates are logged:

+ + + $ ctlogconfig /path/to/conf/log-config configure-url http://log1.example.com/
+ $ ctlogconfig /path/to/conf/log-config configure-url http://log2.example.com/
+ $ ctlogconfig /path/to/conf/log-config dump
+ Log entry:
+ Record 1
+ Log id : (not configured)
+ Public key file: (not configured)
+ URL : http://log1.example.com/
+ Time range : -INF to +INF
+
+ Log entry:
+ Record 2
+ Log id : (not configured)
+ Public key file: (not configured)
+ URL : http://log2.example.com/
+ Time range : -INF to +INF
+ + +

Next we'll set the public key of a log where the certificate of our only + backend server is published. In this case it is the log with URL + http://log2.example.com/ which has already been configured.

+ + + $ ctlogconfig /path/to/conf/log-config configure-public-key \\#2 /path/to/conf/log2-pub.pem
+ $ ctlogconfig /path/to/conf/log-config dump
+ Log entry:
+ Record 1
+ Log id : (not configured)
+ Public key file: (not configured)
+ URL : http://log1.example.com/
+ Time range : -INF to +INF
+
+ Log entry:
+ Record 2
+ Log id : (not configured)
+ Public key file: /path/to/conf/log2-pub.pem
+ URL : http://log2.example.com/
+ Time range : -INF to +INF
+ +
+ + diff --git a/modules/ssl/config.m4 b/modules/ssl/config.m4 index 45eeb43d9d..8b756b011c 100644 --- a/modules/ssl/config.m4 +++ b/modules/ssl/config.m4 @@ -52,6 +52,14 @@ APACHE_MODULE(ssl, [SSL/TLS support (mod_ssl)], $ssl_objs, , most, [ # Ensure that other modules can pick up mod_ssl.h APR_ADDTO(INCLUDES, [-I\$(top_srcdir)/$modpath_current]) +ssl_ct_objs="mod_ssl_ct.lo ssl_ct_log_config.lo ssl_ct_sct.lo ssl_ct_util.lo" +APACHE_MODULE(ssl_ct, [Support for Certificate Transparency (RFC 6962)], $ssl_ct_objs, , no, [ + dnl TODO: Check for OpenSSL >= 1.0.2 + if test "$enable_ssl" = "no"; then + AC_MSG_ERROR([mod_ssl_ct is dependent on mod_ssl, which is not enabled.]) + fi +]) + dnl # end of module specific part APACHE_MODPATH_FINISH diff --git a/modules/ssl/mod_ssl_ct.c b/modules/ssl/mod_ssl_ct.c new file mode 100644 index 0000000000..a2f9d3422a --- /dev/null +++ b/modules/ssl/mod_ssl_ct.c @@ -0,0 +1,2979 @@ +/* Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* + * Issues + * + * + Major limitations + * . ??? + * + * + Known low-level code kludges/problems + * . proxy: an httpd child process validates SCTs from a server only on the + * first time the data is received; but it could fail once due to invalid + * timestamp and succeed later after time elapses; fixit! + * . server: shouldn't have to read file of server SCTs on every handshake + * . split mod_ssl_ct.c into more pieces + * . research: Is it possible to send an SCT that is outside of the known + * valid interval for the log? + */ + +#if defined(WIN32) +#define HAVE_SCT_DAEMON_THREAD +#else +#define HAVE_SCT_DAEMON_CHILD +#endif + +#include + +#if defined(HAVE_SCT_DAEMON_CHILD) +#include +#endif + +#include "apr_version.h" +#if !APR_VERSION_AT_LEAST(1,5,0) +#error mod_ssl_ct requires APR 1.5.0 or later! (for apr_escape.h) +#endif + +#include "apr_escape.h" +#include "apr_global_mutex.h" +#include "apr_signal.h" +#include "apr_strings.h" +#include "apr_thread_rwlock.h" + +#include "apr_dbd.h" + +#include "httpd.h" +#include "http_config.h" +#include "http_core.h" +#include "http_log.h" +#include "http_main.h" +#include "http_protocol.h" +#include "mpm_common.h" +#include "util_mutex.h" +#include "ap_listen.h" +#include "ap_mpm.h" + +#if AP_NEED_SET_MUTEX_PERMS +#include "unixd.h" +#endif + +#include "mod_proxy.h" +#include "mod_ssl.h" +#include "mod_ssl_openssl.h" + +#include "ssl_ct_util.h" +#include "ssl_ct_sct.h" + +#include "openssl/x509v3.h" +#include "openssl/ocsp.h" + +#if OPENSSL_VERSION_NUMBER < 0x10002001L +#error "mod_ssl_ct requires OpenSSL 1.0.2-beta1 or later" +#endif + +#ifdef WIN32 +#define DOTEXE ".exe" +#else +#define DOTEXE "" +#endif + +#define STATUS_VAR "SSL_CT_PEER_STATUS" +#define STATUS_VAR_AWARE_VAL "peer-aware" +#define STATUS_VAR_UNAWARE_VAL "peer-unaware" + +#define PROXY_SCT_SOURCES_VAR "SSL_PROXY_SCT_SOURCES" + +#define DAEMON_NAME "SCT maintenance daemon" +#define DAEMON_THREAD_NAME DAEMON_NAME " thread" +#define SERVICE_THREAD_NAME "service thread" + +/** Limit on size of stored SCTs for a certificate (individual SCTs as well + * as size of all. + */ +#define MAX_SCTS_SIZE 10000 + +/** Limit on size of log URL list for a certificate + */ +#define MAX_LOGLIST_SIZE 1000 + +typedef struct ct_server_config { + apr_array_header_t *db_log_config; + apr_pool_t *db_log_config_pool; + apr_array_header_t *static_log_config; + apr_array_header_t *server_cert_info; /* ct_server_cert_info */ + apr_hash_t *static_cert_sct_dirs; + const char *sct_storage; + const char *audit_storage; + const char *ct_exe; + const char *log_config_fname; + apr_time_t max_sct_age; + int max_sh_sct; +#define PROXY_AWARENESS_UNSET -1 +#define PROXY_OBLIVIOUS 1 +#define PROXY_AWARE 2 /* default */ +#define PROXY_REQUIRE 3 + int proxy_awareness; +} ct_server_config; + +typedef struct ct_conn_config { + int peer_ct_aware; + /* proxy mode only */ + cert_chain *certs; + int server_cert_has_sct_list; + void *cert_sct_list; + apr_size_t cert_sct_list_size; + int serverhello_has_sct_list; + void *serverhello_sct_list; + apr_size_t serverhello_sct_list_size; + int ocsp_has_sct_list; + void *ocsp_sct_list; + apr_size_t ocsp_sct_list_size; + apr_array_header_t *all_scts; /* array of ct_sct_data */ +} ct_conn_config; + +typedef struct ct_server_cert_info { + const char *fingerprint; + const char *sct_dir; +} ct_server_cert_info; + +typedef struct ct_sct_data { + const void *data; + apr_uint16_t len; +} ct_sct_data; + +typedef struct ct_callback_info { + server_rec *s; + conn_rec *c; + ct_conn_config *conncfg; +} ct_callback_info; + +typedef struct ct_cached_server_data { + apr_status_t validation_result; +} ct_cached_server_data; + +/* the log configuration in use -- either db_log_config or static_log_config */ +static apr_array_header_t *active_log_config; + +module AP_MODULE_DECLARE_DATA ssl_ct_module; + +#define SSL_CT_MUTEX_TYPE "ssl-ct-sct-update" + +static apr_global_mutex_t *ssl_ct_sct_update; + +static int refresh_all_scts(server_rec *s_main, apr_pool_t *p, + apr_array_header_t *log_config); + +static apr_thread_t *service_thread; + +static apr_hash_t *cached_server_data; + +static const char *audit_fn_perm, *audit_fn_active; +static apr_file_t *audit_file; +static int audit_file_nonempty; +static apr_thread_mutex_t *audit_file_mutex; +static apr_thread_mutex_t *cached_server_data_mutex; +static apr_thread_rwlock_t *log_config_rwlock; + +#ifdef HAVE_SCT_DAEMON_CHILD + +/* The APR other-child API doesn't tell us how the daemon exited + * (SIGSEGV vs. exit(1)). The other-child maintenance function + * needs to decide whether to restart the daemon after a failure + * based on whether or not it exited due to a fatal startup error + * or something that happened at steady-state. This exit status + * is unlikely to collide with exit signals. + */ +#define DAEMON_STARTUP_ERROR 254 + +static int daemon_start(apr_pool_t *p, server_rec *main_server, apr_proc_t *procnew); +static server_rec *root_server = NULL; +static apr_pool_t *root_pool = NULL; +static pid_t daemon_pid; +static int daemon_should_exit = 0; + +#endif /* HAVE_SCT_DAEMON_CHILD */ + +static apr_pool_t *pdaemon = NULL; + +#ifdef HAVE_SCT_DAEMON_THREAD +static apr_thread_t *daemon_thread; +#endif /* HAVE_SCT_DAEMON_THREAD */ + +static const char *get_cert_fingerprint(apr_pool_t *p, const X509 *x) +{ + const EVP_MD *digest; + unsigned char md[EVP_MAX_MD_SIZE]; + unsigned int n; + digest = EVP_get_digestbyname("sha256"); + X509_digest(x, digest, md, &n); + + return apr_pescape_hex(p, md, n, 0); +} + +/* a server's SCT-related storage on disk: + * + * //servercerts.pem + * Concatenation of leaf certificate and any + * configured intermediate certificates + * + * //logs + * List of log URLs, one per line; this is + * used to recognize when a log URL configuration + * change makes our current SCT set invalid + * + * //AUTO_hostname_port_uri.sct + * SCT for cert with this fingerprint + * from this log (could be any number + * of these) + * + * //.sct + * (file is optional; could be any number + * of these; should not start with "AUTO_") + * Note that the administrator should store + * statically maintained SCTs in a different + * directory for the server certificate (specified + * by the CTStaticSCTs directive). A hypothetical + * external mechanism for maintaining SCTs following + * some other model could store them here for use + * by the server. + * + * //collated + * one or more SCTs ready to send + * (this is all that the web server + * processes care about) + * + * Additionally, the CTStaticSCTs directive specifies a certificate- + * specific directory of statically-maintained SCTs to be sent. + */ + +#define SERVERCERTS_BASENAME "servercerts.pem" +#define COLLATED_SCTS_BASENAME "collated" +#define LOGLIST_BASENAME "logs" +#define LOG_SCT_PREFIX "AUTO_" /* to distinguish from admin-created .sct + * files + */ + +static apr_status_t collate_scts(server_rec *s, apr_pool_t *p, + const char *cert_sct_dir, + const char *static_cert_sct_dir, + int max_sh_sct) +{ + /* Read the various .sct files and stick them together in a single file */ + apr_array_header_t *arr; + apr_status_t rv, tmprv; + apr_file_t *tmpfile; + apr_size_t bytes_written; + apr_uint16_t overall_len = 0; + char *tmp_collated_fn, *collated_fn; + const char *cur_sct_file; + const char * const *elts; + int i, scts_written = 0, skipped = 0; + + rv = ctutil_path_join(&collated_fn, cert_sct_dir, COLLATED_SCTS_BASENAME, p, s); + if (rv != APR_SUCCESS) { + return rv; + } + + /* Note: We rebuild the file that combines the SCTs every time this + * code runs, even if no individual SCTs are new (or at least + * re-fetched). + * That allows the admin to see the last processing by looking + * at the timestamp. + * Rechecking even if no SCTs are new allows SCTs which were not + * yet valid originally (just submitted to a log) to be used as + * soon as practical. + */ + tmp_collated_fn = apr_pstrcat(p, collated_fn, ".tmp", NULL); + + rv = apr_file_open(&tmpfile, tmp_collated_fn, + APR_FOPEN_WRITE|APR_FOPEN_CREATE|APR_FOPEN_TRUNCATE + |APR_FOPEN_BINARY|APR_FOPEN_BUFFERED, + APR_FPROT_OS_DEFAULT, p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "can't create %s", tmp_collated_fn); + return rv; + } + + /* stick a 0 len (for the list) at the start of the file; + * we'll have to patch that later + */ + rv = ctutil_file_write_uint16(s, tmpfile, overall_len); + if (rv != APR_SUCCESS) { + apr_file_close(tmpfile); + return rv; + } + + arr = NULL; /* Build list from scratch, creating array */ + rv = ctutil_read_dir(p, s, cert_sct_dir, "*.sct", &arr); + if (rv != APR_SUCCESS) { + apr_file_close(tmpfile); + return rv; + } + + if (static_cert_sct_dir) { + /* Add in any SCTs that the administrator has configured */ + rv = ctutil_read_dir(p, s, static_cert_sct_dir, "*.sct", &arr); + if (rv != APR_SUCCESS) { + apr_file_close(tmpfile); + return rv; + } + } + + elts = (const char * const *)arr->elts; + + for (i = 0; i < arr->nelts; i++) { + char *scts; + apr_size_t scts_size_wide; + apr_uint16_t scts_size; + sct_fields_t fields; + + cur_sct_file = elts[i]; + + ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s, + "Adding SCT from file %s", cur_sct_file); + + rv = ctutil_read_file(p, s, cur_sct_file, MAX_SCTS_SIZE, &scts, + &scts_size_wide); + if (rv != APR_SUCCESS) { + break; + } + ap_assert(scts_size_wide <= USHRT_MAX); + scts_size = (apr_uint16_t)scts_size_wide; + + rv = sct_parse(cur_sct_file, + s, (const unsigned char *)scts, scts_size, NULL, &fields); + if (rv != APR_SUCCESS) { + sct_release(&fields); + break; + } + + /* If the SCT has a timestamp in the future, it may have just been + * created by the log. + */ + if (fields.time > apr_time_now()) { + ap_log_error(APLOG_MARK, APLOG_WARNING, 0, s, + "SCT in file %s has timestamp in future (%s), skipping", + cur_sct_file, fields.timestr); + sct_release(&fields); + continue; + } + + sct_release(&fields); + + /* Only now do we know that the SCT is valid to send, so + * see if it has to be skipped by configured limit. + */ + if (scts_written >= max_sh_sct) { + skipped++; + continue; + } + + overall_len += scts_size + 2; /* include size header */ + + rv = ctutil_file_write_uint16(s, tmpfile, (apr_uint16_t)scts_size); + if (rv != APR_SUCCESS) { + break; + } + + rv = apr_file_write_full(tmpfile, scts, scts_size, &bytes_written); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "can't write %hu bytes to %s", + scts_size, tmp_collated_fn); + break; + } + + scts_written++; + } + + if (rv == APR_SUCCESS && skipped) { + ap_log_error(APLOG_MARK, APLOG_INFO, 0, s, + "SCTs sent in ServerHello are limited to %d by " + "CTServerHelloSCTLimit (ignoring %d)", + max_sh_sct, + skipped); + } + + if (rv == APR_SUCCESS) { + apr_off_t offset = 0; + + rv = apr_file_seek(tmpfile, APR_SET, &offset); + if (rv == APR_SUCCESS) { + rv = ctutil_file_write_uint16(s, tmpfile, overall_len); + } + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "could not write the SCT list length at the start of the file"); + } + } + + tmprv = apr_file_close(tmpfile); + if (tmprv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, tmprv, s, + "error flushing and closing %s", tmp_collated_fn); + if (rv == APR_SUCCESS) { + rv = tmprv; + } + } + + if (rv == APR_SUCCESS && scts_written) { + int replacing = ctutil_file_exists(p, collated_fn); + + if (replacing) { + if ((rv = apr_global_mutex_lock(ssl_ct_sct_update)) != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "global mutex lock failed"); + return rv; + } + apr_file_remove(collated_fn, p); + } + rv = apr_file_rename(tmp_collated_fn, collated_fn, p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_CRIT, rv, s, + "couldn't rename %s to %s, no SCTs to send for now", + tmp_collated_fn, collated_fn); + } + if (replacing) { + if ((tmprv = apr_global_mutex_unlock(ssl_ct_sct_update)) != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, tmprv, s, + "global mutex unlock failed"); + if (rv == APR_SUCCESS) { + rv = tmprv; + } + } + } + } + + return rv; +} + +static const char *url_to_fn(apr_pool_t *p, const apr_uri_t *log_url) +{ + char *fn = apr_psprintf(p, LOG_SCT_PREFIX "%s_%s_%s.sct", + log_url->hostname, log_url->port_str, log_url->path); + char *ch; + + ch = fn; + while (*ch) { + switch(*ch) { + /* chars that shouldn't be used in a filename */ + case ':': + case '/': + case '\\': + case '*': + case '?': + case '<': + case '>': + case '"': + case '|': + *ch = '-'; + } + ++ch; + } + return fn; +} + +static apr_status_t submission(server_rec *s, apr_pool_t *p, const char *ct_exe, + const apr_uri_t *log_url, const char *cert_file, + const char *sct_fn) +{ + apr_status_t rv; + const char *args[8]; + int i; + + i = 0; + args[i++] = ct_exe; + args[i++] = apr_pstrcat(p, "--ct_server=", log_url->hostinfo, NULL); + args[i++] = "--http_log"; + args[i++] = "--logtostderr"; + args[i++] = apr_pstrcat(p, "--ct_server_submission=", cert_file, NULL); + args[i++] = apr_pstrcat(p, "--ct_server_response_out=", sct_fn, NULL); + args[i++] = "upload"; + args[i++] = NULL; + ap_assert(i == sizeof args / sizeof args[0]); + + rv = ctutil_run_to_log(p, s, args, "log client"); + + return rv; +} + +static apr_status_t fetch_sct(server_rec *s, apr_pool_t *p, + const char *cert_file, + const char *cert_sct_dir, + const apr_uri_t *log_url, + const char *ct_exe, apr_time_t max_sct_age) +{ + apr_status_t rv; + char *sct_fn; + apr_finfo_t finfo; + const char *log_url_basename; + + log_url_basename = url_to_fn(p, log_url); + + rv = ctutil_path_join(&sct_fn, cert_sct_dir, log_url_basename, p, s); + if (rv != APR_SUCCESS) { + return rv; + } + + rv = apr_stat(&finfo, sct_fn, APR_FINFO_MTIME, p); + if (rv == APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s, + "Found SCT for %s in %s", + cert_file, sct_fn); + + if (finfo.mtime + max_sct_age < apr_time_now()) { + ap_log_error(APLOG_MARK, APLOG_INFO, 0, s, + "SCT for %s is older than %d seconds, must refresh", + cert_file, + (int)(apr_time_sec(max_sct_age))); + } + else { + return APR_SUCCESS; + } + } + else { + ap_log_error(APLOG_MARK, APLOG_INFO, + /* no need to print error string for file-not-found err */ + APR_STATUS_IS_ENOENT(rv) ? 0 : rv, + s, + "Did not find SCT for %s in %s, must fetch", + cert_file, sct_fn); + } + + rv = submission(s, p, ct_exe, log_url, cert_file, sct_fn); + + return rv; +} + +static apr_status_t record_log_urls(server_rec *s, apr_pool_t *p, + const char *listfile, apr_array_header_t *log_config) +{ + apr_file_t *f; + apr_status_t rv, tmprv; + ct_log_config **config_elts; + int i; + + rv = apr_file_open(&f, listfile, + APR_FOPEN_WRITE|APR_FOPEN_CREATE|APR_FOPEN_TRUNCATE + |APR_FOPEN_BUFFERED, + APR_FPROT_OS_DEFAULT, p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "can't create %s", listfile); + return rv; + } + + config_elts = (ct_log_config **)log_config->elts; + + for (i = 0; i < log_config->nelts; i++) { + if (!config_elts[i]->uri_str) { + continue; + } + if (!log_valid_for_sent_sct(config_elts[i])) { + continue; + } + rv = apr_file_puts(config_elts[i]->uri_str, f); + if (rv == APR_SUCCESS) { + rv = apr_file_puts("\n", f); + } + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "error writing to %s", listfile); + break; + } + } + + tmprv = apr_file_close(f); + if (tmprv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, tmprv, s, + "error flushing and closing %s", listfile); + if (rv == APR_SUCCESS) { + rv = tmprv; + } + } + + return rv; +} + +static int uri_in_config(const char *needle, const apr_array_header_t *haystack) +{ + ct_log_config **elts; + int i; + + elts = (ct_log_config **)haystack->elts; + for (i = 0; i < haystack->nelts; i++) { + if (!elts[i]->uri_str) { + continue; + } + if (!log_valid_for_sent_sct(elts[i])) { + continue; + } + if (!strcmp(needle, elts[i]->uri_str)) { + return 1; + } + } + + return 0; +} + +static apr_status_t update_log_list_for_cert(server_rec *s, apr_pool_t *p, + const char *cert_sct_dir, + apr_array_header_t *log_config) +{ + apr_array_header_t *old_urls; + apr_size_t contents_size; + apr_status_t rv; + char *contents, *listfile; + + /* The set of logs can change, and we need to remove SCTs retrieved + * from logs that we no longer trust. To track changes we'll use a + * file in the directory for the server certificate. + * + * (When can the set change? Right now they can only change at [re]start, + * but in the future we should be able to find the set of trusted logs + * dynamically.) + */ + + rv = ctutil_path_join(&listfile, cert_sct_dir, LOGLIST_BASENAME, p, s); + if (rv != APR_SUCCESS) { + return rv; + } + + if (ctutil_file_exists(p, listfile)) { + char **elts; + int i; + + rv = ctutil_read_file(p, s, listfile, MAX_LOGLIST_SIZE, &contents, &contents_size); + if (rv != APR_SUCCESS) { + return rv; + } + + ctutil_buffer_to_array(p, contents, contents_size, &old_urls); + + elts = (char **)old_urls->elts; + for (i = 0; i < old_urls->nelts; i++) { + if (!uri_in_config(elts[i], log_config)) { + char *sct_for_log; + int exists; + apr_uri_t uri; + + rv = apr_uri_parse(p, elts[i], &uri); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_CRIT, rv, s, + "unparseable log URL %s in file %s - ignoring", + elts[i], listfile); + /* some garbage in the file? can't map to an auto-maintained SCT, + * so just skip it + */ + continue; + } + + rv = ctutil_path_join(&sct_for_log, cert_sct_dir, url_to_fn(p, &uri), p, s); + ap_assert(rv == APR_SUCCESS); + exists = ctutil_file_exists(p, sct_for_log); + + ap_log_error(APLOG_MARK, + exists ? APLOG_NOTICE : APLOG_DEBUG, 0, s, + "Log %s is no longer enabled%s", + elts[i], + exists ? ", removing SCT" : ", no SCT was present"); + + if (exists) { + rv = apr_file_remove(sct_for_log, p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_CRIT, rv, s, + "can't remove SCT %s from previously trusted log %s", + sct_for_log, elts[i]); + return rv; + } + } + } + } + } + else { + /* can't tell what was trusted before; just remove everything + * that was created automatically + */ + apr_array_header_t *arr; + const char * const *elts; + int i; + + ap_log_error(APLOG_MARK, APLOG_WARNING, 0, s, + "List of previous logs doesn't exist (%s), removing previously obtained SCTs", + listfile); + + arr = NULL; /* Build list from scratch, creating array */ + rv = ctutil_read_dir(p, s, cert_sct_dir, LOG_SCT_PREFIX "*.sct", &arr); + if (rv != APR_SUCCESS) { + return rv; + } + + elts = (const char * const *)arr->elts; + for (i = 0; i < arr->nelts; i++) { + const char *cur_sct_file = elts[i]; + + ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s, + "Removing %s", cur_sct_file); + + rv = apr_file_remove(cur_sct_file, p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_CRIT, rv, s, + "can't remove %s", cur_sct_file); + } + } + } + + if (rv == APR_SUCCESS) { + rv = record_log_urls(s, p, listfile, log_config); + } + + return rv; +} + +static apr_status_t refresh_scts_for_cert(server_rec *s, apr_pool_t *p, + const char *cert_sct_dir, + const char *static_cert_sct_dir, + apr_array_header_t *log_config, + const char *ct_exe, + apr_time_t max_sct_age, + int max_sh_sct) +{ + apr_status_t rv; + ct_log_config **config_elts; + char *cert_fn; + int i; + + rv = ctutil_path_join(&cert_fn, cert_sct_dir, SERVERCERTS_BASENAME, p, s); + if (rv != APR_SUCCESS) { + return rv; + } + + config_elts = (ct_log_config **)log_config->elts; + + rv = update_log_list_for_cert(s, p, cert_sct_dir, log_config); + if (rv != APR_SUCCESS) { + return rv; + } + + for (i = 0; i < log_config->nelts; i++) { + if (!config_elts[i]->url) { + continue; + } + if (!log_valid_for_sent_sct(config_elts[i])) { + continue; + } + rv = fetch_sct(s, p, cert_fn, + cert_sct_dir, + &config_elts[i]->uri, + ct_exe, + max_sct_age); + if (rv != APR_SUCCESS) { + return rv; + } + } + + rv = collate_scts(s, p, cert_sct_dir, static_cert_sct_dir, max_sh_sct); + if (rv != APR_SUCCESS) { + return rv; + } + + return rv; +} + +static void *run_service_thread(apr_thread_t *me, void *data) +{ + server_rec *s = data; + ct_server_config *sconf = ap_get_module_config(s->module_config, + &ssl_ct_module); + int mpmq_s; + apr_status_t rv; + int count = 0; + + ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s, + SERVICE_THREAD_NAME " started"); + + while (1) { + if ((rv = ap_mpm_query(AP_MPMQ_MPM_STATE, &mpmq_s)) != APR_SUCCESS) { + break; + } + if (mpmq_s == AP_MPMQ_STOPPING) { + break; + } + apr_sleep(apr_time_from_sec(1)); + if (++count >= 30) { + count = 0; + if (sconf->db_log_config) { + /* Reload log config DB */ + ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s, + SERVICE_THREAD_NAME " - reloading config"); + ap_assert(apr_thread_rwlock_wrlock(log_config_rwlock) == 0); + active_log_config = NULL; + apr_pool_clear(sconf->db_log_config_pool); + sconf->db_log_config = + apr_array_make(sconf->db_log_config_pool, 2, + sizeof(ct_log_config *)); + rv = read_config_db(sconf->db_log_config_pool, + s, sconf->log_config_fname, + sconf->db_log_config); + ap_assert(apr_thread_rwlock_unlock(log_config_rwlock) == 0); + if (rv != APR_SUCCESS) { + /* specific issue already logged */ + ap_log_error(APLOG_MARK, APLOG_CRIT, 0, s, + SERVICE_THREAD_NAME " - no active configuration until " + "log config DB is corrected"); + } + else { + active_log_config = sconf->db_log_config; + } + } + } + } + + ap_log_error(APLOG_MARK, APLOG_DEBUG, rv, s, + SERVICE_THREAD_NAME " exiting"); + + return NULL; +} + +static apr_status_t wait_for_thread(void *data) +{ + apr_thread_t *thd = data; + apr_status_t retval; + + apr_thread_join(&retval, thd); + return APR_SUCCESS; +} + +static void sct_daemon_cycle(ct_server_config *sconf, server_rec *s_main, + apr_pool_t *ptemp, const char *daemon_name) +{ + apr_status_t rv; + + if (sconf->db_log_config) { /* not using static config */ + ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s_main, + "%s - reloading config", daemon_name); + apr_pool_clear(sconf->db_log_config_pool); + active_log_config = NULL; + sconf->db_log_config = + apr_array_make(sconf->db_log_config_pool, 2, + sizeof(ct_log_config *)); + rv = read_config_db(sconf->db_log_config_pool, + s_main, sconf->log_config_fname, + sconf->db_log_config); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_CRIT, 0, s_main, + "%s - no active configuration until " + "log config DB is corrected", daemon_name); + return; + } + active_log_config = sconf->db_log_config; + } + ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s_main, + "%s - refreshing SCTs as needed", daemon_name); + rv = refresh_all_scts(s_main, ptemp, active_log_config); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s_main, + "%s - SCT refresh failed; will try again later", + daemon_name); + } +} + +#ifdef HAVE_SCT_DAEMON_CHILD + +static void daemon_signal_handler(int sig) +{ + if (sig == SIGHUP) { + ++daemon_should_exit; + } +} + +#if APR_HAS_OTHER_CHILD +static void daemon_maint(int reason, void *data, apr_wait_t status) +{ + apr_proc_t *proc = data; + int mpm_state; + int stopping; + + switch (reason) { + case APR_OC_REASON_DEATH: + apr_proc_other_child_unregister(data); + /* If apache is not terminating or restarting, + * restart the daemon + */ + stopping = 1; /* if MPM doesn't support query, + * assume we shouldn't restart daemon + */ + if (ap_mpm_query(AP_MPMQ_MPM_STATE, &mpm_state) == APR_SUCCESS && + mpm_state != AP_MPMQ_STOPPING) { + stopping = 0; + } + if (!stopping) { + if (status == DAEMON_STARTUP_ERROR) { + ap_log_error(APLOG_MARK, APLOG_CRIT, 0, ap_server_conf, APLOGNO(01238) + DAEMON_NAME " failed to initialize"); + } + else { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, ap_server_conf, APLOGNO(01239) + DAEMON_NAME " process died, restarting"); + daemon_start(root_pool, root_server, proc); + } + } + break; + case APR_OC_REASON_RESTART: + /* don't do anything; server is stopping or restarting */ + apr_proc_other_child_unregister(data); + break; + case APR_OC_REASON_LOST: + /* Restart the child cgid daemon process */ + apr_proc_other_child_unregister(data); + daemon_start(root_pool, root_server, proc); + break; + case APR_OC_REASON_UNREGISTER: + /* we get here when pcgi is cleaned up; pcgi gets cleaned + * up when pconf gets cleaned up + */ + kill(proc->pid, SIGHUP); /* send signal to daemon telling it to die */ + break; + } +} +#endif + +static int sct_daemon(server_rec *s_main) +{ + apr_status_t rv; + apr_pool_t *ptemp; + ct_server_config *sconf = ap_get_module_config(s_main->module_config, + &ssl_ct_module); + int rc; + + /* Ignoring SIGCHLD results in errno ECHILD returned from apr_proc_wait(). + * apr_signal(SIGCHLD, SIG_IGN); + */ + apr_signal(SIGHUP, daemon_signal_handler); + + /* Close our copy of the listening sockets */ + ap_close_listeners(); + + rv = apr_global_mutex_child_init(&ssl_ct_sct_update, + apr_global_mutex_lockfile(ssl_ct_sct_update), pdaemon); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_CRIT, rv, root_server, + "could not initialize " SSL_CT_MUTEX_TYPE + " mutex in " DAEMON_NAME); + return DAEMON_STARTUP_ERROR; + } + + if (!geteuid()) { + /* Fix up permissions of the directories written to by the daemon + */ + int i; + apr_array_header_t *subdirs = apr_array_make(pdaemon, 5, sizeof(char *)); + + *(const char **)apr_array_push(subdirs) = sconf->sct_storage; + if (sconf->audit_storage) { + *(const char **)apr_array_push(subdirs) = sconf->audit_storage; + } + + rv = ctutil_read_dir(pdaemon, root_server, sconf->sct_storage, "*", + &subdirs); + if (rv == APR_SUCCESS && subdirs->nelts > 0) { + const char * const *elts = (const char * const *)subdirs->elts; + + for (i = 0; i < subdirs->nelts; i++) { + if (elts[i] && chown(elts[i], ap_unixd_config.user_id, + ap_unixd_config.group_id) < 0) { + ap_log_error(APLOG_MARK, APLOG_ERR, errno, root_server, + "Couldn't change owner or group of directory %s", + elts[i]); + return errno; + } + } + } + else { + ap_log_error(APLOG_MARK, APLOG_WARNING, rv, root_server, + "Did not read any entries from %s (no server certificate?)", + sconf->sct_storage); + } + } + + /* if running as root, switch to configured user/group */ + if ((rc = ap_run_drop_privileges(pdaemon, ap_server_conf)) != 0) { + return rc; + } + + /* ptemp - temporary pool for refresh cycles */ + apr_pool_create(&ptemp, pdaemon); + + while (!daemon_should_exit) { + sct_daemon_cycle(sconf, s_main, ptemp, DAEMON_NAME); + apr_sleep(apr_time_from_sec(30)); /* SIGHUP at restart/stop will break out */ + } + + ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s_main, + DAEMON_NAME " - exiting"); + + return 0; +} + +static int daemon_start(apr_pool_t *p, server_rec *main_server, + apr_proc_t *procnew) +{ + daemon_should_exit = 0; /* clear setting from previous generation */ + if ((daemon_pid = fork()) < 0) { + ap_log_error(APLOG_MARK, APLOG_ERR, errno, main_server, + "Couldn't create " DAEMON_NAME " process"); + return DECLINED; + } + else if (daemon_pid == 0) { + if (pdaemon == NULL) { + apr_pool_create(&pdaemon, p); + } + exit(sct_daemon(main_server) > 0 ? DAEMON_STARTUP_ERROR : -1); + } + procnew->pid = daemon_pid; + procnew->err = procnew->in = procnew->out = NULL; + apr_pool_note_subprocess(p, procnew, APR_KILL_AFTER_TIMEOUT); +#if APR_HAS_OTHER_CHILD + apr_proc_other_child_register(procnew, daemon_maint, procnew, NULL, p); +#endif + return OK; +} +#endif /* HAVE_SCT_DAEMON_CHILD */ + +#ifdef HAVE_SCT_DAEMON_THREAD +static void *sct_daemon_thread(apr_thread_t *me, void *data) +{ + server_rec *s = data; + ct_server_config *sconf = ap_get_module_config(s->module_config, + &ssl_ct_module); + int mpmq_s; + apr_pool_t *ptemp; + apr_status_t rv; + int count = 0; + + ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s, + DAEMON_THREAD_NAME " started"); + + /* ptemp - temporary pool for refresh cycles */ + apr_pool_create(&ptemp, pdaemon); + + while (1) { + if ((rv = ap_mpm_query(AP_MPMQ_MPM_STATE, &mpmq_s)) != APR_SUCCESS) { + break; + } + if (mpmq_s == AP_MPMQ_STOPPING) { + break; + } + apr_sleep(apr_time_from_sec(1)); + if (++count >= 30) { + count = 0; + sct_daemon_cycle(sconf, s, ptemp, DAEMON_THREAD_NAME); + } + } + + ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s, + DAEMON_THREAD_NAME " - exiting"); + + return NULL; +} + +static int daemon_thread_start(apr_pool_t *pconf, server_rec *s_main) +{ + apr_status_t rv; + + apr_pool_create(&pdaemon, pconf); + rv = apr_thread_create(&daemon_thread, NULL, sct_daemon_thread, s_main, + pconf); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_CRIT, rv, s_main, + "could not create " DAEMON_THREAD_NAME " in parent"); + return HTTP_INTERNAL_SERVER_ERROR; + } + + apr_pool_cleanup_register(pconf, daemon_thread, wait_for_thread, + apr_pool_cleanup_null); + + return OK; +} +#endif /* HAVE_SCT_DAEMON_THREAD */ + +static apr_status_t ssl_ct_mutex_remove(void *data) +{ + apr_global_mutex_destroy(ssl_ct_sct_update); + ssl_ct_sct_update = NULL; + return APR_SUCCESS; +} + +static int refresh_all_scts(server_rec *s_main, apr_pool_t *p, + apr_array_header_t *log_config) +{ + apr_hash_t *already_processed; + apr_status_t rv = APR_SUCCESS; + server_rec *s; + + already_processed = apr_hash_make(p); + + s = s_main; + while (s) { + ct_server_config *sconf = ap_get_module_config(s->module_config, + &ssl_ct_module); + int i; + const ct_server_cert_info *cert_info_elts; + + if (sconf && sconf->server_cert_info) { + cert_info_elts = + (const ct_server_cert_info *)sconf->server_cert_info->elts; + for (i = 0; i < sconf->server_cert_info->nelts; i++) { + /* we may have already processed this cert for another + * server_rec + */ + if (!apr_hash_get(already_processed, cert_info_elts[i].sct_dir, + APR_HASH_KEY_STRING)) { + const char *static_cert_sct_dir = + apr_hash_get(sconf->static_cert_sct_dirs, + cert_info_elts[i].fingerprint, + APR_HASH_KEY_STRING); + + apr_hash_set(already_processed, cert_info_elts[i].sct_dir, + APR_HASH_KEY_STRING, "done"); + rv = refresh_scts_for_cert(s_main, p, + cert_info_elts[i].sct_dir, + static_cert_sct_dir, + log_config, + sconf->ct_exe, + sconf->max_sct_age, + sconf->max_sh_sct); + if (rv != APR_SUCCESS) { + return rv; + } + } + } + } + + s = s->next; + } + + return rv; +} + +static int num_server_certs(server_rec *s_main) +{ + int num = 0; + server_rec *s; + + s = s_main; + while (s) { + ct_server_config *sconf = ap_get_module_config(s->module_config, + &ssl_ct_module); + + if (sconf && sconf->server_cert_info) { + num += sconf->server_cert_info->nelts; + } + s = s->next; + } + + return num; +} + +static int ssl_ct_post_config(apr_pool_t *pconf, apr_pool_t *plog, + apr_pool_t *ptemp, server_rec *s_main) +{ + ct_server_config *sconf = ap_get_module_config(s_main->module_config, + &ssl_ct_module); + apr_status_t rv; +#ifdef HAVE_SCT_DAEMON_CHILD + apr_proc_t *procnew = NULL; + const char *userdata_key = "sct_daemon_init"; + + root_server = s_main; + root_pool = pconf; + + procnew = ap_retained_data_get(userdata_key); + if (!procnew) { + procnew = ap_retained_data_create(userdata_key, sizeof(*procnew)); + procnew->pid = -1; + procnew->err = procnew->in = procnew->out = NULL; + } +#endif /* HAVE_SCT_DAEMON_CHILD */ + + if (num_server_certs(s_main) == 0) { + /* Theoretically this module could operate in a proxy-only + * configuration, where httpd does not act as a TLS server but proxy is + * configured as a TLS client. That isn't currently implemented. + */ + ap_log_error(APLOG_MARK, APLOG_ERR, 0, s_main, + "No server certificates were found."); + ap_log_error(APLOG_MARK, APLOG_ERR, 0, s_main, + "mod_ssl_ct only supports configurations with a TLS server."); + return HTTP_INTERNAL_SERVER_ERROR; + } + + rv = ap_global_mutex_create(&ssl_ct_sct_update, NULL, + SSL_CT_MUTEX_TYPE, NULL, s_main, pconf, 0); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s_main, + "could not create global mutex"); + return HTTP_INTERNAL_SERVER_ERROR; + } + + apr_pool_cleanup_register(pconf, (void *)s_main, ssl_ct_mutex_remove, + apr_pool_cleanup_null); + + if (sconf->log_config_fname) { + if (!sconf->db_log_config) { + /* log config db in separate pool that can be cleared */ + apr_pool_create(&sconf->db_log_config_pool, pconf); + sconf->db_log_config = + apr_array_make(sconf->db_log_config_pool, 2, + sizeof(ct_log_config *)); + } + rv = read_config_db(sconf->db_log_config_pool, + s_main, sconf->log_config_fname, + sconf->db_log_config); + if (rv != APR_SUCCESS) { + return HTTP_INTERNAL_SERVER_ERROR; + } + } + + if (sconf->static_log_config && sconf->db_log_config) { + if (sconf->static_log_config->nelts > 0 + && sconf->db_log_config->nelts > 0) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, s_main, + "Either the static log configuration or the db log " + "configuration must be empty"); + return HTTP_INTERNAL_SERVER_ERROR; + } + } + + if (sconf->static_log_config && sconf->static_log_config->nelts > 0) { + active_log_config = sconf->static_log_config; + } + else if (sconf->db_log_config && sconf->db_log_config->nelts > 0) { + active_log_config = sconf->db_log_config; + } + else { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, s_main, + "No non-empty log configuration was provided"); + return HTTP_INTERNAL_SERVER_ERROR; + } + + /* Ensure that we already have, or can fetch, fresh SCTs for each + * certificate. If so, start the daemon to maintain these and let + * startup continue. (Otherwise abort startup.) + * + * Except when we start up as root. We don't want to run external + * certificate-transparency tools as root, and we don't want to have + * to fix up the permissions of everything we created so that the + * SCT maintenance daemon can continue to maintain the SCTs as the + * configured User/Group. + */ + +#if AP_NEED_SET_MUTEX_PERMS /* Unix :) */ + if (!geteuid()) { /* root */ + ap_log_error(APLOG_MARK, APLOG_INFO, 0, s_main, + "SCTs will be fetched from configured logs as needed " + "and may not be available immediately"); + } + else { +#endif + rv = refresh_all_scts(s_main, pconf, active_log_config); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s_main, + "refresh_all_scts() failed"); + return HTTP_INTERNAL_SERVER_ERROR; + } +#if AP_NEED_SET_MUTEX_PERMS + } +#endif + +#ifdef HAVE_SCT_DAEMON_CHILD + if (ap_state_query(AP_SQ_MAIN_STATE) != AP_SQ_MS_CREATE_PRE_CONFIG) { + int ret = daemon_start(pconf, s_main, procnew); + if (ret != OK) { + return ret; + } + } +#endif /* HAVE_SCT_DAEMON_CHILD */ + +#ifdef HAVE_SCT_DAEMON_THREAD + /* WIN32-ism: ensure this is the parent by checking AP_PARENT_PID, + * which is only set in WinNT children. + */ + if (ap_state_query(AP_SQ_MAIN_STATE) != AP_SQ_MS_CREATE_PRE_CONFIG + && !getenv("AP_PARENT_PID")) { + int ret = daemon_thread_start(pconf, s_main); + if (ret != OK) { + return ret; + } + } +#endif /* HAVE_SCT_DAEMON_THREAD */ + + return OK; +} + +static int ssl_ct_check_config(apr_pool_t *pconf, apr_pool_t *plog, + apr_pool_t *ptemp, server_rec *s_main) +{ + ct_server_config *sconf = ap_get_module_config(s_main->module_config, + &ssl_ct_module); + + if (!sconf->sct_storage) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, s_main, + "Directive CTSCTStorage is required"); + return HTTP_INTERNAL_SERVER_ERROR; + } + + if (!sconf->audit_storage) { + /* umm, hard to tell if needed... must have server with + * SSL proxy enabled and server-specific-sconf->proxy_awareness + * != PROXY_OBLIVIOUS... + */ + ap_log_error(APLOG_MARK, APLOG_INFO, 0, s_main, + "Directive CTAuditStorage isn't set; proxy will not save " + "data for off-line audit"); + } + + if (!sconf->ct_exe) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, s_main, + "Directive CTLogClient is required"); + return HTTP_INTERNAL_SERVER_ERROR; + } + + if (sconf->log_config_fname) { + const char *msg = NULL; + if (!log_config_readable(pconf, sconf->log_config_fname, &msg)) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, s_main, + "Log config file %s cannot be read", + sconf->log_config_fname); + if (msg) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, s_main, + "%s", msg); + } + return HTTP_INTERNAL_SERVER_ERROR; + } + } + + return OK; +} + +static apr_status_t read_scts(apr_pool_t *p, const char *fingerprint, + const char *sct_dir, + server_rec *s, + char **scts, apr_size_t *scts_len) +{ + apr_status_t rv, tmprv; + char *cert_dir, *sct_fn; + + rv = ctutil_path_join(&cert_dir, sct_dir, fingerprint, p, s); + if (rv != APR_SUCCESS) { + return rv; + } + + rv = ctutil_path_join(&sct_fn, cert_dir, COLLATED_SCTS_BASENAME, p, s); + if (rv != APR_SUCCESS) { + return rv; + } + + if ((rv = apr_global_mutex_lock(ssl_ct_sct_update)) != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "global mutex lock failed"); + return rv; + } + + rv = ctutil_read_file(p, s, sct_fn, MAX_SCTS_SIZE, scts, scts_len); + + if ((tmprv = apr_global_mutex_unlock(ssl_ct_sct_update)) != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, tmprv, s, + "global mutex unlock failed"); + } + + return rv; +} + +static void look_for_server_certs(server_rec *s, SSL_CTX *ctx, const char *sct_dir) +{ + ct_server_config *sconf = ap_get_module_config(s->module_config, + &ssl_ct_module); + apr_pool_t *p = s->process->pool; + apr_status_t rv; + FILE *concat; + X509 *x; + STACK_OF(X509) *chain; + int i, rc; + char *cert_sct_dir, *servercerts_pem; + const char *fingerprint; + ct_server_cert_info *cert_info; + + sconf->server_cert_info = apr_array_make(p, 2, sizeof(ct_server_cert_info)); + + rc = SSL_CTX_set_current_cert(ctx, SSL_CERT_SET_FIRST); + while (rc) { + x = SSL_CTX_get0_certificate(ctx); /* UNDOC */ + if (x) { + fingerprint = get_cert_fingerprint(s->process->pool, x); + rv = ctutil_path_join(&cert_sct_dir, sct_dir, fingerprint, p, s); + ap_assert(rv == APR_SUCCESS); + + if (!ctutil_dir_exists(p, cert_sct_dir)) { + rv = apr_dir_make(cert_sct_dir, APR_FPROT_OS_DEFAULT, p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "can't create directory %s", + cert_sct_dir); + ap_assert(rv == APR_SUCCESS); + } + } + + rv = ctutil_path_join(&servercerts_pem, cert_sct_dir, + SERVERCERTS_BASENAME, p, s); + ap_assert(rv == APR_SUCCESS); + + rv = ctutil_fopen(servercerts_pem, "wb", &concat); + ap_assert(rv == APR_SUCCESS); + + ap_assert(1 == PEM_write_X509(concat, x)); /* leaf */ + + chain = NULL; + + /* Not this: SSL_CTX_get0_chain_certs(ctx, &chain); + * + * See this thread: + * http://mail-archives.apache.org/mod_mbox/httpd-dev/ + * 201402.mbox/%3CCAKUrXK5-2_Sg8FokxBP8nW7tmSuTZZWL-%3 + * DBDhNnwyK-Z4dmQiQ%40mail.gmail.com%3E + */ + SSL_CTX_get_extra_chain_certs(ctx, &chain); /* UNDOC */ + + if (chain) { + for (i = 0; i < sk_X509_num(chain); i++) { /* UNDOC */ + X509 *x = sk_X509_value(chain, i); /* UNDOC */ + ap_assert(1 == PEM_write_X509(concat, x)); + } + } + ap_assert(0 == fclose(concat)); + + ap_log_error(APLOG_MARK, APLOG_INFO, 0, s, + "wrote server cert and chain to %s", servercerts_pem); + + cert_info = (ct_server_cert_info *)apr_array_push(sconf->server_cert_info); + cert_info->sct_dir = cert_sct_dir; + cert_info->fingerprint = fingerprint; + } + else { + ap_log_error(APLOG_MARK, APLOG_WARNING, 0, s, + "could not find leaf certificate"); + } + rc = SSL_CTX_set_current_cert(ctx, SSL_CERT_SET_NEXT); + } +} + +static ct_conn_config *get_conn_config(conn_rec *c) +{ + ct_conn_config *conncfg = + ap_get_module_config(c->conn_config, &ssl_ct_module); + + if (!conncfg) { + conncfg = apr_pcalloc(c->pool, sizeof *conncfg); + ap_set_module_config(c->conn_config, &ssl_ct_module, conncfg); + } + + return conncfg; +} + +static void client_is_ct_aware(conn_rec *c) +{ + ct_conn_config *conncfg = get_conn_config(c); + conncfg->peer_ct_aware = 1; +} + +static int is_client_ct_aware(conn_rec *c) +{ + ct_conn_config *conncfg = get_conn_config(c); + + return conncfg->peer_ct_aware; +} + +static void server_cert_has_sct_list(conn_rec *c) +{ + ct_conn_config *conncfg = get_conn_config(c); + conncfg->server_cert_has_sct_list = 1; + conncfg->peer_ct_aware = 1; +} + +/* Look at SSLClient::VerifyCallback() and WriteSSLClientCTData() + * for validation and saving of data for auditing in a form that + * the c-t tools can use. + */ + +static cert_chain *cert_chain_init(apr_pool_t *p, STACK_OF(X509) *chain) +{ + cert_chain *cc = apr_pcalloc(p, sizeof(cert_chain)); + int i; + + cc->cert_arr = apr_array_make(p, 4, sizeof(X509 *)); + + for (i = 0; i < sk_X509_num(chain); i++) { + X509 **spot = apr_array_push(cc->cert_arr); + *spot = X509_dup(sk_X509_value(chain, i)); /* UNDOC */ + if (i == 0) { + cc->leaf = *spot; + } + } + + return cc; +} + +static void cert_chain_free(cert_chain *cc) { + X509 **elts = (X509 **)cc->cert_arr->elts; + int i; + + for (i = 0; i < cc->cert_arr->nelts; i++) { + X509_free(elts[i]); + } +} + +/* Create hash of leaf certificate and any SCTs so that + * we can determine whether or not we've seen this exact + * info from the server before. + */ +static const char *gen_key(conn_rec *c, cert_chain *cc, + ct_conn_config *conncfg) +{ + const char *fp; + SHA256_CTX sha256ctx; + unsigned char digest[SHA256_DIGEST_LENGTH]; + + fp = get_cert_fingerprint(c->pool, cc->leaf); + + SHA256_Init(&sha256ctx); /* UNDOC */ + SHA256_Update(&sha256ctx, (unsigned char *)fp, strlen(fp)); /* UNDOC */ + if (conncfg->cert_sct_list) { + SHA256_Update(&sha256ctx, conncfg->cert_sct_list, + conncfg->cert_sct_list_size); + } + if (conncfg->serverhello_sct_list) { + SHA256_Update(&sha256ctx, conncfg->serverhello_sct_list, + conncfg->serverhello_sct_list_size); + } + if (conncfg->ocsp_sct_list) { + SHA256_Update(&sha256ctx, conncfg->ocsp_sct_list, + conncfg->ocsp_sct_list_size); + } + SHA256_Final(digest, &sha256ctx); /* UNDOC */ + return apr_pescape_hex(c->pool, digest, sizeof digest, 0); +} + +static apr_status_t deserialize_SCTs(apr_pool_t *p, + ct_conn_config *conncfg, + void *sct_list, + apr_size_t sct_list_size) +{ + apr_size_t avail, len_of_data; + apr_status_t rv; + const unsigned char *mem, *start_of_data; + + mem = sct_list; + avail = sct_list_size; + + /* Make sure the overall length is correct */ + + rv = ctutil_read_var_bytes((const unsigned char **)&mem, + &avail, &start_of_data, &len_of_data); + if (rv != APR_SUCCESS) { + return rv; + } + + if (len_of_data + sizeof(apr_uint16_t) != sct_list_size) { + return APR_EINVAL; + } + + /* add each SCT in the list to the all_scts array */ + + mem = (unsigned char *)sct_list + sizeof(apr_uint16_t); + avail = sct_list_size - sizeof(apr_uint16_t); + + while (rv == APR_SUCCESS && avail > 0) { + rv = ctutil_read_var_bytes((const unsigned char **)&mem, &avail, + &start_of_data, &len_of_data); + if (rv == APR_SUCCESS) { + ct_sct_data *sct = (ct_sct_data *)apr_array_push(conncfg->all_scts); + + sct->data = start_of_data; + ap_assert(len_of_data <= USHRT_MAX); + sct->len = (apr_uint16_t)len_of_data; + } + } + + if (rv == APR_SUCCESS && avail != 0) { + return APR_EINVAL; + } + + return APR_SUCCESS; +} + +/* perform quick sanity check of server SCT(s) during handshake; + * errors should result in fatal alert + */ +static apr_status_t validate_server_data(apr_pool_t *p, conn_rec *c, + cert_chain *cc, ct_conn_config *conncfg, + ct_server_config *sconf) +{ + apr_status_t rv = APR_SUCCESS; + +#if AP_MODULE_MAGIC_AT_LEAST(20130702,2) + if (conncfg->serverhello_sct_list) { + ap_log_cdata(APLOG_MARK, APLOG_TRACE6, c, "SCT(s) from ServerHello", + conncfg->serverhello_sct_list, + conncfg->serverhello_sct_list_size, + AP_LOG_DATA_SHOW_OFFSET); + } + + if (conncfg->cert_sct_list) { + ap_log_cdata(APLOG_MARK, APLOG_TRACE6, c, "SCT(s) from certificate", + conncfg->cert_sct_list, + conncfg->cert_sct_list_size, + AP_LOG_DATA_SHOW_OFFSET); + } + + if (conncfg->ocsp_sct_list) { + ap_log_cdata(APLOG_MARK, APLOG_TRACE6, c, "SCT(s) from stapled OCSP response", + conncfg->ocsp_sct_list, + conncfg->ocsp_sct_list_size, + AP_LOG_DATA_SHOW_OFFSET); + } +#endif /* httpd has ap_log_*data() */ + + if (!conncfg->all_scts) { + conncfg->all_scts = apr_array_make(p, 4, sizeof(ct_sct_data)); + } + + /* deserialize all the SCTs */ + if (conncfg->cert_sct_list) { + rv = deserialize_SCTs(p, conncfg, conncfg->cert_sct_list, + conncfg->cert_sct_list_size); + if (rv != APR_SUCCESS) { + ap_log_cerror(APLOG_MARK, APLOG_ERR, rv, c, + "couldn't deserialize SCT list from certificate"); + } + } + if (rv == APR_SUCCESS && conncfg->serverhello_sct_list) { + rv = deserialize_SCTs(p, conncfg, conncfg->serverhello_sct_list, + conncfg->serverhello_sct_list_size); + if (rv != APR_SUCCESS) { + ap_log_cerror(APLOG_MARK, APLOG_ERR, rv, c, + "couldn't deserialize SCT list from ServerHello"); + } + } + if (rv == APR_SUCCESS && conncfg->ocsp_sct_list) { + rv = deserialize_SCTs(p, conncfg, conncfg->ocsp_sct_list, + conncfg->ocsp_sct_list_size); + if (rv != APR_SUCCESS) { + ap_log_cerror(APLOG_MARK, APLOG_ERR, rv, c, + "couldn't deserialize SCT list from stapled OCSP response"); + } + } + + if (rv == APR_SUCCESS) { + if (conncfg->all_scts->nelts < 1) { + /* How did we get here without at least one SCT? */ + ap_log_cerror(APLOG_MARK, APLOG_CRIT, 0, c, + "SNAFU: No deserialized SCTs found in validate_server_data()"); + rv = APR_EINVAL; + } + else { + apr_status_t tmprv; + int i, verification_failures, verification_successes, unknown_log_ids; + ct_sct_data *sct_elts; + ct_sct_data sct; + sct_fields_t fields; + + ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, c, + "%d SCTs received total", conncfg->all_scts->nelts); + + verification_failures = verification_successes = unknown_log_ids = 0; + sct_elts = (ct_sct_data *)conncfg->all_scts->elts; + for (i = 0; i < conncfg->all_scts->nelts; i++) { + sct = sct_elts[i]; + tmprv = sct_parse("backend server", c->base_server, + sct.data, sct.len, cc, + &fields); + if (tmprv != APR_SUCCESS) { + rv = tmprv; + } + else { + tmprv = sct_verify_timestamp(c, &fields); + if (tmprv != APR_SUCCESS) { + verification_failures++; + } + + if (active_log_config) { + /* will only block if we have a DB-based log + * configuration which is currently being refreshed + */ + ap_assert(apr_thread_rwlock_rdlock(log_config_rwlock) + == APR_SUCCESS); + tmprv = sct_verify_signature(c, &fields, + active_log_config); + ap_assert(apr_thread_rwlock_unlock(log_config_rwlock) + == APR_SUCCESS); + if (tmprv == APR_NOTFOUND) { + ap_log_cerror(APLOG_MARK, APLOG_WARNING, 0, c, + "Server sent SCT from unrecognized log"); + unknown_log_ids++; + } + else if (tmprv != APR_SUCCESS) { + ap_log_cerror(APLOG_MARK, APLOG_ERR, 0, c, + "Server sent SCT with invalid signature"); + tmprv = APR_EINVAL; + verification_failures++; + } + else { + verification_successes++; + } + } + else { + unknown_log_ids++; + ap_log_cerror(APLOG_MARK, APLOG_WARNING, 0, c, + "Signature of SCT from server could not be " + "verified (no configured log public keys)"); + } + } + sct_release(&fields); + } + if (verification_failures && !verification_successes) { + /* If no SCTs are valid, don't communicate. */ + rv = APR_EINVAL; + } + ap_log_cerror(APLOG_MARK, + rv != APR_SUCCESS ? APLOG_ERR : APLOG_INFO, 0, c, + "Signature/timestamp validation for %d SCTs: %d successes, " + "%d failures, %d from unknown logs", + conncfg->all_scts->nelts, verification_successes, + verification_failures, unknown_log_ids); + } + } + + return rv; +} + +/* Enqueue data from server for off-line audit (cert, SCT(s)) + * We already filtered out duplicate data being saved from this + * process. (With reverse proxy it will be the same data over + * and over.) + */ +#define SERVER_START 0x0001 +#define KEY_START 0x0002 +#define CERT_START 0x0003 +#define SCT_START 0x0004 + +static void save_server_data(conn_rec *c, cert_chain *cc, + ct_conn_config *conncfg, + const char *key) +{ + if (audit_file_mutex && audit_file) { /* child init successful, no + * subsequent error + */ + apr_size_t bytes_written; + apr_status_t rv; + int i; + ct_sct_data *sct_elts; + X509 **x509elts; + server_rec *s = c->base_server; + + /* Any error in this function is a file I/O error; + * if such an error occurs, the audit file will be closed + * and removed, and this child won't be able to queue + * anything for audit. (It is likely that other child + * processes will have the same problem.) + */ + + ctutil_thread_mutex_lock(audit_file_mutex); + + if (audit_file) { /* no error just occurred... */ + audit_file_nonempty = 1; + + rv = ctutil_file_write_uint16(s, audit_file, + SERVER_START); + + if (rv == APR_SUCCESS) { + rv = ctutil_file_write_uint16(s, audit_file, KEY_START); + } + + if (rv == APR_SUCCESS) { + ap_assert(strlen(key) <= USHRT_MAX); + rv = ctutil_file_write_uint16(s, audit_file, + (apr_uint16_t)strlen(key)); + } + + if (rv == APR_SUCCESS) { + rv = apr_file_write_full(audit_file, key, strlen(key), + &bytes_written); + } + + /* Write each certificate, starting with leaf */ + x509elts = (X509 **)cc->cert_arr->elts; + for (i = 0; rv == APR_SUCCESS && i < cc->cert_arr->nelts; i++) { + unsigned char *der_buf = NULL; + int der_length; + + rv = ctutil_file_write_uint16(s, audit_file, CERT_START); + + /* now write the cert!!! */ + + if (rv == APR_SUCCESS) { + der_length = i2d_X509(x509elts[i], &der_buf); + ap_assert(der_length > 0); + + rv = ctutil_file_write_uint24(s, audit_file, der_length); + } + + if (rv == APR_SUCCESS) { + rv = apr_file_write_full(audit_file, der_buf, der_length, + &bytes_written); + } + + OPENSSL_free(der_buf); + } + + /* Write each SCT */ + sct_elts = (ct_sct_data *)conncfg->all_scts->elts; + for (i = 0; rv == APR_SUCCESS && i < conncfg->all_scts->nelts; i++) { + ct_sct_data sct; + + rv = ctutil_file_write_uint16(s, audit_file, SCT_START); + + sct = sct_elts[i]; + + if (rv == APR_SUCCESS) { + rv = ctutil_file_write_uint16(s, audit_file, sct.len); + } + + if (rv == APR_SUCCESS) { + rv = apr_file_write_full(audit_file, sct.data, sct.len, + &bytes_written); + } + } + + if (rv != APR_SUCCESS) { + /* an I/O error occurred; file is not usable */ + ap_log_error(APLOG_MARK, APLOG_CRIT, rv, ap_server_conf, + "Failed to write to %s, disabling audit for this " + "child", audit_fn_active); + apr_file_close(audit_file); + audit_file = NULL; + apr_file_remove(audit_fn_active, + /* not used in current implementations */ + c->pool); + } + } + + ctutil_thread_mutex_unlock(audit_file_mutex); + } +} + +/* signed_certificate_timestamp */ +static const unsigned short CT_EXTENSION_TYPE = 18; + +/* See function of this name in openssl/apps/s_client.c */ +static int ocsp_resp_cb(SSL *ssl, void *arg) +{ + conn_rec *c = (conn_rec *)SSL_get_app_data(ssl); + ct_conn_config *conncfg = get_conn_config(c); + const unsigned char *p; + int i, len; + OCSP_RESPONSE *rsp; + OCSP_BASICRESP *br; + OCSP_RESPDATA *rd; + OCSP_SINGLERESP *single; + STACK_OF(X509_EXTENSION) *exts; + + len = SSL_get_tlsext_status_ocsp_resp(ssl, &p); /* UNDOC */ + if (!p) { + /* normal case */ + ap_log_cerror(APLOG_MARK, APLOG_TRACE1, 0, c, + "OCSP response callback called but no stapled response from server"); + return 1; + } + + rsp = d2i_OCSP_RESPONSE(NULL, &p, len); /* UNDOC */ + if (!rsp) { + ap_log_cerror(APLOG_MARK, APLOG_ERR, 0, c, + "Error parsing OCSP response"); + return 0; + } + + br = OCSP_response_get1_basic(rsp); /* UNDOC */ + if (!br) { + ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, c, + "no OCSP basic response"); + return 0; + } + + rd = br->tbsResponseData; + + for (i = 0; i < sk_OCSP_SINGLERESP_num(rd->responses); i++) { /* UNDOC */ + X509_EXTENSION *ext; + int idx; + ASN1_OCTET_STRING *oct; + + single = sk_OCSP_SINGLERESP_value(rd->responses, i); /* UNDOC */ + if (!single) { + continue; + } + + idx = OCSP_SINGLERESP_get_ext_by_NID(single, + NID_ct_cert_scts, -1); /* UNDOC */ + + if (idx == -1) { + continue; + } + + ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, c, + "index of NID_ct_cert_scts: %d", idx); + + exts = single->singleExtensions; + + ext = sk_X509_EXTENSION_value(exts, idx); /* UNDOC */ + oct = X509_EXTENSION_get_data(ext); /* UNDOC */ + + conncfg->ocsp_has_sct_list = 1; + conncfg->peer_ct_aware = 1; + conncfg->ocsp_sct_list_size = oct->length - 2; + conncfg->ocsp_sct_list = apr_pmemdup(c->pool, oct->data + 2, + conncfg->ocsp_sct_list_size); + } + + OCSP_RESPONSE_free(rsp); /* UNDOC */ + + return 1; +} + +/* Callbacks and structures for handling custom TLS Extensions: + * cli_ext_first_cb - sends data for ClientHello TLS Extension + * cli_ext_second_cb - receives data from ServerHello TLS Extension + */ +static int client_extension_callback_1(SSL *ssl, unsigned short ext_type, + const unsigned char **out, + unsigned short *outlen, int *al, + void *arg) +{ + conn_rec *c = (conn_rec *)SSL_get_app_data(ssl); + + /* nothing to send in ClientHello */ + + ap_log_cerror(APLOG_MARK, APLOG_TRACE2, 0, c, + "client_extension_callback_1 called, " + "ext %hu will be in ClientHello", + ext_type); + + return 1; +} + +/* Get SCT(s) from ServerHello */ +static int client_extension_callback_2(SSL *ssl, unsigned short ext_type, + const unsigned char *in, unsigned short inlen, + int *al, void *arg) +{ + conn_rec *c = (conn_rec *)SSL_get_app_data(ssl); + ct_conn_config *conncfg = get_conn_config(c); + + ap_log_cerror(APLOG_MARK, APLOG_TRACE2, 0, c, + "client_extension_callback_2 called, " + "ext %hu was in ServerHello (len %hu)", + ext_type, inlen); + + /* Note: Peer certificate is not available in this callback via + * SSL_get_peer_certificate(ssl) + */ + + conncfg->serverhello_has_sct_list = 1; + conncfg->peer_ct_aware = 1; + conncfg->serverhello_sct_list = apr_pmemdup(c->pool, in, inlen); + conncfg->serverhello_sct_list_size = inlen; + return 1; +} + +/* See SSLClient::VerifyCallback() in c-t/src/client/ssl_client.cc + * (That's a beast and hard to duplicate in depth when you consider + * all the support classes it relies on; mod_ssl_ct needs to be a + * C++ module so that the bugs are fixed in one place.) + * + * . This code should care about stapled SCTs but doesn't. + * . This code, unlike SSLClient::VerifyCallback(), doesn't look + * at the OpenSSL "input" chain. + */ +static int ssl_ct_ssl_proxy_verify(server_rec *s, conn_rec *c, + STACK_OF(X509) *chain) +{ + apr_pool_t *p = c->pool; + ct_conn_config *conncfg = get_conn_config(c); + ct_server_config *sconf = ap_get_module_config(s->module_config, + &ssl_ct_module); + int chain_size = sk_X509_num(chain); + int extension_index; + cert_chain *certs; + + if (sconf->proxy_awareness == PROXY_OBLIVIOUS) { + return OK; + } + + ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, c, + "ssl_ct_ssl_proxy_verify() - get server certificate info"); + + if (chain_size < 1) { + ap_log_cerror(APLOG_MARK, APLOG_ERR, 0, c, + "odd chain size %d -- cannot proceed", chain_size); + return APR_EINVAL; + } + + /* Note: SSLClient::Verify looks in both the input chain and the + * verified chain. + */ + + certs = cert_chain_init(p, chain); + conncfg->certs = certs; + + extension_index = + X509_get_ext_by_NID(certs->leaf, + NID_ct_precert_scts, + -1); + /* use X509_get_ext(certs->leaf, extension_index) to obtain X509_EXTENSION * */ + + if (extension_index >= 0) { + void *ext_struct; + + server_cert_has_sct_list(c); + /* as in Cert::ExtensionStructure() */ + ext_struct = X509_get_ext_d2i(certs->leaf, + NID_ct_precert_scts, + NULL, /* ignore criticality of extension */ + NULL); /* UNDOC */ + + if (ext_struct == NULL) { + ap_log_cerror(APLOG_MARK, APLOG_ERR, 0, c, + "Could not retrieve SCT list from certificate (unexpected)"); + } + else { + /* as in Cert::OctetStringExtensionData */ + ASN1_OCTET_STRING *octet = (ASN1_OCTET_STRING *)ext_struct; + conncfg->cert_sct_list = apr_pmemdup(p, + octet->data, + octet->length); + conncfg->cert_sct_list_size = octet->length; + ASN1_OCTET_STRING_free(octet); /* UNDOC */ + } + } + + return OK; +} + +static int ssl_ct_proxy_post_handshake(conn_rec *c, SSL *ssl) +{ + apr_pool_t *p = c->pool; + apr_status_t rv = APR_SUCCESS; + const char *key; + ct_cached_server_data *cached; + ct_conn_config *conncfg = get_conn_config(c); + server_rec *s = c->base_server; + ct_server_config *sconf = ap_get_module_config(s->module_config, + &ssl_ct_module); + int validation_error = 0, missing_sct_error = 0; + STACK_OF(X509) *chain = SSL_get_peer_cert_chain(ssl); + + if (sconf->proxy_awareness == PROXY_OBLIVIOUS) { + return OK; + } + + ssl_ct_ssl_proxy_verify(s, c, chain); + + ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, c, + "finally at the point where we can see where SCTs came from" + " %pp/%pp/%pp (c %pp)", + conncfg->cert_sct_list, conncfg->serverhello_sct_list, + conncfg->ocsp_sct_list, c); + + /* At this point we have the SCTs from the cert (if any) and the + * SCTs from the TLS extension (if any) in ct_conn_config. + */ + + if (conncfg->cert_sct_list || conncfg->serverhello_sct_list + || conncfg->ocsp_sct_list) { + + /* The key is critical to avoiding validating and queueing of + * the same stuff over and over. + * + * Is there any cheaper check than server cert and SCTs all exactly + * the same as before? + */ + + key = gen_key(c, conncfg->certs, conncfg); + + ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, c, + "key for server data: %s", key); + + ctutil_thread_mutex_lock(cached_server_data_mutex); + + cached = apr_hash_get(cached_server_data, key, APR_HASH_KEY_STRING); + + ctutil_thread_mutex_unlock(cached_server_data_mutex); + + if (!cached) { + ct_cached_server_data *new_server_data = + (ct_cached_server_data *)calloc(1, sizeof(ct_cached_server_data)); + + new_server_data->validation_result = + rv = validate_server_data(p, c, conncfg->certs, conncfg, sconf); + + if (rv != APR_SUCCESS) { + validation_error = 1; + } + + ctutil_thread_mutex_lock(cached_server_data_mutex); + + if ((cached = apr_hash_get(cached_server_data, key, APR_HASH_KEY_STRING))) { + /* some other thread snuck in + * we assume that the other thread got the same validation + * result that we did + */ + free(new_server_data); + new_server_data = NULL; + } + else { + /* no other thread snuck in */ + apr_hash_set(cached_server_data, key, APR_HASH_KEY_STRING, + new_server_data); + new_server_data = NULL; + } + + ctutil_thread_mutex_unlock(cached_server_data_mutex); + + if (rv == APR_SUCCESS && !cached) { + save_server_data(c, conncfg->certs, conncfg, key); + } + } + else { + /* cached */ + rv = cached->validation_result; + if (rv != APR_SUCCESS) { + validation_error = 1; + ap_log_cerror(APLOG_MARK, APLOG_INFO, rv, c, "bad cached validation result"); + } + } + } + else { + /* No SCTs at all; consult configuration to know what to do. */ + missing_sct_error = 1; + } + + if (conncfg->certs) { + cert_chain_free(conncfg->certs); + conncfg->certs = NULL; + } + + ap_log_cerror(APLOG_MARK, + rv == APR_SUCCESS ? APLOG_DEBUG : APLOG_ERR, rv, c, + "SCT list received in: %s%s%s(%s) (c %pp)", + conncfg->serverhello_has_sct_list ? "ServerHello " : "", + conncfg->server_cert_has_sct_list ? "certificate-extension " : "", + conncfg->ocsp_has_sct_list ? "OCSP " : "", + cached ? "already saved" : "seen for the first time", + c); + + if (sconf->proxy_awareness == PROXY_REQUIRE) { + if (missing_sct_error || validation_error) { + ap_log_cerror(APLOG_MARK, APLOG_ERR, 0, c, + "Forbidding access to backend server; no valid SCTs"); + return HTTP_FORBIDDEN; + } + } + + return OK; +} + +static int server_extension_callback_1(SSL *ssl, unsigned short ext_type, + const unsigned char *in, + unsigned short inlen, int *al, + void *arg) +{ + conn_rec *c = (conn_rec *)SSL_get_app_data(ssl); + + /* this callback tells us that client is CT-aware; + * there's nothing of interest in the extension data + */ + client_is_ct_aware(c); + + ap_log_cerror(APLOG_MARK, APLOG_TRACE2, 0, c, + "server_extension_callback_1 called, " + "ext %hu was in ClientHello (len %hu)", + ext_type, inlen); + + return 1; +} + +static int server_extension_callback_2(SSL *ssl, unsigned short ext_type, + const unsigned char **out, + unsigned short *outlen, int *al, + void *arg) +{ + conn_rec *c = (conn_rec *)SSL_get_app_data(ssl); + ct_server_config *sconf = ap_get_module_config(c->base_server->module_config, + &ssl_ct_module); + X509 *server_cert; + const char *fingerprint; + const unsigned char *scts; + apr_size_t scts_len; + apr_status_t rv; + + if (!is_client_ct_aware(c)) { + /* Hmmm... Is this actually called if the client doesn't include + * the extension in the ClientHello? I don't think so. + */ + ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, c, + "server_extension_callback_2: client isn't CT-aware"); + /* Skip this extension for ServerHello */ + return -1; + } + + /* need to reply with SCT */ + + server_cert = SSL_get_certificate(ssl); /* no need to free! */ + fingerprint = get_cert_fingerprint(c->pool, server_cert); + + ap_log_cerror(APLOG_MARK, APLOG_TRACE2, 0, c, + "server_extension_callback_2 called, " + "ext %hu will be in ServerHello", + ext_type); + + rv = read_scts(c->pool, fingerprint, + sconf->sct_storage, + c->base_server, (char **)&scts, &scts_len); + if (rv == APR_SUCCESS) { + *out = scts; + ap_assert(scts_len <= USHRT_MAX); + *outlen = (unsigned short)scts_len; + } + else { + /* Skip this extension for ServerHello */ + return -1; + } + + return 1; +} + +static void tlsext_cb(SSL *ssl, int client_server, int type, + unsigned char *data, int len, + void *arg) +{ + conn_rec *c = arg; + + if (type == CT_EXTENSION_TYPE) { + ap_log_cerror(APLOG_MARK, APLOG_TRACE2, 0, c, + "tlsext_cb called, got CT TLS extension"); + + client_is_ct_aware(c); + } +} + +static int ssl_ct_pre_handshake(conn_rec *c, SSL *ssl) +{ + ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, c, "client connected (pre-handshake)"); + + SSL_set_tlsext_status_type(ssl, TLSEXT_STATUSTYPE_ocsp); /* UNDOC */ + + /* This callback is needed only to determine that the peer is CT-aware + * when resuming a session. For an initial handshake, the callbacks + * registered via SSL_CTX_set_custom_srv_ext() are sufficient. + */ + SSL_set_tlsext_debug_callback(ssl, tlsext_cb); /* UNDOC */ + SSL_set_tlsext_debug_arg(ssl, c); /* UNDOC */ + + return OK; +} + +static int ssl_ct_init_server(server_rec *s, apr_pool_t *p, int is_proxy, + SSL_CTX *ssl_ctx) +{ + ct_callback_info *cbi = apr_pcalloc(p, sizeof *cbi); + ct_server_config *sconf = ap_get_module_config(s->module_config, + &ssl_ct_module); + + cbi->s = s; + + if (is_proxy && sconf->proxy_awareness != PROXY_OBLIVIOUS) { + /* _cli_ = "client" */ + if (!SSL_CTX_set_custom_cli_ext(ssl_ctx, CT_EXTENSION_TYPE, + client_extension_callback_1, + client_extension_callback_2, cbi)) { /* UNDOC */ + ap_log_error(APLOG_MARK, APLOG_EMERG, 0, s, + "Unable to initalize Certificate Transparency client " + "extension callbacks (callback for %d already registered?)", + CT_EXTENSION_TYPE); + return HTTP_INTERNAL_SERVER_ERROR; + } + + /* Uhh, hopefully this doesn't collide with anybody else. mod_ssl + * currently only sets this on the server SSL_CTX, when OCSP is + * enabled. + */ + SSL_CTX_set_tlsext_status_cb(ssl_ctx, ocsp_resp_cb); /* UNDOC */ + SSL_CTX_set_tlsext_status_arg(ssl_ctx, cbi); /* UNDOC */ + } + else if (!is_proxy) { + look_for_server_certs(s, ssl_ctx, sconf->sct_storage); + + /* _srv_ = "server" */ + if (!SSL_CTX_set_custom_srv_ext(ssl_ctx, CT_EXTENSION_TYPE, + server_extension_callback_1, + server_extension_callback_2, cbi)) { /* UNDOC */ + ap_log_error(APLOG_MARK, APLOG_EMERG, 0, s, + "Unable to initalize Certificate Transparency server " + "extension callback (callbacks for %d already registered?)", + CT_EXTENSION_TYPE); + return HTTP_INTERNAL_SERVER_ERROR; + } + } + + return OK; +} + +static int ssl_ct_post_read_request(request_rec *r) +{ + ct_conn_config *conncfg = + ap_get_module_config(r->connection->conn_config, &ssl_ct_module); + + if (conncfg && conncfg->peer_ct_aware) { + apr_table_set(r->subprocess_env, STATUS_VAR, STATUS_VAR_AWARE_VAL); + } + else { + apr_table_set(r->subprocess_env, STATUS_VAR, STATUS_VAR_UNAWARE_VAL); + } + + return DECLINED; +} + +static int ssl_ct_pre_config(apr_pool_t *pconf, apr_pool_t *plog, + apr_pool_t *ptemp) +{ + apr_status_t rv = ap_mutex_register(pconf, SSL_CT_MUTEX_TYPE, NULL, + APR_LOCK_DEFAULT, 0); + if (rv != APR_SUCCESS) { + return rv; + } + + apr_dbd_init(pconf); + + ctutil_run_internal_tests(ptemp); + + return OK; +} + +static apr_status_t inactivate_audit_file(void *data) +{ + apr_status_t rv; + server_rec *s = data; + + if (!audit_file) { /* something bad happened after child init */ + return APR_SUCCESS; + } + + /* the normal cleanup was disabled in the call to apr_file_open */ + rv = apr_file_close(audit_file); + audit_file = NULL; + if (rv == APR_SUCCESS) { + if (audit_file_nonempty) { + rv = apr_file_rename(audit_fn_active, audit_fn_perm, + /* not used in current implementations */ + s->process->pool); + } + else { + /* No data written; just remove the file */ + apr_file_remove(audit_fn_active, + /* not used in current implementations */ + s->process->pool); + } + } + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_CRIT, rv, s, + "error flushing/closing %s or renaming it to %s", + audit_fn_active, audit_fn_perm); + } + + return APR_SUCCESS; /* what, you think anybody cares? */ +} + +static void ssl_ct_child_init(apr_pool_t *p, server_rec *s) +{ + apr_status_t rv; + const char *audit_basename; + ct_server_config *sconf = ap_get_module_config(s->module_config, + &ssl_ct_module); + + cached_server_data = apr_hash_make(p); + + rv = apr_global_mutex_child_init(&ssl_ct_sct_update, + apr_global_mutex_lockfile(ssl_ct_sct_update), p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_CRIT, rv, s, + "could not initialize " SSL_CT_MUTEX_TYPE + " mutex in child"); + /* might crash otherwise due to lack of checking for initialized data + * in all the right places, but this is going to skip pchild cleanup + */ + exit(APEXIT_CHILDSICK); + } + + rv = apr_thread_rwlock_create(&log_config_rwlock, p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_CRIT, rv, s, + "could not create rwlock in child"); + exit(APEXIT_CHILDSICK); + } + + rv = apr_thread_create(&service_thread, NULL, run_service_thread, s, p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_CRIT, rv, s, + "could not create " SERVICE_THREAD_NAME " in child"); + /* might crash otherwise due to lack of checking for initialized data + * in all the right places, but this is going to skip pchild cleanup + */ + exit(APEXIT_CHILDSICK); + } + + apr_pool_cleanup_register(p, service_thread, wait_for_thread, + apr_pool_cleanup_null); + + if (sconf->proxy_awareness != PROXY_OBLIVIOUS) { + rv = apr_thread_mutex_create(&cached_server_data_mutex, + APR_THREAD_MUTEX_DEFAULT, + p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_CRIT, rv, s, + "could not allocate a thread mutex"); + /* might crash otherwise due to lack of checking for initialized data + * in all the right places, but this is going to skip pchild cleanup + */ + exit(APEXIT_CHILDSICK); + } + } + + if (sconf->proxy_awareness != PROXY_OBLIVIOUS && sconf->audit_storage) { + rv = apr_thread_mutex_create(&audit_file_mutex, + APR_THREAD_MUTEX_DEFAULT, p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_CRIT, rv, s, + "could not allocate a thread mutex"); + /* might crash otherwise due to lack of checking for initialized data + * in all the right places, but this is going to skip pchild cleanup + */ + exit(APEXIT_CHILDSICK); + } + + audit_basename = apr_psprintf(p, "audit_%" APR_PID_T_FMT, + getpid()); + rv = ctutil_path_join((char **)&audit_fn_perm, sconf->audit_storage, + audit_basename, p, s); + if (rv != APR_SUCCESS) { + /* might crash otherwise due to lack of checking for initialized data + * in all the right places, but this is going to skip pchild cleanup + */ + exit(APEXIT_CHILDSICK); + } + + audit_fn_active = apr_pstrcat(p, audit_fn_perm, ".tmp", NULL); + audit_fn_perm = apr_pstrcat(p, audit_fn_perm, ".out", NULL); + + if (ctutil_file_exists(p, audit_fn_active)) { + ap_log_error(APLOG_MARK, APLOG_CRIT, 0, s, + "ummm, pid-specific file %s was reused before audit grabbed it! (removing)", + audit_fn_active); + apr_file_remove(audit_fn_active, p); + } + + if (ctutil_file_exists(p, audit_fn_perm)) { + ap_log_error(APLOG_MARK, APLOG_CRIT, 0, s, + "ummm, pid-specific file %s was reused before audit grabbed it! (removing)", + audit_fn_perm); + apr_file_remove(audit_fn_perm, p); + } + + rv = apr_file_open(&audit_file, audit_fn_active, + APR_FOPEN_WRITE|APR_FOPEN_CREATE|APR_FOPEN_TRUNCATE + |APR_FOPEN_BINARY|APR_FOPEN_BUFFERED|APR_FOPEN_NOCLEANUP, + APR_FPROT_OS_DEFAULT, p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "can't create %s", audit_fn_active); + audit_file = NULL; + } + + if (audit_file) { + apr_pool_cleanup_register(p, s, inactivate_audit_file, apr_pool_cleanup_null); + } + } /* !PROXY_OBLIVIOUS */ +} + +static void *create_ct_server_config(apr_pool_t *p, server_rec *s) +{ + ct_server_config *conf = + (ct_server_config *)apr_pcalloc(p, sizeof(ct_server_config)); + + conf->max_sct_age = apr_time_from_sec(3600 * 24); + conf->proxy_awareness = PROXY_AWARENESS_UNSET; + conf->max_sh_sct = 100; + conf->static_cert_sct_dirs = apr_hash_make(p); + + return conf; +} + +static void *merge_ct_server_config(apr_pool_t *p, void *basev, void *virtv) +{ + ct_server_config *base = (ct_server_config *)basev; + ct_server_config *virt = (ct_server_config *)virtv; + ct_server_config *conf; + + conf = (ct_server_config *)apr_pmemdup(p, virt, sizeof(ct_server_config)); + + conf->sct_storage = base->sct_storage; + conf->audit_storage = base->audit_storage; + conf->ct_exe = base->ct_exe; + conf->max_sct_age = base->max_sct_age; + conf->log_config_fname = base->log_config_fname; + conf->db_log_config = base->db_log_config; + conf->static_log_config = base->static_log_config; + conf->max_sh_sct = base->max_sh_sct; + conf->static_cert_sct_dirs = base->static_cert_sct_dirs; + + conf->proxy_awareness = (virt->proxy_awareness != PROXY_AWARENESS_UNSET) + ? virt->proxy_awareness + : base->proxy_awareness; + + return conf; +} + +static int ssl_ct_detach_backend(request_rec *r, + proxy_conn_rec *backend) +{ + conn_rec *origin = backend->connection; + + if (origin) { + ct_conn_config *conncfg = get_conn_config(origin); + char *list, *last; + + ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r, + "ssl_ct_detach_backend, %d%d%d", + conncfg->server_cert_has_sct_list, + conncfg->serverhello_has_sct_list, + conncfg->ocsp_has_sct_list); + + apr_table_set(r->subprocess_env, STATUS_VAR, + conncfg->peer_ct_aware ? STATUS_VAR_AWARE_VAL : STATUS_VAR_UNAWARE_VAL); + + list = apr_pstrcat(r->pool, + conncfg->server_cert_has_sct_list ? "certext," : "", + conncfg->serverhello_has_sct_list ? "tlsext," : "", + conncfg->ocsp_has_sct_list ? "ocsp" : "", + NULL); + if (*list) { + last = list + strlen(list) - 1; + if (*last == ',') { + *last = '\0'; + } + } + + apr_table_set(r->subprocess_env, PROXY_SCT_SOURCES_VAR, list); + } + else { + ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r, + "No backend connection available in " + "ssl_ct_detach_backend(); assuming peer unaware"); + apr_table_set(r->subprocess_env, STATUS_VAR, + STATUS_VAR_UNAWARE_VAL); + } + + return OK; +} + +static void ct_register_hooks(apr_pool_t *p) +{ + ap_hook_pre_config(ssl_ct_pre_config, NULL, NULL, APR_HOOK_MIDDLE); + ap_hook_check_config(ssl_ct_check_config, NULL, NULL, APR_HOOK_MIDDLE); + ap_hook_post_config(ssl_ct_post_config, NULL, NULL, APR_HOOK_MIDDLE); + ap_hook_post_read_request(ssl_ct_post_read_request, NULL, NULL, APR_HOOK_MIDDLE); + ap_hook_child_init(ssl_ct_child_init, NULL, NULL, APR_HOOK_MIDDLE); + APR_OPTIONAL_HOOK(proxy, detach_backend, ssl_ct_detach_backend, NULL, NULL, + APR_HOOK_MIDDLE); + APR_OPTIONAL_HOOK(ssl, init_server, ssl_ct_init_server, NULL, NULL, + APR_HOOK_MIDDLE); + APR_OPTIONAL_HOOK(ssl, pre_handshake, + ssl_ct_pre_handshake, + NULL, NULL, APR_HOOK_MIDDLE); + APR_OPTIONAL_HOOK(ssl, proxy_post_handshake, ssl_ct_proxy_post_handshake, + NULL, NULL, APR_HOOK_MIDDLE); +} + +static const char *parse_num(apr_pool_t *p, + const char *arg, long min_val, + long max_val, long *val, + const char *cmd_name) +{ + char *endptr; + + errno = 0; + *val = strtol(arg, &endptr, 10); + if (errno != 0 + || *endptr != '\0' + || *val < min_val + || *val > max_val) { + return apr_psprintf(p, "%s must be between %ld " + "and %ld (was '%s')", cmd_name, min_val, + max_val, arg); + } + + return NULL; +} + +static const char *ct_audit_storage(cmd_parms *cmd, void *x, const char *arg) +{ + ct_server_config *sconf = ap_get_module_config(cmd->server->module_config, + &ssl_ct_module); + const char *err = ap_check_cmd_context(cmd, GLOBAL_ONLY); + + if (err) { + return err; + } + + sconf->audit_storage = ap_runtime_dir_relative(cmd->pool, arg); + + if (!ctutil_dir_exists(cmd->pool, sconf->audit_storage)) { + return apr_pstrcat(cmd->pool, "CTAuditStorage: Directory ", + sconf->audit_storage, + " does not exist", NULL); + } + + return NULL; +} + +static const char *ct_log_config_db(cmd_parms *cmd, void *x, const char *arg) +{ + ct_server_config *sconf = ap_get_module_config(cmd->server->module_config, + &ssl_ct_module); + const char *err = ap_check_cmd_context(cmd, GLOBAL_ONLY); + + if (err) { + return err; + } + + sconf->log_config_fname = ap_server_root_relative(cmd->pool, arg); + + return NULL; +} + +static const char *ct_max_sct_age(cmd_parms *cmd, void *x, const char *arg) +{ + ct_server_config *sconf = ap_get_module_config(cmd->server->module_config, + &ssl_ct_module); + const char *err = ap_check_cmd_context(cmd, GLOBAL_ONLY); + long val; + + if (err) { + return err; + } + + err = parse_num(cmd->pool, arg, 10, 3600 * 12, &val, "CTMaxSCTAge"); + if (err) { + return err; + } + + sconf->max_sct_age = apr_time_from_sec(val); + return NULL; +} + +static const char *ct_proxy_awareness(cmd_parms *cmd, void *x, const char *arg) +{ + ct_server_config *sconf = ap_get_module_config(cmd->server->module_config, + &ssl_ct_module); + + if (!strcasecmp(arg, "oblivious")) { + sconf->proxy_awareness = PROXY_OBLIVIOUS; + } + else if (!strcasecmp(arg, "aware")) { + sconf->proxy_awareness = PROXY_AWARE; + } + else if (!strcasecmp(arg, "require")) { + sconf->proxy_awareness = PROXY_REQUIRE; + } + else { + return apr_pstrcat(cmd->pool, "CTProxyAwareness: Invalid argument \"", + arg, "\"", NULL); + } + + return NULL; +} + +static const char *ct_sct_storage(cmd_parms *cmd, void *x, const char *arg) +{ + ct_server_config *sconf = ap_get_module_config(cmd->server->module_config, + &ssl_ct_module); + const char *err = ap_check_cmd_context(cmd, GLOBAL_ONLY); + + if (err) { + return err; + } + + sconf->sct_storage = ap_runtime_dir_relative(cmd->pool, arg); + + if (!ctutil_dir_exists(cmd->pool, sconf->sct_storage)) { + return apr_pstrcat(cmd->pool, "CTSCTStorage: Directory ", + sconf->sct_storage, + " does not exist", NULL); + } + + return NULL; +} + +static const char *ct_sct_limit(cmd_parms *cmd, void *x, const char *arg) +{ + ct_server_config *sconf = ap_get_module_config(cmd->server->module_config, + &ssl_ct_module); + const char *err = ap_check_cmd_context(cmd, GLOBAL_ONLY); + long val; + + if (err) { + return err; + } + + err = parse_num(cmd->pool, arg, 1, 100, &val, + "CTServerHelloSCTLimit"); + if (err) { + return err; + } + + sconf->max_sh_sct = val; + return NULL; +} + +static const char *ct_static_log_config(cmd_parms *cmd, void *x, int argc, + char *const argv[]) +{ + apr_status_t rv; + const char *err = ap_check_cmd_context(cmd, GLOBAL_ONLY); + const char *log_id, *public_key, *distrusted, *min_valid_time, + *max_valid_time, *url; + ct_server_config *sconf = ap_get_module_config(cmd->server->module_config, + &ssl_ct_module); + int cur_arg; + + if (err) { + return err; + } + + if (argc != 6) { + return "CTStaticLogConfig: 6 arguments are required"; + } + + cur_arg = 0; + log_id = argv[cur_arg++]; + if (!strcmp(log_id, "-")) { + log_id = NULL; + } + + public_key = argv[cur_arg++]; + if (!strcmp(public_key, "-")) { + public_key = NULL; + } + else { + public_key = ap_server_root_relative(cmd->pool, public_key); + } + + distrusted = argv[cur_arg++]; + if (!strcmp(distrusted, "-")) { + distrusted = NULL; + } + + min_valid_time = argv[cur_arg++]; + if (!strcmp(min_valid_time, "-")) { + min_valid_time = NULL; + } + + max_valid_time = argv[cur_arg++]; + if (!strcmp(max_valid_time, "-")) { + max_valid_time = NULL; + } + + url = argv[cur_arg++]; + if (!strcmp(url, "-")) { + url = NULL; + } + + if (!sconf->static_log_config) { + sconf->static_log_config = + apr_array_make(cmd->pool, 2, sizeof(ct_log_config *)); + } + rv = save_log_config_entry(sconf->static_log_config, cmd->pool, + log_id, public_key, distrusted, + min_valid_time, max_valid_time, url); + if (rv != APR_SUCCESS) { + return "Error processing static log configuration"; + } + + return NULL; +} + +static const char *ct_static_scts(cmd_parms *cmd, void *x, const char *cert_fn, + const char *sct_dn) +{ + apr_pool_t *p = cmd->pool; + apr_status_t rv; + ct_server_config *sconf = ap_get_module_config(cmd->server->module_config, + &ssl_ct_module); + const char *err = ap_check_cmd_context(cmd, GLOBAL_ONLY); + const char *fingerprint; + FILE *pemfile; + X509 *cert; + + if (err) { + return err; + } + + cert_fn = ap_server_root_relative(cmd->pool, cert_fn); + sct_dn = ap_server_root_relative(cmd->pool, sct_dn); + + rv = ctutil_fopen(cert_fn, "r", &pemfile); + if (rv != APR_SUCCESS) { + return apr_psprintf(p, "could not open certificate file %s (%pm)", + cert_fn, &rv); + } + + cert = PEM_read_X509(pemfile, NULL, NULL, NULL); + if (!cert) { + return apr_psprintf(p, "could not read certificate from file %s", + cert_fn); + } + + fclose(pemfile); + + fingerprint = get_cert_fingerprint(cmd->pool, cert); + X509_free(cert); + + if (!ctutil_dir_exists(p, sct_dn)) { + return apr_pstrcat(p, "CTStaticSCTs: Directory ", sct_dn, + " does not exist", NULL); + } + + apr_hash_set(sconf->static_cert_sct_dirs, fingerprint, + APR_HASH_KEY_STRING, sct_dn); + + return NULL; +} + +static const char *ct_log_client(cmd_parms *cmd, void *x, const char *arg) +{ + ct_server_config *sconf = ap_get_module_config(cmd->server->module_config, + &ssl_ct_module); + const char *err = ap_check_cmd_context(cmd, GLOBAL_ONLY); + + if (err) { + return err; + } + + if (strcmp(DOTEXE, "")) { + if (!ctutil_file_exists(cmd->pool, arg)) { + arg = apr_pstrcat(cmd->pool, arg, DOTEXE, NULL); + } + } + + if (!ctutil_file_exists(cmd->pool, arg)) { + return apr_pstrcat(cmd->pool, + "CTLogClient: File ", + arg, + " does not exist", + NULL); + } + + sconf->ct_exe = arg; + + return NULL; +} + +static const command_rec ct_cmds[] = +{ + AP_INIT_TAKE1("CTAuditStorage", ct_audit_storage, NULL, + RSRC_CONF, /* GLOBAL_ONLY - audit data spans servers */ + "Location to store files of audit data"), + AP_INIT_TAKE1("CTLogConfigDB", ct_log_config_db, NULL, + RSRC_CONF, /* GLOBAL_ONLY - otherwise, you couldn't share + * the same SCT list for a cert used by two + * different vhosts (and the SCT maintenance daemon + * would be more complex) + */ + "Log configuration database"), + AP_INIT_TAKE1("CTMaxSCTAge", ct_max_sct_age, NULL, + RSRC_CONF, /* GLOBAL_ONLY - otherwise, you couldn't share + * the same SCT list for a cert used by two + * different vhosts + */ + "Max age of SCT obtained from log before refresh"), + AP_INIT_TAKE1("CTProxyAwareness", ct_proxy_awareness, NULL, + RSRC_CONF, /* per-server */ + "\"oblivious\" to neither ask for nor check SCTs, " + "\"aware\" to ask for and process SCTs but allow all connections, " + "or \"require\" to abort backend connections if an acceptable " + "SCT is not provided"), + AP_INIT_TAKE1("CTServerHelloSCTLimit", ct_sct_limit, NULL, + RSRC_CONF, /* GLOBAL_ONLY - otherwise, you couldn't share + * the same SCT list for a cert used by two + * different vhosts + */ + "Limit on number of SCTs sent in ServerHello"), + AP_INIT_TAKE1("CTSCTStorage", ct_sct_storage, NULL, + RSRC_CONF, /* GLOBAL_ONLY - otherwise, you couldn't share + * the same SCT list for a cert used by two + * different vhosts (and the SCT maintenance daemon + * would be more complex) + */ + "Location to store SCTs obtained from logs"), + AP_INIT_TAKE_ARGV("CTStaticLogConfig", ct_static_log_config, NULL, + RSRC_CONF, /* GLOBAL_ONLY */ + "Static log configuration record"), + AP_INIT_TAKE2("CTStaticSCTs", ct_static_scts, NULL, + RSRC_CONF, /* GLOBAL_ONLY - otherwise, you couldn't share + * the same SCT list for a cert used by two + * different vhosts (and the SCT maintenance daemon + * would be more complex) + */ + "Point to directory with static SCTs corresponding to the " + "specified certificate"), + AP_INIT_TAKE1("CTLogClient", ct_log_client, NULL, + RSRC_CONF, /* GLOBAL_ONLY - otherwise, you couldn't share + * the same SCTs for a cert used by two + * different vhosts (and it would be just plain + * silly :) ) + */ + "Location of certificate-transparency.org (or compatible) log client tool"), + {NULL} +}; + +AP_DECLARE_MODULE(ssl_ct) = +{ + STANDARD20_MODULE_STUFF, + NULL, + NULL, + create_ct_server_config, + merge_ct_server_config, + ct_cmds, + ct_register_hooks, +}; diff --git a/modules/ssl/ssl_ct_log_config.c b/modules/ssl/ssl_ct_log_config.c new file mode 100644 index 0000000000..de068ca57d --- /dev/null +++ b/modules/ssl/ssl_ct_log_config.c @@ -0,0 +1,419 @@ +/* Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "apr_dbd.h" +#include "apr_escape.h" +#include "apr_strings.h" + +#include "httpd.h" +#include "http_log.h" +#include "http_main.h" + +#include "ssl_ct_sct.h" +#include "ssl_ct_log_config.h" +#include "ssl_ct_util.h" + +APLOG_USE_MODULE(ssl_ct); + +int log_config_readable(apr_pool_t *p, const char *logconfig, + const char **msg) +{ + const apr_dbd_driver_t *driver; + apr_dbd_t *handle; + apr_status_t rv; + apr_dbd_results_t *res; + int rc; + + rv = apr_dbd_get_driver(p, "sqlite3", &driver); + if (rv != APR_SUCCESS) { + if (msg) { + *msg = "SQLite3 driver cannot be loaded"; + } + return 0; + } + + rv = apr_dbd_open(driver, p, logconfig, &handle); + if (rv != APR_SUCCESS) { + return 0; + } + + /* is there a cheaper way? */ + res = NULL; + rc = apr_dbd_select(driver, p, handle, &res, + "SELECT * FROM loginfo WHERE id = 0", 0); + + apr_dbd_close(driver, handle); + + if (rc != 0) { + return 0; + } + + return 1; +} + +static apr_status_t public_key_cleanup(void *data) +{ + EVP_PKEY *pubkey = data; + + EVP_PKEY_free(pubkey); + return APR_SUCCESS; +} + +static apr_status_t read_public_key(apr_pool_t *p, const char *pubkey_fname, + EVP_PKEY **ppkey) +{ + apr_status_t rv; + EVP_PKEY *pubkey; + FILE *pubkeyf; + + *ppkey = NULL; + + rv = ctutil_fopen(pubkey_fname, "r", &pubkeyf); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, ap_server_conf, + "could not open log public key file %s", + pubkey_fname); + return rv; + } + + pubkey = PEM_read_PUBKEY(pubkeyf, NULL, NULL, NULL); + if (!pubkey) { + fclose(pubkeyf); + rv = APR_EINVAL; + ap_log_error(APLOG_MARK, APLOG_ERR, 0, ap_server_conf, + "PEM_read_PUBKEY() failed to process public key file %s", + pubkey_fname); + return rv; + } + + fclose(pubkeyf); + + *ppkey = pubkey; + + apr_pool_cleanup_register(p, (void *)pubkey, public_key_cleanup, + apr_pool_cleanup_null); + + return APR_SUCCESS; +} + +static void digest_public_key(EVP_PKEY *pubkey, unsigned char digest[LOG_ID_SIZE]) +{ + int len = i2d_PUBKEY(pubkey, NULL); + unsigned char *val = malloc(len); + unsigned char *tmp = val; + SHA256_CTX sha256ctx; + + ap_assert(LOG_ID_SIZE == SHA256_DIGEST_LENGTH); + + i2d_PUBKEY(pubkey, &tmp); + SHA256_Init(&sha256ctx); + SHA256_Update(&sha256ctx, (unsigned char *)val, len); + SHA256_Final(digest, &sha256ctx); + free(val); +} + +static apr_status_t parse_log_url(apr_pool_t *p, const char *lu, apr_uri_t *puri) +{ + apr_status_t rv; + apr_uri_t uri; + + rv = apr_uri_parse(p, lu, &uri); + if (rv == APR_SUCCESS) { + if (!uri.scheme + || !uri.hostname + || !uri.path) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, ap_server_conf, + "Error in log url \"%s\": URL can't be parsed or is missing required " + "elements", lu); + rv = APR_EINVAL; + } + if (strcmp(uri.scheme, "http")) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, ap_server_conf, + "Error in log url \"%s\": Only scheme \"http\" (instead of \"%s\") " + "is currently accepted", + lu, uri.scheme); + rv = APR_EINVAL; + } + if (strcmp(uri.path, "/")) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, ap_server_conf, + "Error in log url \"%s\": Only path \"/\" (instead of \"%s\") " + "is currently accepted", + lu, uri.path); + rv = APR_EINVAL; + } + } + if (rv == APR_SUCCESS) { + *puri = uri; + } + return rv; +} + +static apr_status_t parse_time_str(apr_pool_t *p, const char *time_str, + apr_time_t *time) +{ + apr_int64_t val; + const char *end; + + errno = 0; + val = apr_strtoi64(time_str, (char **)&end, 10); + if (errno || *end != '\0') { + return APR_EINVAL; + } + + *time = apr_time_from_msec(val); + return APR_SUCCESS; +} + +/* The log_config array should have already been allocated from p. */ +apr_status_t save_log_config_entry(apr_array_header_t *log_config, + apr_pool_t *p, + const char *log_id, + const char *pubkey_fname, + const char *distrusted_str, + const char *min_time_str, + const char *max_time_str, + const char *url) +{ + apr_size_t len; + apr_status_t rv; + apr_time_t min_time, max_time; + apr_uri_t uri; + char *computed_log_id = NULL, *log_id_bin = NULL; + ct_log_config *newconf, **pnewconf; + int distrusted; + EVP_PKEY *public_key; + + if (!distrusted_str) { + distrusted = DISTRUSTED_UNSET; + } + else if (!strcasecmp(distrusted_str, "1")) { + distrusted = DISTRUSTED; + } + else if (!strcasecmp(distrusted_str, "0")) { + distrusted = DISTRUSTED; + } + else { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, ap_server_conf, + "Trusted status \"%s\" not valid", distrusted_str); + return APR_EINVAL; + } + + if (log_id) { + rv = apr_unescape_hex(NULL, log_id, strlen(log_id), 0, &len); + if (rv != 0 || len != 32) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, ap_server_conf, + "Log id \"%s\" not valid", log_id); + log_id_bin = apr_palloc(p, len); + apr_unescape_hex(log_id_bin, log_id, strlen(log_id), 0, NULL); + } + } + + if (pubkey_fname) { + rv = read_public_key(p, pubkey_fname, &public_key); + if (rv != APR_SUCCESS) { + return rv; + } + } + else { + public_key = NULL; + } + + if (min_time_str) { + rv = parse_time_str(p, min_time_str, &min_time); + if (rv) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, ap_server_conf, + "Invalid min time \"%s\"", min_time_str); + return rv; + } + } + else { + min_time = 0; + } + + if (max_time_str) { + rv = parse_time_str(p, max_time_str, &max_time); + if (rv) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, ap_server_conf, + "Invalid max time \"%s\"", max_time_str); + return rv; + } + } + else { + max_time = 0; + } + + if (url) { + rv = parse_log_url(p, url, &uri); + if (rv != APR_SUCCESS) { + return rv; + } + } + + newconf = apr_pcalloc(p, sizeof(ct_log_config)); + pnewconf = (ct_log_config **)apr_array_push(log_config); + *pnewconf = newconf; + + newconf->distrusted = distrusted; + newconf->public_key = public_key; + + if (newconf->public_key) { + computed_log_id = apr_palloc(p, LOG_ID_SIZE); + digest_public_key(newconf->public_key, + (unsigned char *)computed_log_id); + } + + if (computed_log_id && log_id_bin) { + if (memcmp(computed_log_id, log_id_bin, LOG_ID_SIZE)) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, ap_server_conf, + "Provided log id doesn't match digest of public key"); + return APR_EINVAL; + } + } + + newconf->log_id = log_id_bin ? log_id_bin : computed_log_id; + + newconf->min_valid_time = min_time; + newconf->max_valid_time = max_time; + + newconf->url = url; + if (url) { + newconf->uri = uri; + newconf->uri_str = apr_uri_unparse(p, &uri, 0); + } + newconf->public_key_pem = pubkey_fname; + + return APR_SUCCESS; +} + +apr_status_t read_config_db(apr_pool_t *p, server_rec *s_main, + const char *log_config_fname, + apr_array_header_t *log_config) +{ + apr_status_t rv; + const apr_dbd_driver_t *driver; + apr_dbd_t *handle; + apr_dbd_results_t *res; + apr_dbd_row_t *row; + int rc; + + ap_assert(log_config); + + rv = apr_dbd_get_driver(p, "sqlite3", &driver); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s_main, + "APR SQLite3 driver can't be loaded"); + return rv; + } + + rv = apr_dbd_open(driver, p, log_config_fname, &handle); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s_main, + "Can't open SQLite3 db %s", log_config_fname); + return rv; + } + + res = NULL; + rc = apr_dbd_select(driver, p, handle, &res, + "SELECT * FROM loginfo", 0); + + if (rc != 0) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, s_main, + "SELECT of loginfo records failed"); + apr_dbd_close(driver, handle); + return APR_EINVAL; + } + + rc = apr_dbd_num_tuples(driver, res); + switch (rc) { + case -1: + ap_log_error(APLOG_MARK, APLOG_ERR, 0, s_main, + "Unexpected asynchronous result reading %s", + log_config_fname); + apr_dbd_close(driver, handle); + return APR_EINVAL; + case 0: + ap_log_error(APLOG_MARK, APLOG_WARNING, 0, s_main, + "Log configuration in %s is empty", + log_config_fname); + apr_dbd_close(driver, handle); + return APR_SUCCESS; + default: + /* quiet some lints */ + break; + } + + for (rv = apr_dbd_get_row(driver, p, res, &row, -1); + rv == APR_SUCCESS; + rv = apr_dbd_get_row(driver, p, res, &row, -1)) { + int cur = 0; + const char *id = apr_dbd_get_entry(driver, row, cur++); + const char *log_id = apr_dbd_get_entry(driver, row, cur++); + const char *public_key = apr_dbd_get_entry(driver, row, cur++); + const char *distrusted = apr_dbd_get_entry(driver, row, cur++); + const char *min_timestamp = apr_dbd_get_entry(driver, row, cur++); + const char *max_timestamp = apr_dbd_get_entry(driver, row, cur++); + const char *url = apr_dbd_get_entry(driver, row, cur++); + + ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s_main, + "Log config: Record %s, log id %s, public key file %s, distrusted %s, URL %s, time %s->%s", + id, + log_id ? log_id : "(unset)", + public_key ? public_key : "(unset)", + distrusted ? distrusted : "(unset, defaults to trusted)", + url ? url : "(unset)", + min_timestamp ? min_timestamp : "-INF", + max_timestamp ? max_timestamp : "+INF"); + + rv = save_log_config_entry(log_config, p, log_id, + public_key, distrusted, + min_timestamp, max_timestamp, url); + if (rv != APR_SUCCESS) { + apr_dbd_close(driver, handle); + return rv; + } + } + + apr_dbd_close(driver, handle); + + return APR_SUCCESS; +} + +int log_valid_for_received_sct(const ct_log_config *l, apr_time_t to_check) +{ + if (l->distrusted == DISTRUSTED) { + return 0; + } + + if (l->max_valid_time && l->max_valid_time < to_check) { + return 0; + } + + if (l->min_valid_time && l->min_valid_time < to_check) { + return 0; + } + + return 1; +} + +int log_valid_for_sent_sct(const ct_log_config *l) +{ + /* The log could return us an SCT with an older timestamp which + * is within the trusted time interval for the log, but for + * simplicity let's just assume that if the log isn't still + * within a trusted interval we won't send SCTs from the log. + */ + return log_valid_for_received_sct(l, apr_time_now()); +} diff --git a/modules/ssl/ssl_ct_log_config.h b/modules/ssl/ssl_ct_log_config.h new file mode 100644 index 0000000000..1feb613264 --- /dev/null +++ b/modules/ssl/ssl_ct_log_config.h @@ -0,0 +1,57 @@ +/* Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef SSL_CT_LOG_CONFIG_H +#define SSL_CT_LOG_CONFIG_H + +#include "httpd.h" +#include "mod_ssl_openssl.h" /* cheap way to get OpenSSL headers */ + +typedef struct ct_log_config { + const char *log_id; /* binary form */ + const char *public_key_pem; + EVP_PKEY *public_key; +#define DISTRUSTED_UNSET -1 +#define TRUSTED 0 +#define DISTRUSTED 1 + int distrusted; + apr_time_t min_valid_time, max_valid_time; + const char *url; + const char *uri_str; + apr_uri_t uri; +} ct_log_config; + +int log_config_readable(apr_pool_t *p, const char *logconfig, + const char **msg); + +apr_status_t read_config_db(apr_pool_t *p, server_rec *s_main, + const char *log_config_fname, + apr_array_header_t *log_config); + +apr_status_t save_log_config_entry(apr_array_header_t *log_config, + apr_pool_t *p, + const char *log_id, + const char *pubkey_fname, + const char *distrusted, + const char *min_time, + const char *max_time, + const char *url); + +int log_valid_for_sent_sct(const ct_log_config *l); + +int log_valid_for_received_sct(const ct_log_config *l, apr_time_t to_check); + +#endif /* SSL_CT_LOG_CONFIG_H */ diff --git a/modules/ssl/ssl_ct_sct.c b/modules/ssl/ssl_ct_sct.c new file mode 100644 index 0000000000..c58423522e --- /dev/null +++ b/modules/ssl/ssl_ct_sct.c @@ -0,0 +1,300 @@ +/* Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + + +#include "ssl_ct_sct.h" +#include "ssl_ct_util.h" + +#include "http_log.h" + +APLOG_USE_MODULE(ssl_ct); + +static apr_status_t verify_signature(sct_fields_t *sctf, + EVP_PKEY *pkey) +{ + EVP_MD_CTX ctx; + int rc; + + if (sctf->signed_data == NULL) { + return APR_EINVAL; + } + + EVP_MD_CTX_init(&ctx); + ap_assert(1 == EVP_VerifyInit(&ctx, EVP_sha256())); + ap_assert(1 == EVP_VerifyUpdate(&ctx, sctf->signed_data, + sctf->signed_data_len)); + rc = EVP_VerifyFinal(&ctx, sctf->sig, sctf->siglen, pkey); + EVP_MD_CTX_cleanup(&ctx); + + return rc == 1 ? APR_SUCCESS : APR_EINVAL; +} + +apr_status_t sct_verify_signature(conn_rec *c, sct_fields_t *sctf, + apr_array_header_t *log_config) +{ + apr_status_t rv = APR_EINVAL; + int i; + ct_log_config **config_elts; + int nelts = log_config->nelts; + + ap_assert(sctf->signed_data != NULL); + + config_elts = (ct_log_config **)log_config->elts; + + for (i = 0; i < nelts; i++) { + EVP_PKEY *pubkey = config_elts[i]->public_key; + const char *logid = config_elts[i]->log_id; + + if (!pubkey || !logid) { + continue; + } + + if (!memcmp(logid, sctf->logid, LOG_ID_SIZE)) { + if (!log_valid_for_received_sct(config_elts[i], sctf->time)) { + ap_log_cerror(APLOG_MARK, APLOG_ERR, 0, c, + "Got SCT from distrusted log, or out of trusted time interval"); + return APR_EINVAL; + } + rv = verify_signature(sctf, pubkey); + if (rv != APR_SUCCESS) { + ap_log_cerror(APLOG_MARK, + APLOG_ERR, + rv, c, + "verify_signature failed"); + } + else { + ap_log_cerror(APLOG_MARK, APLOG_DEBUG, 0, c, + "verify_signature succeeded"); + } + return rv; + } + } + + return APR_NOTFOUND; +} + +apr_status_t sct_parse(const char *source, + server_rec *s, const unsigned char *sct, + apr_size_t len, cert_chain *cc, + sct_fields_t *fields) +{ + const unsigned char *cur; + apr_size_t orig_len = len; + apr_status_t rv; + + memset(fields, 0, sizeof *fields); + + if (len < 1 + LOG_ID_SIZE + 8) { + /* no room for header */ + ap_log_error(APLOG_MARK, APLOG_ERR, 0, s, + "SCT size %" APR_SIZE_T_FMT " is too small", + len); + return APR_EINVAL; + } + + cur = sct; + + fields->version = *cur; + cur++; + len -= 1; + memcpy(fields->logid, cur, LOG_ID_SIZE); + cur += LOG_ID_SIZE; + len -= LOG_ID_SIZE; + rv = ctutil_deserialize_uint64(&cur, &len, &fields->timestamp); + ap_assert(rv == APR_SUCCESS); + + fields->time = apr_time_from_msec(fields->timestamp); + + /* XXX maybe do this only if log level is such that we'll + * use it later? + */ + apr_rfc822_date(fields->timestr, fields->time); + + + if (len < 2) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, s, + "SCT size %" APR_SIZE_T_FMT " has no space for extension " + "len", orig_len); + return APR_EINVAL; + } + + rv = ctutil_deserialize_uint16(&cur, &len, &fields->extlen); + ap_assert(rv == APR_SUCCESS); + + if (fields->extlen != 0) { + if (fields->extlen < len) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, s, + "SCT size %" APR_SIZE_T_FMT " has no space for " + "%hu bytes of extensions", + orig_len, fields->extlen); + return APR_EINVAL; + } + + fields->extensions = cur; + cur += fields->extlen; + len -= fields->extlen; + } + else { + fields->extensions = 0; + } + + if (len < 4) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, s, + "SCT size %" APR_SIZE_T_FMT " has no space for " + "hash algorithm, signature algorithm, and signature len", + orig_len); + return APR_EINVAL; + } + + fields->hash_alg = *cur; + cur += 1; + len -= 1; + fields->sig_alg = *cur; + cur += 1; + len -= 1; + rv = ctutil_deserialize_uint16(&cur, &len, &fields->siglen); + ap_assert(rv == APR_SUCCESS); + + if (fields->siglen < len) { + ap_log_error(APLOG_MARK, APLOG_ERR, 0, s, + "SCT has no space for signature"); + return APR_EINVAL; + } + + fields->sig = cur; + cur += fields->siglen; + len -= fields->siglen; + + if (cc) { + /* If we have the server certificate, we can construct the + * data over which the signature is computed. + */ + + /* XXX Which part is signed? */ + /* See certificate-transparency/src/proto/serializer.cc, + * method Serializer::SerializeV1CertSCTSignatureInput() + */ + + apr_size_t orig_len; + apr_size_t avail; + int der_length; + unsigned char *mem; + unsigned char *orig_mem; + + der_length = i2d_X509(cc->leaf, NULL); + if (der_length < 0) { + rv = APR_EINVAL; + } + + if (rv == APR_SUCCESS) { + orig_len = 0 + + 1 /* version 1 */ + + 1 /* CERTIFICATE_TIMESTAMP */ + + 8 /* timestamp */ + + 2 /* X509_ENTRY */ + + 3 + der_length /* 24-bit length + X509 */ + + 2 + fields->extlen /* 16-bit length + extensions */ + ; + avail = orig_len; + mem = malloc(avail); + orig_mem = mem; + + rv = ctutil_serialize_uint8(&mem, &avail, 0); /* version 1 */ + if (rv == APR_SUCCESS) { + rv = ctutil_serialize_uint8(&mem, &avail, 0); /* CERTIFICATE_TIMESTAMP */ + } + if (rv == APR_SUCCESS) { + rv = ctutil_serialize_uint64(&mem, &avail, fields->timestamp); + } + if (rv == APR_SUCCESS) { + rv = ctutil_serialize_uint16(&mem, &avail, 0); /* X509_ENTRY */ + } + if (rv == APR_SUCCESS) { + /* Get DER encoding of leaf certificate */ + unsigned char *der_buf + /* get OpenSSL to allocate: */ + = NULL; + + der_length = i2d_X509(cc->leaf, &der_buf); + if (der_length < 0) { + rv = APR_EINVAL; + } + else { + rv = ctutil_write_var24_bytes(&mem, &avail, + der_buf, der_length); + OPENSSL_free(der_buf); + } + } + if (rv == APR_SUCCESS) { + rv = ctutil_write_var16_bytes(&mem, &avail, fields->extensions, + fields->extlen); + } + } + + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_CRIT, rv, s, + "Failed to reconstruct signed data for SCT"); + free(orig_mem); + } + else { + if (avail != 0) { + ap_log_error(APLOG_MARK, APLOG_CRIT, 0, s, + "length miscalculation for signed data (%" APR_SIZE_T_FMT + " vs. %" APR_SIZE_T_FMT ")", + orig_len, avail); + } + fields->signed_data_len = orig_len - avail; + fields->signed_data = orig_mem; + /* Force invalid signature error: orig_mem[0] = orig_mem[0] + 1; */ + } + } + + ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s, + "SCT from %s: version %d timestamp %s hash alg %d sig alg %d", + source, fields->version, fields->timestr, + fields->hash_alg, fields->sig_alg); +#if AP_MODULE_MAGIC_AT_LEAST(20130702,2) + ap_log_data(APLOG_MARK, APLOG_DEBUG, s, "Log Id", + fields->logid, sizeof(fields->logid), + AP_LOG_DATA_SHOW_OFFSET); + ap_log_data(APLOG_MARK, APLOG_DEBUG, s, "Signature", + fields->sig, fields->siglen, + AP_LOG_DATA_SHOW_OFFSET); +#endif /* httpd has ap_log_*data() */ + + ap_assert(!(fields->signed_data && rv != APR_SUCCESS)); + + return rv; +} + +void sct_release(sct_fields_t *sctf) +{ + if (sctf->signed_data) { + free((void *)sctf->signed_data); + sctf->signed_data = NULL; + } +} + +apr_status_t sct_verify_timestamp(conn_rec *c, sct_fields_t *sctf) +{ + if (sctf->time > apr_time_now()) { + ap_log_cerror(APLOG_MARK, APLOG_ERR, 0, c, + "Server sent SCT not yet valid (timestamp %s)", + sctf->timestr); + return APR_EINVAL; + } + return APR_SUCCESS; +} diff --git a/modules/ssl/ssl_ct_sct.h b/modules/ssl/ssl_ct_sct.h new file mode 100644 index 0000000000..2b1d89291d --- /dev/null +++ b/modules/ssl/ssl_ct_sct.h @@ -0,0 +1,64 @@ +/* Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef SSL_CT_SCT_H +#define SSL_CT_SCT_H + +#include "apr_pools.h" +#include "apr_tables.h" + +#include "httpd.h" +#include "mod_ssl.h" + +#include "ssl_ct_log_config.h" + +#define LOG_ID_SIZE 32 + +typedef struct cert_chain { + apr_pool_t *p; + apr_array_header_t *cert_arr; /* array of X509 * */ + X509 *leaf; +} cert_chain; + +typedef struct { + unsigned char version; + unsigned char logid[LOG_ID_SIZE]; + apr_uint64_t timestamp; + apr_time_t time; + char timestr[APR_RFC822_DATE_LEN]; + const unsigned char *extensions; + apr_uint16_t extlen; + unsigned char hash_alg; + unsigned char sig_alg; + apr_uint16_t siglen; + const unsigned char *sig; + const unsigned char *signed_data; + apr_size_t signed_data_len; +} sct_fields_t; + +apr_status_t sct_parse(const char *source, + server_rec *s, const unsigned char *sct, + apr_size_t len, cert_chain *cc, + sct_fields_t *fields); + +void sct_release(sct_fields_t *sctf); + +apr_status_t sct_verify_signature(conn_rec *c, sct_fields_t *sctf, + apr_array_header_t *log_config); + +apr_status_t sct_verify_timestamp(conn_rec *c, sct_fields_t *sctf); + +#endif /* SSL_CT_SCT_H */ diff --git a/modules/ssl/ssl_ct_util.c b/modules/ssl/ssl_ct_util.c new file mode 100644 index 0000000000..381e911fd5 --- /dev/null +++ b/modules/ssl/ssl_ct_util.c @@ -0,0 +1,781 @@ +/* Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "apr_fnmatch.h" +#include "apr_lib.h" +#include "apr_strings.h" + +#include "httpd.h" +#include "http_log.h" + +#include "ssl_ct_util.h" + +APLOG_USE_MODULE(ssl_ct); + +apr_status_t ctutil_path_join(char **out, const char *dirname, const char *basename, + apr_pool_t *p, server_rec *s) +{ + apr_status_t rv; + + rv = apr_filepath_merge(out, dirname, basename, 0, p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "can't build filename from %s and %s", + dirname, basename); + } + + return rv; +} + +int ctutil_dir_exists(apr_pool_t *p, const char *dirname) +{ + apr_finfo_t finfo; + apr_status_t rv = apr_stat(&finfo, dirname, APR_FINFO_TYPE, p); + + return rv == APR_SUCCESS && finfo.filetype == APR_DIR; +} + +int ctutil_file_exists(apr_pool_t *p, const char *filename) +{ + apr_finfo_t finfo; + apr_status_t rv = apr_stat(&finfo, filename, APR_FINFO_TYPE, p); + + return rv == APR_SUCCESS && finfo.filetype == APR_REG; +} + +void ctutil_buffer_to_array(apr_pool_t *p, const char *b, + apr_size_t b_size, apr_array_header_t **out) +{ + apr_array_header_t *arr = apr_array_make(p, 10, sizeof(char *)); + const char *ch, *last; + + ch = b; + last = b + b_size - 1; + while (ch < last) { + const char *end = memchr(ch, '\n', last - ch); + const char *line; + + if (!end) { + end = last + 1; + } + while (apr_isspace(*ch) && ch < end) { + ch++; + } + if (ch < end) { + const char *tmpend = end - 1; + + while (tmpend > ch + && isspace(*tmpend)) { + --tmpend; + } + + line = apr_pstrndup(p, ch, 1 + tmpend - ch); + *(const char **)apr_array_push(arr) = line; + } + ch = end + 1; + } + + *out = arr; +} + +int ctutil_in_array(const char *needle, const apr_array_header_t *haystack) +{ + const char * const *elts; + int i; + + elts = (const char * const *)haystack->elts; + for (i = 0; i < haystack->nelts; i++) { + if (!strcmp(needle, elts[i])) { + return 1; + } + } + + return 0; +} + +apr_status_t ctutil_fopen(const char *fn, const char *mode, FILE **f) +{ + apr_status_t rv; + + *f = fopen(fn, mode); + if (*f == NULL) { + rv = errno; /* XXX Windows equivalent -- CreateFile + fdopen? */ + } + else { + rv = APR_SUCCESS; + } + + return rv; +} + +/* read_dir() is remarkably like apr_match_glob(), which could + * probably use some processing flags to indicate variations on + * the basic behavior (and implement better error checking). + */ +apr_status_t ctutil_read_dir(apr_pool_t *p, + server_rec *s, + const char *dirname, + const char *pattern, + apr_array_header_t **outarr) +{ + apr_array_header_t *arr; + apr_dir_t *d; + apr_finfo_t finfo; + apr_status_t rv; + int reported = 0; + + /* add to existing array if it already exists */ + arr = *outarr; + if (arr == NULL) { + arr = apr_array_make(p, 4, sizeof(char *)); + } + + rv = apr_dir_open(&d, dirname, p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, "couldn't read dir %s", + dirname); + return rv; + } + + while ((rv = apr_dir_read(&finfo, APR_FINFO_NAME, d)) == APR_SUCCESS) { + const char *fn; + + if (APR_SUCCESS == apr_fnmatch(pattern, finfo.name, APR_FNM_CASE_BLIND)) { + rv = ctutil_path_join((char **)&fn, dirname, finfo.name, p, s); + if (rv != APR_SUCCESS) { + reported = 1; + break; + } + + *(char **)apr_array_push(arr) = apr_pstrdup(p, fn); + } + } + + if (APR_STATUS_IS_ENOENT(rv)) { + rv = APR_SUCCESS; + } + else if (rv != APR_SUCCESS && !reported) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "couldn't read entry from dir %s", dirname); + } + + apr_dir_close(d); + + if (rv == APR_SUCCESS) { + *outarr = arr; + } + + return rv; +} + +apr_status_t ctutil_read_file(apr_pool_t *p, + server_rec *s, + const char *fn, + apr_off_t limit, + char **contents, + apr_size_t *contents_size) +{ + apr_file_t *f; + apr_finfo_t finfo; + apr_status_t rv; + apr_size_t nbytes; + + *contents = NULL; + *contents_size = 0; + + rv = apr_file_open(&f, fn, APR_READ | APR_BINARY, APR_FPROT_OS_DEFAULT, p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "couldn't read %s", fn); + return rv; + } + + rv = apr_file_info_get(&finfo, APR_FINFO_SIZE, f); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "couldn't retrieve size of %s", fn); + apr_file_close(f); + return rv; + } + + if (finfo.size > limit) { + rv = APR_ENOSPC; + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "size %" APR_OFF_T_FMT " of %s exceeds limit (%" + APR_SIZE_T_FMT ")", finfo.size, fn, limit); + apr_file_close(f); + return rv; + } + + nbytes = (apr_size_t)finfo.size; + *contents = apr_palloc(p, nbytes); + rv = apr_file_read_full(f, *contents, nbytes, contents_size); + if (rv != APR_SUCCESS) { /* shouldn't get APR_EOF since we know + * how big the file is + */ + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "apr_file_read_full"); + } + apr_file_close(f); + + return rv; +} + +#if APR_FILES_AS_SOCKETS +static void io_loop(apr_pool_t *p, server_rec *s, apr_proc_t *proc, + const char *desc_for_log) +{ + apr_status_t rv; + apr_pollfd_t pfd = {0}; + apr_pollset_t *pollset; + int fds_waiting; + + rv = apr_pollset_create(&pollset, 2, p, 0); + ap_assert(rv == APR_SUCCESS); + + fds_waiting = 0; + + pfd.p = p; + pfd.desc_type = APR_POLL_FILE; + pfd.reqevents = APR_POLLIN; + pfd.desc.f = proc->err; + rv = apr_pollset_add(pollset, &pfd); + ap_assert(rv == APR_SUCCESS); + ++fds_waiting; + + pfd.desc.f = proc->out; + rv = apr_pollset_add(pollset, &pfd); + ap_assert(rv == APR_SUCCESS); + ++fds_waiting; + + while (fds_waiting) { + int i, num_events; + const apr_pollfd_t *pdesc; + char buf[4096]; + apr_size_t len; + + rv = apr_pollset_poll(pollset, apr_time_from_sec(10), + &num_events, &pdesc); + if (rv != APR_SUCCESS && !APR_STATUS_IS_EINTR(rv)) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "apr_pollset_poll"); + break; + } + + for (i = 0; i < num_events; i++) { + len = sizeof buf; + rv = apr_file_read(pdesc[i].desc.f, buf, &len); + if (APR_STATUS_IS_EOF(rv)) { + apr_file_close(pdesc[i].desc.f); + apr_pollset_remove(pollset, &pdesc[i]); + --fds_waiting; + } + else if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "apr_file_read"); + } + else { + ap_log_error(APLOG_MARK, APLOG_TRACE2, 0, s, + "%s: %.*s", desc_for_log, (int)len, buf); + } + } + } +} +#else /* APR_FILES_AS_SOCKETS */ +static void io_loop(apr_pool_t *p, server_rec *s, apr_proc_t *proc, + const char *desc_for_log) +{ + apr_status_t rv; + apr_file_t *fds[2] = {proc->out, proc->err}; + apr_size_t len; + char buf[4096]; + int fds_waiting = 2; + + while (fds_waiting) { + int i; + int read = 0; + + for (i = 0; i < sizeof fds / sizeof fds[0]; i++) { + if (!fds[i]) { + continue; + } + len = sizeof buf; + rv = apr_file_read(fds[i], buf, &len); + if (APR_STATUS_IS_EOF(rv)) { + apr_file_close(fds[i]); + fds[i] = NULL; + --fds_waiting; + } + else if (APR_STATUS_IS_EAGAIN(rv)) { + /* we don't actually know if data is ready before reading, so + * this isn't an error + */ + } + else if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "apr_file_read"); + } + else { + ap_log_error(APLOG_MARK, APLOG_TRACE2, 0, s, + "%s: %.*s", desc_for_log, (int)len, buf); + ++read; + } + } + if (fds_waiting && !read) { + /* no tight loop */ + apr_sleep(apr_time_from_msec(100)); + } + } +} +#endif /* APR_FILES_AS_SOCKETS */ + +apr_status_t ctutil_run_to_log(apr_pool_t *p, + server_rec *s, + const char *args[8], + const char *desc_for_log) +{ + apr_exit_why_e exitwhy; + apr_proc_t proc = {0}; + apr_procattr_t *attr; + apr_status_t rv; + int exitcode; + + rv = apr_procattr_create(&attr, p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, "apr_procattr_create failed"); + return rv; + } + + rv = apr_procattr_io_set(attr, + APR_NO_PIPE, + APR_CHILD_BLOCK, + APR_CHILD_BLOCK); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, "apr_procattr_io_set failed"); + return rv; + } + + rv = apr_proc_create(&proc, args[0], args, NULL, attr, p); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, "apr_proc_create failed"); + return rv; + } + + io_loop(p, s, &proc, desc_for_log); + + rv = apr_proc_wait(&proc, &exitcode, &exitwhy, APR_WAIT); + rv = rv == APR_CHILD_DONE ? APR_SUCCESS : rv; + + ap_log_error(APLOG_MARK, + rv != APR_SUCCESS || exitcode ? APLOG_ERR : APLOG_DEBUG, + rv, s, + "exit code from %s: %d (%s)", + desc_for_log, exitcode, + exitwhy == APR_PROC_EXIT ? "exited normally" : "exited due to a signal"); + + if (rv == APR_SUCCESS && exitcode) { + rv = APR_EGENERAL; + } + + return rv; +} + +void ctutil_thread_mutex_lock(apr_thread_mutex_t *m) +{ + apr_status_t rv = apr_thread_mutex_lock(m); + ap_assert(rv == APR_SUCCESS); +} + +void ctutil_thread_mutex_unlock(apr_thread_mutex_t *m) +{ + apr_status_t rv = apr_thread_mutex_unlock(m); + ap_assert(rv == APR_SUCCESS); +} + +apr_status_t ctutil_file_write_uint16(server_rec *s, + apr_file_t *f, + apr_uint16_t in_val) +{ + apr_size_t nbytes; + apr_status_t rv; + char vals[2]; + + vals[0] = (in_val & 0xFF00) >> 8; + vals[1] = (in_val & 0x00FF); + nbytes = sizeof(vals); + rv = apr_file_write(f, vals, &nbytes); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "can't write 2-byte length to file"); + } + return rv; +} + +apr_status_t ctutil_file_write_uint24(server_rec *s, + apr_file_t *f, + apr_uint32_t in_val) +{ + apr_size_t nbytes; + apr_status_t rv; + char vals[3]; + + vals[0] = (in_val & 0xFF0000) >> 16; + vals[1] = (in_val & 0x00FF00) >> 8; + vals[2] = (in_val & 0x0000FF) >> 0; + nbytes = sizeof(vals); + rv = apr_file_write(f, vals, &nbytes); + if (rv != APR_SUCCESS) { + ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, + "can't write 3-byte length to file"); + } + return rv; +} + +void ctutil_log_array(const char *file, int line, int module_index, + int level, server_rec *s, const char *desc, + apr_array_header_t *arr) +{ + const char **elts = (const char **)arr->elts; + int i; + + ap_log_error(file, line, module_index, level, + 0, s, "%s", desc); + for (i = 0; i < arr->nelts; i++) { + ap_log_error(file, line, module_index, level, + 0, s, ">>%s", elts[i]); + } +} + +static apr_status_t deserialize_uint(const unsigned char **mem, + apr_size_t *avail, + apr_byte_t num_bits, apr_uint64_t *pval) +{ + apr_byte_t num_bytes = num_bits / 8; + apr_uint64_t val = 0; + int i; + + if (*avail < num_bytes || num_bits > 64) { + return APR_EINVAL; + } + + for (i = 0; i < num_bytes; i++) { + val = (val << 8) | **mem; + *mem += 1; + *avail -= 1; + } + + *pval = val; + return APR_SUCCESS; +} + +apr_status_t ctutil_deserialize_uint64(const unsigned char **mem, + apr_size_t *avail, + apr_uint64_t *pval) +{ + return deserialize_uint(mem, avail, 64, pval); +} + +apr_status_t ctutil_deserialize_uint16(const unsigned char **mem, + apr_size_t *avail, + apr_uint16_t *pval) +{ + apr_status_t rv; + apr_uint64_t val64; + + rv = deserialize_uint(mem, avail, 16, &val64); + *pval = (apr_uint16_t)val64; + return rv; +} + +static apr_status_t serialize_uint(unsigned char **mem, apr_size_t *avail, + apr_byte_t num_bits, apr_uint64_t val) +{ + apr_byte_t num_bytes = num_bits / 8; + int i; + apr_uint64_t mask; + apr_byte_t shift; + + if (*avail < num_bytes || num_bits > 64) { + return APR_EINVAL; + } + + mask = (apr_uint64_t)0xFF << (num_bits - 8); + shift = num_bits - 8; + for (i = 0; i < num_bytes; i++) { + **mem = (unsigned char)((val & mask) >> shift); + *mem += 1; + *avail -= 1; + mask = mask >> 8; + shift -= 8; + } + + return APR_SUCCESS; +} + +apr_status_t ctutil_serialize_uint64(unsigned char **mem, apr_size_t *avail, + apr_uint64_t val) +{ + return serialize_uint(mem, avail, 64, val); +} + +apr_status_t ctutil_serialize_uint24(unsigned char **mem, apr_size_t *avail, + apr_uint32_t val) +{ + return serialize_uint(mem, avail, 24, val); +} + +apr_status_t ctutil_serialize_uint16(unsigned char **mem, apr_size_t *avail, + apr_uint16_t val) +{ + return serialize_uint(mem, avail, 16, val); +} + +apr_status_t ctutil_serialize_uint8(unsigned char **mem, apr_size_t *avail, + unsigned char val) +{ + return serialize_uint(mem, avail, 8, val); +} + +apr_status_t ctutil_write_var16_bytes(unsigned char **mem, apr_size_t *avail, + const unsigned char *val, + apr_uint16_t len) +{ + apr_status_t rv; + + if (*avail < (sizeof(apr_uint16_t) + len)) { + return APR_EINVAL; + } + + rv = ctutil_serialize_uint16(mem, avail, len); + if (rv != APR_SUCCESS) { /* should not occur */ + return rv; + } + + memcpy(*mem, val, len); + *mem += len; + *avail -= len; + return APR_SUCCESS; +} + +apr_status_t ctutil_write_var24_bytes(unsigned char **mem, apr_size_t *avail, + const unsigned char *val, + apr_uint32_t len) +{ + apr_status_t rv; + + if (*avail < (3 + len)) { + return APR_EINVAL; + } + + rv = ctutil_serialize_uint24(mem, avail, len); + if (rv != APR_SUCCESS) { /* should not occur */ + return rv; + } + + memcpy(*mem, val, len); + *mem += len; + *avail -= len; + return APR_SUCCESS; +} + +/* all this deserialization crap is of course from + * c-t/src/proto/serializer.cc + */ +static apr_status_t read_length_prefix(const unsigned char **mem, apr_size_t *avail, + apr_size_t *result) +{ + apr_status_t rv; + apr_uint16_t val; + + rv = ctutil_deserialize_uint16(mem, avail, &val); + if (rv == APR_SUCCESS) { + *result = val; + } + + return rv; +} + +static apr_status_t read_fixed_bytes(const unsigned char **mem, apr_size_t *avail, + apr_size_t len, + const unsigned char **start) +{ + if (*avail < len) { + return APR_EINVAL; + } + + *start = *mem; + *avail -= len; + *mem += len; + + return APR_SUCCESS; +} + +apr_status_t ctutil_read_var_bytes(const unsigned char **mem, apr_size_t *avail, + const unsigned char **start, apr_size_t *len) +{ + apr_status_t rv; + + rv = read_length_prefix(mem, avail, len); + if (rv != APR_SUCCESS) { + return rv; + } + + rv = read_fixed_bytes(mem, avail, *len, start); + return rv; +} + +#define TESTURL1 "http://127.0.0.1:8888" +#define TESTURL2 "http://127.0.0.1:9999" +#define TESTURL3 "http://127.0.0.1:10000" + +void ctutil_run_internal_tests(apr_pool_t *p) +{ + apr_array_header_t *arr; + const char *filecontents = + " " TESTURL1 " \r\n" TESTURL2 "\n" + TESTURL3 /* no "\n" */ ; + unsigned char buf[8], *ch; + const unsigned char *const_ch; + apr_size_t avail; + apr_status_t rv; + apr_uint16_t val16; + apr_uint64_t val64; + + ctutil_buffer_to_array(p, filecontents, strlen(filecontents), &arr); + + ap_assert(ctutil_in_array(TESTURL1, arr)); + ap_assert(ctutil_in_array(TESTURL2, arr)); + ap_assert(ctutil_in_array(TESTURL3, arr)); + ap_assert(!ctutil_in_array(TESTURL1 "x", arr)); + + ch = buf; + avail = 8; + rv = ctutil_serialize_uint64(&ch, &avail, 0xDEADBEEFCAFEBABE); + ap_assert(rv == APR_SUCCESS); + ap_assert(avail == 0); + ap_assert(ch == buf + 8); + ap_assert(buf[0] == 0xDE); + ap_assert(buf[1] == 0xAD); + ap_assert(buf[2] == 0xBE); + ap_assert(buf[3] == 0xEF); + ap_assert(buf[4] == 0xCA); + ap_assert(buf[5] == 0xFE); + ap_assert(buf[6] == 0xBA); + ap_assert(buf[7] == 0xBE); + + const_ch = buf; + avail = 8; + rv = ctutil_deserialize_uint64(&const_ch, &avail, &val64); + ap_assert(rv == APR_SUCCESS); + ap_assert(avail == 0); + ap_assert(const_ch == buf + 8); + ap_assert(val64 == 0xDEADBEEFCAFEBABE); + + ch = buf; + avail = 7; + ap_assert(ctutil_serialize_uint64(&ch, &avail, 0xDEADBEEFCAFEBABE) + == APR_EINVAL); + + ch = buf; + avail = 3; + rv = ctutil_serialize_uint24(&ch, &avail, 0xDEADBE); + ap_assert(rv == APR_SUCCESS); + ap_assert(avail == 0); + ap_assert(ch == buf + 3); + ap_assert(buf[0] == 0xDE); + ap_assert(buf[1] == 0xAD); + ap_assert(buf[2] == 0xBE); + + ch = buf; + avail = 1; + ap_assert(ctutil_serialize_uint16(&ch, &avail, 0xDEAD) + == APR_EINVAL); + + ch = buf; + avail = 2; + rv = ctutil_serialize_uint16(&ch, &avail, 0xDEAD); + ap_assert(rv == APR_SUCCESS); + ap_assert(avail == 0); + ap_assert(ch == buf + 2); + ap_assert(buf[0] == 0xDE); + ap_assert(buf[1] == 0xAD); + + const_ch = buf; + avail = 2; + rv = ctutil_deserialize_uint16(&const_ch, &avail, &val16); + ap_assert(rv == APR_SUCCESS); + ap_assert(avail == 0); + ap_assert(val16 == 0xDEAD); + + ch = buf; + avail = 1; + ap_assert(ctutil_serialize_uint16(&ch, &avail, 0xDEAD) + == APR_EINVAL); + + ch = buf; + avail = 1; + rv = ctutil_serialize_uint8(&ch, &avail, 0xDE); + ap_assert(rv == APR_SUCCESS); + ap_assert(avail == 0); + ap_assert(ch == buf + 1); + ap_assert(buf[0] == 0xDE); + + ch = buf; + avail = 0; + ap_assert(ctutil_serialize_uint8(&ch, &avail, 0xDE) + == APR_EINVAL); + + ch = buf; + avail = 8; + rv = ctutil_write_var16_bytes(&ch, &avail, + (unsigned char *)"\x01""\x02""\x03""\x04", 4); + ap_assert(rv == APR_SUCCESS); + ap_assert(avail == 2); + ap_assert(ch == buf + 6); + ap_assert(buf[0] == 0); + ap_assert(buf[1] == 4); + ap_assert(buf[2] == 0x01); + ap_assert(buf[3] == 0x02); + ap_assert(buf[4] == 0x03); + ap_assert(buf[5] == 0x04); + + ch = buf; + avail = 3; + rv = ctutil_write_var16_bytes(&ch, &avail, + (unsigned char *)"\x01""\x02""\x03""\x04", 4); + ap_assert(rv == APR_EINVAL); + + ch = buf; + avail = 8; + rv = ctutil_write_var24_bytes(&ch, &avail, + (unsigned char *)"\x01""\x02""\x03""\x04", 4); + ap_assert(rv == APR_SUCCESS); + ap_assert(avail == 1); + ap_assert(ch == buf + 7); + ap_assert(buf[0] == 0); + ap_assert(buf[1] == 0); + ap_assert(buf[2] == 4); + ap_assert(buf[3] == 0x01); + ap_assert(buf[4] == 0x02); + ap_assert(buf[5] == 0x03); + ap_assert(buf[6] == 0x04); + + ch = buf; + avail = 4; + rv = ctutil_write_var24_bytes(&ch, &avail, + (unsigned char *)"\x01""\x02""\x03""\x04", 4); + ap_assert(rv == APR_EINVAL); +} diff --git a/modules/ssl/ssl_ct_util.h b/modules/ssl/ssl_ct_util.h new file mode 100644 index 0000000000..d1f58beef3 --- /dev/null +++ b/modules/ssl/ssl_ct_util.h @@ -0,0 +1,102 @@ +/* Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef SSL_CT_UTIL_H +#define SSL_CT_UTIL_H + +#include "httpd.h" + +apr_status_t ctutil_path_join(char **out, const char *dirname, const char *basename, + apr_pool_t *p, server_rec *s); + +int ctutil_dir_exists(apr_pool_t *p, const char *dirname); + +int ctutil_file_exists(apr_pool_t *p, const char *filename); + +void ctutil_buffer_to_array(apr_pool_t *p, const char *b, + apr_size_t b_size, apr_array_header_t **out); + +int ctutil_in_array(const char *needle, const apr_array_header_t *haystack); + +apr_status_t ctutil_fopen(const char *fn, const char *mode, FILE **f); + +apr_status_t ctutil_read_dir(apr_pool_t *p, + server_rec *s, + const char *dirname, + const char *pattern, + apr_array_header_t **outarr); + +apr_status_t ctutil_read_file(apr_pool_t *p, + server_rec *s, + const char *fn, + apr_off_t limit, + char **contents, + apr_size_t *contents_size); + +apr_status_t ctutil_run_to_log(apr_pool_t *p, + server_rec *s, + const char *args[8], + const char *desc_for_log); + +void ctutil_thread_mutex_lock(apr_thread_mutex_t *m); +void ctutil_thread_mutex_unlock(apr_thread_mutex_t *m); + +apr_status_t ctutil_file_write_uint16(server_rec *s, + apr_file_t *f, + apr_uint16_t val); + +apr_status_t ctutil_file_write_uint24(server_rec *s, + apr_file_t *f, + apr_uint32_t val); + +void ctutil_log_array(const char *file, int line, int module_index, + int level, server_rec *s, const char *desc, + apr_array_header_t *arr); + +apr_status_t ctutil_read_var_bytes(const unsigned char **mem, + apr_size_t *avail, + const unsigned char **start, + apr_size_t *len); + +apr_status_t ctutil_deserialize_uint64(const unsigned char **mem, + apr_size_t *avail, apr_uint64_t *pval); +apr_status_t ctutil_deserialize_uint16(const unsigned char **mem, + apr_size_t *avail, + apr_uint16_t *pval); + +apr_status_t ctutil_serialize_uint64(unsigned char **mem, apr_size_t *avail, + apr_uint64_t val); + +apr_status_t ctutil_serialize_uint24(unsigned char **mem, apr_size_t *avail, + apr_uint32_t val); + +apr_status_t ctutil_serialize_uint16(unsigned char **mem, apr_size_t *avail, + apr_uint16_t val); + +apr_status_t ctutil_serialize_uint8(unsigned char **mem, apr_size_t *avail, + unsigned char val); + +apr_status_t ctutil_write_var16_bytes(unsigned char **mem, apr_size_t *avail, + const unsigned char *val, + apr_uint16_t len); + +apr_status_t ctutil_write_var24_bytes(unsigned char **mem, apr_size_t *avail, + const unsigned char *val, + apr_uint32_t len); + +void ctutil_run_internal_tests(apr_pool_t *p); + +#endif /* SSL_CT_UTIL_H */ diff --git a/support/Makefile.in b/support/Makefile.in index 635c247df4..65b77c0e4f 100644 --- a/support/Makefile.in +++ b/support/Makefile.in @@ -23,7 +23,7 @@ install: chmod 755 $(DESTDIR)$(bindir)/$$i; \ fi ; \ done - @for i in apachectl; do \ + @for i in apachectl ctlogconfig; do \ if test -f "$(builddir)/$$i"; then \ cp -p $$i $(DESTDIR)$(sbindir); \ chmod 755 $(DESTDIR)$(sbindir)/$$i; \ diff --git a/support/ctauditscts b/support/ctauditscts new file mode 100755 index 0000000000..6f4c12d12d --- /dev/null +++ b/support/ctauditscts @@ -0,0 +1,160 @@ +#!/usr/bin/env python +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import binascii +import os +import sqlite3 +import ssl +import struct +import sys +import tempfile + +from contextlib import closing + +SERVER_START = 1 +KEY_START = 2 +CERT_START = 3 +SCT_START = 4 + + +def usage(): + print >> sys.stderr, ('Usage: %s /path/to/audit/files ' + + '[/path/to/log-config-db]') % sys.argv[0] + sys.exit(1) + + +def audit(fn, tmp, already_checked, cur): + print 'Auditing %s...' % fn + + # First, parse the audit file into a series of related + # + # 1. PEM file with certificate chain + # 2. Individual SCT files + # + # Next, for each SCT, invoke verify_single_proof to verify. + log_bytes = open(fn, 'rb').read() + offset = 0 + while offset < len(log_bytes): + print 'Got package from server...' + val = struct.unpack_from('>H', log_bytes, offset) + assert val[0] == SERVER_START + offset += 2 + + assert struct.unpack_from('>H', log_bytes, offset)[0] == KEY_START + offset += 2 + + key_size = struct.unpack_from('>H', log_bytes, offset)[0] + assert key_size > 0 + offset += 2 + + key = log_bytes[offset:offset + key_size] + offset += key_size + + # at least one certificate + assert struct.unpack_from('>H', log_bytes, offset)[0] == CERT_START + + # for each certificate: + leaf = None + while struct.unpack_from('>H', log_bytes, offset)[0] == CERT_START: + offset += 2 + val = struct.unpack_from('BBB', log_bytes, offset) + offset += 3 + der_size = (val[0] << 16) | (val[1] << 8) | (val[2] << 0) + print ' Certificate size:', hex(der_size) + if not leaf: + leaf = (offset, der_size) + offset += der_size + + pem = ssl.DER_cert_to_PEM_cert(log_bytes[leaf[0]:leaf[0] + leaf[1]]) + + tmp_leaf_pem = tempfile.mkstemp(text=True) + with closing(os.fdopen(tmp_leaf_pem[0], 'w')) as f: + f.write(pem) + + # at least one SCT + assert struct.unpack_from('>H', log_bytes, offset)[0] == SCT_START + + # for each SCT: + while offset < len(log_bytes) and \ + struct.unpack_from('>H', log_bytes, offset)[0] == SCT_START: + offset += 2 + len_offset = offset + sct_size = struct.unpack_from('>H', log_bytes, len_offset)[0] + offset += 2 + print ' SCT size:', hex(sct_size) + log_id = log_bytes[offset + 1:offset + 1 + 32] + log_id_hex = binascii.hexlify(log_id).upper() + print ' Log id: %s' % log_id_hex + timestamp_ms = struct.unpack_from('>Q', log_bytes, offset + 33)[0] + print ' Timestamp: %s' % timestamp_ms + + # If we ever need the full SCT: sct = (offset, sct_size) + offset += sct_size + + if key in already_checked: + print ' (SCTs already checked)' + continue + + already_checked[key] = True + + log_url_arg = '' + if cur: + stmt = 'SELECT * FROM loginfo WHERE log_id = ?' + cur.execute(stmt, [log_id_hex]) + recs = list(cur.fetchall()) + if len(recs) > 0 and recs[0][6] is not None: + log_url = recs[0][6] + + # verify_single_proof doesn't accept :// + if '://' in log_url: + log_url = log_url.split('://')[1] + log_url_arg = '--log_url %s' % log_url + + print ' Log URL: ' + log_url + + cmd = 'verify_single_proof.py --cert %s --timestamp %s %s' % \ + (tmp_leaf_pem[1], timestamp_ms, log_url_arg) + print '>%s<' % cmd + os.system(cmd) + + os.unlink(tmp_leaf_pem[1]) + + +def main(): + if len(sys.argv) != 2 and len(sys.argv) != 3: + usage() + + top = sys.argv[1] + tmp = '/tmp' + + if len(sys.argv) == 3: + cxn = sqlite3.connect(sys.argv[2]) + cur = cxn.cursor() + else: + cur = None + + # could serialize this between runs to further limit duplicate checking + already_checked = dict() + + for dirpath, dnames, fnames in os.walk(top): + fnames = [fn for fn in fnames if fn[-4:] == '.out'] + for fn in fnames: + audit(os.path.join(dirpath, fn), tmp, already_checked, cur) + + +if __name__ == "__main__": + main() diff --git a/support/ctlogconfig b/support/ctlogconfig new file mode 100755 index 0000000000..4ebef5cb07 --- /dev/null +++ b/support/ctlogconfig @@ -0,0 +1,313 @@ +#!/usr/bin/env python +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import os +import re +import sqlite3 +import sys + + +def create_tables(db_name): + cxn = sqlite3.connect(db_name) + cur = cxn.cursor() + + cur.execute( + 'CREATE TABLE loginfo(' + + 'id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, ' + + 'log_id TEXT, ' + + 'public_key TEXT, ' # path to PEM-encoded file + + 'distrusted INTEGER, ' # non-zero if not trusted + + 'min_valid_timestamp INTEGER, ' + + 'max_valid_timestamp INTEGER, ' + + 'url TEXT)' + ) + cur.close() + cxn.commit() + cxn.close() + + +def record_id_arg(cur, args, required=False): + if len(args) < 1 or args[0][0] != '#' or len(args[0]) < 2: + if required: + print >> sys.stderr, 'A record id was not provided' + sys.exit(1) + return None + record_id = args.pop(0)[1:] + stmt = 'SELECT * FROM loginfo WHERE id = ?' + cur.execute(stmt, [record_id]) + recs = list(cur.fetchall()) + assert len(recs) < 2 + if len(recs) == 0: + print >> sys.stderr, 'Record #%s was not found' % record_id + sys.exit(1) + return record_id + + +def log_id_arg(cur, args, required=True): + if len(args) < 1 or len(args[0]) != 64: + if not required: + return None + print >> sys.stderr, 'A log id was not provided' + sys.exit(1) + log_id = args.pop(0).upper() + if len(re.compile(r'[A-Z0-9]').findall(log_id)) != len(log_id): + print >> sys.stderr, 'The log id is not formatted properly' + sys.exit(1) + return log_id + + +def public_key_arg(args): + if len(args) < 1: + print >> sys.stderr, 'A public key file was not provided' + sys.exit(1) + pubkey = args.pop(0) + if not os.path.exists(pubkey): + print >> sys.stderr, 'Public key file %s could not be read' % pubkey + sys.exit(1) + return pubkey + + +def time_arg(args): + if len(args) < 1: + print >> sys.stderr, 'A timestamp was not provided' + sys.exit(1) + t = args.pop(0) + if t == '-': + return None + try: + return int(t) + except ValueError: + print >> sys.stderr, 'The timestamp "%s" is invalid' % t + sys.exit(1) + + +def configure_public_key(cur, args): + record_id = record_id_arg(cur, args, False) + public_key = public_key_arg(args) + if len(args) != 0: + usage() + if not record_id: + stmt = 'INSERT INTO loginfo (public_key) VALUES(?)' + cur.execute(stmt, [public_key]) + else: + stmt = 'UPDATE loginfo SET public_key = ? WHERE id = ?' + cur.execute(stmt, [public_key, record_id]) + + +def configure_url(cur, args): + # can't specify more than one of record-id and log-id + log_id = None + record_id = record_id_arg(cur, args, False) + if not record_id: + log_id = log_id_arg(cur, args, False) + if len(args) != 1: + usage() + url = args.pop(0) + + if record_id: + stmt = 'UPDATE loginfo SET url = ? WHERE id = ?' + args = [url, record_id] + elif log_id: + stmt = 'INSERT INTO loginfo (log_id, url) VALUES(?, ?)' + args = [log_id, url] + else: + stmt = 'INSERT INTO loginfo (url) VALUES(?)' + args = [url] + + cur.execute(stmt, args) + + +def forget_log(cur, args): + record_id = record_id_arg(cur, args, False) + log_id = None + if not record_id: + log_id = log_id_arg(cur, args, True) + if len(args) != 0: + usage() + if record_id: + stmt = 'DELETE FROM loginfo WHERE id = ?' + args = [record_id] + else: + stmt = 'DELETE FROM loginfo WHERE log_id = ?' + args = [log_id] + cur.execute(stmt, args) + + +def trust_distrust_log(cur, args): + # could take a record id or a log id + record_id = record_id_arg(cur, args, False) + if record_id: + log_id = None + else: + log_id = log_id_arg(cur, args) + + if len(args) != 1: + usage() + flag = args.pop(0) + + if not record_id: + stmt = 'INSERT INTO loginfo (log_id, distrusted) VALUES(?, ?)' + cur.execute(stmt, [log_id, flag]) + else: + stmt = 'UPDATE loginfo SET distrusted = ? WHERE id = ?' + cur.execute(stmt, [flag, record_id]) + + +def trust_log(cur, args): + trust_distrust_log(cur, args + [0]) + + +def distrust_log(cur, args): + trust_distrust_log(cur, args + [1]) + + +def time_range(cur, args): + # could take a record id or a log id + record_id = record_id_arg(cur, args, False) + if record_id: + log_id = None + else: + log_id = log_id_arg(cur, args) + + min_valid_time = time_arg(args) + max_valid_time = time_arg(args) + if len(args) != 0: + usage() + if not record_id: + stmt = 'INSERT INTO loginfo ' + \ + '(log_id, min_valid_timestamp, max_valid_timestamp) ' + \ + 'VALUES(?, ?, ?)' + cur.execute(stmt, [log_id, min_valid_time, max_valid_time]) + else: + stmt = 'UPDATE loginfo SET min_valid_timestamp = ?, ' + \ + 'max_valid_timestamp = ? WHERE id = ?' + cur.execute(stmt, [min_valid_time, max_valid_time, record_id]) + + +class ConfigEntry: + + pass + + +def dump_ll(cur): + stmt = 'SELECT * FROM loginfo' + cur.execute(stmt) + recs = [] + for row in cur.fetchall(): + obj = ConfigEntry() + obj.id = row[0] + obj.log_id = row[1] + obj.public_key = row[2] + obj.distrusted = row[3] + obj.min_valid_timestamp = row[4] + obj.max_valid_timestamp = row[5] + obj.url = row[6] + recs += [obj] + return recs + + +def dump(cur, args): + if len(args) != 0: + usage() + recs = dump_ll(cur) + for rec in recs: + not_conf = '(not configured)' + + mint = \ + str(rec.min_valid_timestamp) if rec.min_valid_timestamp else '-INF' + maxt = \ + str(rec.max_valid_timestamp) if rec.max_valid_timestamp else '+INF' + print 'Log entry:' + print ' Record ' + str(rec.id) + \ + (' (DISTRUSTED)' if rec.distrusted else '') + print ' Log id : ' + (rec.log_id if rec.log_id else not_conf) + print ' Public key file: ' + \ + (rec.public_key if rec.public_key else not_conf) + print ' URL : ' + (rec.url if rec.url else not_conf) + print ' Time range : ' + mint + ' to ' + maxt + print '' + + +def usage(): + help = """Usage: %s /path/to/log-config-db command args + +Commands: + display config-db contents: + dump + configure public key: + configure-public-key [log-id|record-id] /path/log-pub-key.pem + configure URL: + configure-url [log-id|record-id] http://www.example.com/path/ + configure min and/or max valid timestamps: + valid-time-range log-id|record-id min-range max-range + mark log as trusted (default): + trust log-id|record-id + mark log as untrusted: + distrust log-id|record-id + remove log config from config-db: + forget log-id|record-id + +log-id is a 64-character hex string representation of a log id + +record-id references an existing entry and is in the form: + # + (displayable with the dump command) +""" % sys.argv[0] + print >> sys.stderr, help + sys.exit(1) + + +def main(argv): + if len(argv) < 3: + usage() + + db_name = argv[1] + cmd = argv[2] + args = argv[3:] + + cmds = {'configure-public-key': configure_public_key, + 'configure-url': configure_url, + 'distrust': distrust_log, + 'trust': trust_log, + 'forget': forget_log, + 'valid-time-range': time_range, + 'dump': dump, + } + + cmds_requiring_db = ['dump', 'forget'] # db must already exist + + if not cmd in cmds: + usage() + + if not os.path.exists(db_name): + if not cmd in cmds_requiring_db: + create_tables(db_name) + else: + print >> sys.stderr, 'Database "%s" does not exist' % db_name + sys.exit(1) + + cxn = sqlite3.connect(db_name) + cur = cxn.cursor() + + cmds[cmd](cur, args) + + cur.close() + cxn.commit() + cxn.close() + +if __name__ == "__main__": + main(sys.argv)