This is done with the [`distributor-threads`](settings.md#distributor-threads) setting which says how many distributors will be opened for each receiver thread. Of special importance is the choice between 1 or more backends. In case of only 1 thread, PowerDNS reverts to unthreaded operation which may be a lot faster, depending on your operating system and architecture.
-Another very important setting is [`cache-ttl`](settings.md#cache-ttl). PowerDNS caches entire packets it sends out so as to save the time to query backends to assemble all data. The default setting of 20 seconds may be low for high traffic sites, a value of 60 seconds rarely leads to problems. Please be aware that if any TTL in the answer is shorter than this setting, the packet cache will respect the answer's shortest TTL.
+Other very important settings are [`cache-ttl`](settings.md#cache-ttl). PowerDNS caches entire packets it sends out so as to save the time to query backends to assemble all data. The default setting of 20 seconds may be low for high traffic sites, a value of 60 seconds rarely leads to problems. Please be aware that if any TTL in the answer is shorter than this setting, the packet cache will respect the answer's shortest TTL.
Some PowerDNS operators set cache-ttl to many hours or even days, and use [`pdns_control`](running.md#pdns_control)` purge` to selectively or globally notify PowerDNS of changes made in the backend. Also look at the [Query Cache](#query-cache) described in this chapter. It may materially improve your performance.
Logging truly kills performance as answering a question from the cache is an order of magnitude less work than logging a line about it. Busy sites will prefer to turn [`log-dns-details`](settings.md#log-dns-details) off.
# Packet Cache
-PowerDNS by default uses the 'Packet Cache' to recognise identical questions and supply them with identical answers, without any further processing. The default time to live is 10 seconds. It has been observed that the utility of the packet cache increases with the load on your nameserver.
+PowerDNS by default uses the 'Packet Cache' to recognise identical questions and supply them with identical answers, without any further processing. The default time to live is 20 seconds and can be changed by setting `cache-ttl`. It has been observed that the utility of the packet cache increases with the load on your nameserver.
-Not all backends may benefit from the packetcache. If your backend is memory based and does not lead to context switches, the packetcache may actually hurt performance.
+Not all backends may benefit from the packet cache. If your backend is memory based and does not lead to context switches, the packet cache may actually hurt performance.
-The size of the packetcache can be observed with `/etc/init.d/pdns show packetcache-size`
+The maximum size of the packet cache is controlled by the `max-packet-cache-entries` entries since 4.1. Before that both the query cache and the packet cache used the `max-cache-entries` setting.
# Query Cache
Besides entire packets, PowerDNS can also cache individual backend queries. Each DNS query leads to a number of backend queries, the most obvious additional backend query is the check for a possible CNAME. So, when a query comes in for the 'A' record for 'www.powerdns.com', PowerDNS must first check for a CNAME for 'www.powerdns.com'.
-The Query Cache caches these backend queries, many of which are quite repetitive. PowerDNS only caches queries with no answer, or with exactly one. In the future this may be expanded but this lightweight solution is very simple and therefore fast.
+The Query Cache caches these backend queries, many of which are quite repetitive. The maximum number of entries in the cache is controlled by the `max-cache-entries` setting. Before 4.1 this setting also controls the maximum number of entries in the packet cache.
Most gain is made from caching negative entries, ie, queries that have no answer. As these take little memory to store and are typically not a real problem in terms of speed-of-propagation, the default TTL for negative queries is a rather high 60 seconds.
* `corrupt-packets`: Number of corrupt packets received
* `deferred-cache-inserts`: Number of cache inserts that were deferred because of maintenance
* `deferred-cache-lookup`: Number of cache lookups that were deferred because of maintenance
+* `deferred-packetcache-inserts`: Number of packet cache inserts that were deferred because of maintenance
+* `deferred-packetcache-lookup`: Number of packet cache lookups that were deferred because of maintenance
* `dnsupdate-answers`: Number of DNS update packets successfully answered
* `dnsupdate-changes`: Total number of changes to records from DNS update
* `dnsupdate-queries`: Number of DNS update packets received
* `qsize-q`: Number of packets waiting for database attention
* `query-cache-hit`: Number of hits on the [query cache](performance.md#query-cache)
* `query-cache-miss`: Number of misses on the [query cache](performance.md#query-cache)
+* `query-cache-size`: Number of entries in the query cache
* `rd-queries`: Number of packets sent by clients requesting recursion (regardless of if we'll be providing them with recursion). Since 3.4.0.
* `recursing-answers`: Number of packets we supplied an answer to after recursive processing
* `recursing-questions`: Number of packets we performed recursive processing for
* Integer
* Default: 1000000
-Maximum number of cache entries. 1 million (the default) will generally suffice
-for most installations.
+Maximum number of entries in the query cache. 1 million (the default) will generally suffice
+for most installations. Starting with 4.1, the packet and query caches are distinct so you might
+also want to see `max-packet-cache-entries`.
## `max-ent-entries`
* Integer
Limit the number of NSEC3 hash iterations
+## `max-packet-cache-entries`
+* Integer
+* Default: 1000000
+
+Maximum number of entries in the packet cache. 1 million (the default) will generally suffice
+for most installations. This setting has been introduced in 4.1, previous used the `max-cache-entries`
+setting for both the packet and query caches.
+
## `max-queue-length`
* Integer
* Default: 5000
libtestremotebackend_la_SOURCES = \
../../pdns/arguments.hh ../../pdns/arguments.cc \
+ ../../pdns/auth-packetcache.cc ../../pdns/auth-packetcache.hh \
+ ../../pdns/auth-querycache.cc ../../pdns/auth-querycache.hh \
../../pdns/base32.cc \
../../pdns/base64.cc \
../../pdns/dnsbackend.hh ../../pdns/dnsbackend.cc \
../../pdns/dnsrecords.cc \
../../pdns/dnssecinfra.cc \
../../pdns/ednssubnet.cc \
+ ../../pdns/ednsoptions.cc ../../pdns/ednsoptions.hh \
../../pdns/iputils.cc \
../../pdns/logger.cc \
../../pdns/misc.cc \
../../pdns/nsecrecords.cc \
- ../../pdns/packetcache.hh ../../pdns/packetcache.cc \
+ ../../pdns/packetcache.hh \
../../pdns/qtype.cc \
../../pdns/sillyrecords.cc \
../../pdns/statbag.cc \
#include "pdns/arguments.hh"
#include "pdns/json.hh"
#include "pdns/statbag.hh"
-#include "pdns/packetcache.hh"
+#include "pdns/auth-packetcache.hh"
+#include "pdns/auth-querycache.hh"
StatBag S;
-PacketCache PC;
+AuthPacketCache PC;
+AuthQueryCache QC;
ArgvMap &arg()
{
static ArgvMap arg;
#include "pdns/arguments.hh"
#include "pdns/json.hh"
#include "pdns/statbag.hh"
-#include "pdns/packetcache.hh"
+#include "pdns/auth-packetcache.hh"
+#include "pdns/auth-querycache.hh"
StatBag S;
-PacketCache PC;
+AuthPacketCache PC;
+AuthQueryCache QC;
ArgvMap &arg()
{
static ArgvMap arg;
#include "pdns/dnsrecords.hh"
#include "pdns/json.hh"
#include "pdns/statbag.hh"
-#include "pdns/packetcache.hh"
+#include "pdns/auth-packetcache.hh"
+#include "pdns/auth-querycache.hh"
StatBag S;
-PacketCache PC;
+AuthPacketCache PC;
+AuthQueryCache QC;
ArgvMap &arg()
{
static ArgvMap arg;
#include "pdns/arguments.hh"
#include "pdns/json.hh"
#include "pdns/statbag.hh"
-#include "pdns/packetcache.hh"
+#include "pdns/auth-packetcache.hh"
+#include "pdns/auth-querycache.hh"
StatBag S;
-PacketCache PC;
+AuthPacketCache PC;
+AuthQueryCache QC;
ArgvMap &arg()
{
static ArgvMap arg;
#include "pdns/dnsrecords.hh"
#include "pdns/json.hh"
#include "pdns/statbag.hh"
-#include "pdns/packetcache.hh"
+#include "pdns/auth-packetcache.hh"
+#include "pdns/auth-querycache.hh"
StatBag S;
-PacketCache PC;
+AuthPacketCache PC;
+AuthQueryCache QC;
ArgvMap &arg()
{
static ArgvMap arg;
#include "pdns/dnsrecords.hh"
#include "pdns/json.hh"
#include "pdns/statbag.hh"
-#include "pdns/packetcache.hh"
+#include "pdns/auth-packetcache.hh"
+#include "pdns/auth-querycache.hh"
StatBag S;
-PacketCache PC;
+AuthPacketCache PC;
+AuthQueryCache QC;
ArgvMap &arg()
{
static ArgvMap arg;
#include "pdns/arguments.hh"
#include "pdns/json.hh"
#include "pdns/statbag.hh"
-#include "pdns/packetcache.hh"
#include "test-remotebackend-keys.hh"
pdns_server_SOURCES = \
arguments.cc arguments.hh \
auth-carbon.cc \
+ auth-caches.cc auth-caches.hh \
+ auth-packetcache.cc auth-packetcache.hh \
+ auth-querycache.cc auth-querycache.hh \
backends/gsql/gsqlbackend.cc backends/gsql/gsqlbackend.hh \
backends/gsql/ssql.hh \
base32.cc base32.hh \
dynhandler.cc dynhandler.hh \
dynlistener.cc dynlistener.hh \
dynmessenger.hh \
+ ednsoptions.cc ednsoptions.hh \
ednssubnet.cc ednssubnet.hh \
gss_context.cc gss_context.hh \
iputils.cc iputils.hh \
namespaces.hh \
nsecrecords.cc \
opensslsigners.cc opensslsigners.hh \
- packetcache.cc packetcache.hh \
+ packetcache.hh \
packethandler.cc packethandler.hh \
pdnsexception.hh \
qtype.cc qtype.hh \
pdnsutil_SOURCES = \
arguments.cc \
+ auth-caches.cc auth-caches.hh \
+ auth-packetcache.cc auth-packetcache.hh \
+ auth-querycache.cc auth-querycache.hh \
backends/gsql/gsqlbackend.cc backends/gsql/gsqlbackend.hh \
backends/gsql/ssql.hh \
base32.cc \
dnssecsigner.cc \
dnswriter.cc dnswriter.hh \
dynlistener.cc \
+ ednsoptions.cc ednsoptions.hh \
ednssubnet.cc \
gss_context.cc gss_context.hh \
iputils.cc iputils.hh \
misc.cc misc.hh \
nsecrecords.cc \
opensslsigners.cc opensslsigners.hh \
- packetcache.cc \
pdnsutil.cc \
qtype.cc \
randomhelper.cc \
dnsrecords.cc \
dnsreplay.cc \
dnswriter.cc dnswriter.hh \
- ednssubnet.cc ednssubnet.hh \
ednsoptions.cc ednsoptions.hh \
+ ednssubnet.cc ednssubnet.hh \
iputils.cc \
logger.cc \
misc.cc \
testrunner_SOURCES = \
arguments.cc \
+ auth-caches.cc auth-caches.hh \
+ auth-packetcache.cc auth-packetcache.hh \
+ auth-querycache.cc auth-querycache.hh \
base32.cc \
base64.cc \
bindlexer.l \
misc.cc \
nameserver.cc \
nsecrecords.cc \
- packetcache.cc \
qtype.cc \
rcpgenerator.cc \
recpacketcache.cc recpacketcache.hh \
--- /dev/null
+/*
+ * This file is part of PowerDNS or dnsdist.
+ * Copyright -- PowerDNS.COM B.V. and its contributors
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * In addition, for the avoidance of any doubt, permission is granted to
+ * link this program with OpenSSL and to (re)distribute the binaries
+ * produced as the result of such linking.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ */
+
+#include "auth-querycache.hh"
+#include "auth-packetcache.hh"
+
+extern AuthPacketCache PC;
+extern AuthQueryCache QC;
+
+/* empty all caches */
+uint64_t purgeAuthCaches()
+{
+ uint64_t ret = 0;
+ ret += PC.purge();
+ ret += QC.purge();
+ return ret;
+}
+
+ /* remove specific entries from all caches, can be $ terminated */
+uint64_t purgeAuthCaches(const std::string& match)
+{
+ uint64_t ret = 0;
+ ret += PC.purge(match);
+ ret += QC.purge(match);
+ return ret;
+}
+
+/* remove specific entries from all caches, no wildcard matching */
+uint64_t purgeAuthCachesExact(const DNSName& qname)
+{
+ uint64_t ret = 0;
+ ret += PC.purgeExact(qname);
+ ret += QC.purgeExact(qname);
+ return ret;
+}
+
+
+
--- /dev/null
+/*
+ * This file is part of PowerDNS or dnsdist.
+ * Copyright -- PowerDNS.COM B.V. and its contributors
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * In addition, for the avoidance of any doubt, permission is granted to
+ * link this program with OpenSSL and to (re)distribute the binaries
+ * produced as the result of such linking.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ */
+#ifndef AUTH_CACHES_HH
+#define AUTH_CACHES_HH
+
+#include <string>
+#include <cstdint>
+
+#include "dnsname.hh"
+
+uint64_t purgeAuthCaches(); /* empty all caches */
+uint64_t purgeAuthCaches(const std::string& match); /* remove specific entries from all caches, can be $ terminated */
+uint64_t purgeAuthCachesExact(const DNSName& qname); /* remove specific entries from all caches, no wildcard matching */
+
+#endif /* AUTH_CACHES_HH */
--- /dev/null
+/*
+ * This file is part of PowerDNS or dnsdist.
+ * Copyright -- PowerDNS.COM B.V. and its contributors
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * In addition, for the avoidance of any doubt, permission is granted to
+ * link this program with OpenSSL and to (re)distribute the binaries
+ * produced as the result of such linking.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ */
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+#include "auth-packetcache.hh"
+#include "logger.hh"
+#include "statbag.hh"
+#include "cachecleaner.hh"
+extern StatBag S;
+
+const unsigned int AuthPacketCache::s_mincleaninterval, AuthPacketCache::s_maxcleaninterval;
+
+AuthPacketCache::AuthPacketCache(size_t mapsCount): d_lastclean(time(nullptr))
+{
+ d_maps.resize(mapsCount);
+ for(auto& mc : d_maps) {
+ pthread_rwlock_init(&mc.d_mut, 0);
+ }
+
+ S.declare("packetcache-hit", "Number of hits on the packet cache");
+ S.declare("packetcache-miss", "Number of misses on the packet cache");
+ S.declare("packetcache-size", "Number of entries in the packet cache");
+ S.declare("deferred-packetcache-inserts","Amount of packet cache inserts that were deferred because of maintenance");
+ S.declare("deferred-packetcache-lookup","Amount of packet cache lookups that were deferred because of maintenance");
+
+ d_statnumhit=S.getPointer("packetcache-hit");
+ d_statnummiss=S.getPointer("packetcache-miss");
+ d_statnumentries=S.getPointer("packetcache-size");
+}
+
+AuthPacketCache::~AuthPacketCache()
+{
+ try {
+ vector<WriteLock*> locks;
+ for(auto& mc : d_maps) {
+ locks.push_back(new WriteLock(&mc.d_mut));
+ }
+ for(auto wl : locks) {
+ delete wl;
+ }
+ }
+ catch(...) {
+ }
+}
+
+bool AuthPacketCache::get(DNSPacket *p, DNSPacket *cached)
+{
+ cleanupIfNeeded();
+
+ if(!d_ttl) {
+ (*d_statnummiss)++;
+ return false;
+ }
+
+ uint32_t hash = canHashPacket(p->getString(), false);
+ p->setHash(hash);
+
+ string value;
+ bool haveSomething;
+ time_t now = time(nullptr);
+ auto& mc = getMap(p->qdomain);
+ {
+ TryReadLock rl(&mc.d_mut);
+ if(!rl.gotIt()) {
+ S.inc("deferred-packetcache-lookup");
+ return false;
+ }
+
+ haveSomething = getEntryLocked(mc.d_map, hash, p->qdomain, p->qtype.getCode(), p->d_tcp, now, value);
+ }
+
+ if (!haveSomething) {
+ (*d_statnummiss)++;
+ return false;
+ }
+
+ if(cached->noparse(value.c_str(), value.size()) < 0) {
+ return false;
+ }
+
+ (*d_statnumhit)++;
+ cached->spoofQuestion(p); // for correct case
+ cached->qdomain = p->qdomain;
+ cached->qtype = p->qtype;
+
+ return true;
+}
+
+void AuthPacketCache::insert(DNSPacket *q, DNSPacket *r, unsigned int maxTTL)
+{
+ cleanupIfNeeded();
+
+ if (ntohs(q->d.qdcount) != 1) {
+ return; // do not try to cache packets with multiple questions
+ }
+
+ if (q->qclass != QClass::IN) // we only cache the INternet
+ return;
+
+ uint32_t ourttl = std::min(d_ttl, maxTTL);
+ if (ourttl == 0) {
+ return;
+ }
+
+ uint32_t hash = q->getHash();
+ time_t now = time(nullptr);
+ CacheEntry entry;
+ entry.hash = hash;
+ entry.created = now;
+ entry.ttd = now + ourttl;
+ entry.qname = q->qdomain;
+ entry.qtype = q->qtype.getCode();
+ entry.value = r->getString();
+ entry.tcp = r->d_tcp;
+
+ auto& mc = getMap(entry.qname);
+ {
+ TryWriteLock l(&mc.d_mut);
+ if (!l.gotIt()) {
+ S.inc("deferred-packetcache-inserts");
+ return;
+ }
+
+ auto& idx = mc.d_map.get<HashTag>();
+ auto range = idx.equal_range(hash);
+ auto iter = range.first;
+
+ for( ; iter != range.second ; ++iter) {
+ if (iter->tcp != entry.tcp || iter->qtype != entry.qtype || iter->qname != entry.qname)
+ continue;
+
+ iter->value = entry.value;
+ iter->ttd = now + ourttl;
+ iter->created = now;
+ return;
+ }
+
+ /* no existing entry found to refresh */
+ mc.d_map.insert(entry);
+ (*d_statnumentries)++;
+ }
+}
+
+bool AuthPacketCache::getEntryLocked(cmap_t& map, uint32_t hash, const DNSName &qname, uint16_t qtype, bool tcp, time_t now, string& value)
+{
+ auto& idx = map.get<HashTag>();
+ auto range = idx.equal_range(hash);
+
+ for(auto iter = range.first; iter != range.second ; ++iter) {
+ if (iter->ttd < now)
+ continue;
+
+ if (iter->tcp != tcp || iter->qtype != qtype || iter->qname != qname)
+ continue;
+
+ value = iter->value;
+ return true;
+ }
+
+ return false;
+}
+
+/* clears the entire cache. */
+uint64_t AuthPacketCache::purge()
+{
+ d_statnumentries->store(0);
+
+ return purgeLockedCollectionsVector(d_maps);
+}
+
+uint64_t AuthPacketCache::purgeExact(const DNSName& qname)
+{
+ auto& mc = getMap(qname);
+ uint64_t delcount = purgeExactLockedCollection(mc, qname);
+
+ *d_statnumentries -= delcount;
+
+ return delcount;
+}
+
+/* purges entries from the packetcache. If match ends on a $, it is treated as a suffix */
+uint64_t AuthPacketCache::purge(const string &match)
+{
+ uint64_t delcount = 0;
+
+ if(ends_with(match, "$")) {
+ delcount = purgeLockedCollectionsVector(d_maps, match);
+ *d_statnumentries -= delcount;
+ }
+ else {
+ delcount = purgeExact(DNSName(match));
+ }
+
+ return delcount;
+}
+
+void AuthPacketCache::cleanup()
+{
+ uint64_t maxCached = d_maxEntries;
+ uint64_t cacheSize = *d_statnumentries;
+ uint64_t totErased = 0;
+
+ totErased = pruneLockedCollectionsVector(d_maps, maxCached, cacheSize);
+ *d_statnumentries -= totErased;
+
+ DLOG(L<<"Done with cache clean, cacheSize: "<<(*d_statnumentries)<<", totErased"<<totErased<<endl);
+}
+
+/* the logic:
+ after d_nextclean operations, we clean. We also adjust the cleaninterval
+ a bit so we slowly move it to a value where we clean roughly every 30 seconds.
+
+ If d_nextclean has reached its maximum value, we also test if we were called
+ within 30 seconds, and if so, we skip cleaning. This means that under high load,
+ we will not clean more often than every 30 seconds anyhow.
+*/
+
+void AuthPacketCache::cleanupIfNeeded()
+{
+ if (d_ops++ == d_nextclean) {
+ time_t now = time(nullptr);
+ int timediff = max((int)(now - d_lastclean), 1);
+
+ DLOG(L<<"cleaninterval: "<<d_cleaninterval<<", timediff: "<<timediff<<endl);
+
+ if (d_cleaninterval == s_maxcleaninterval && timediff < 30) {
+ d_cleanskipped = true;
+ d_nextclean += d_cleaninterval;
+
+ DLOG(L<<"cleaning skipped, timediff: "<<timediff<<endl);
+
+ return;
+ }
+
+ if(!d_cleanskipped) {
+ d_cleaninterval=(int)(0.6*d_cleaninterval)+(0.4*d_cleaninterval*(30.0/timediff));
+ d_cleaninterval=std::max(d_cleaninterval, s_mincleaninterval);
+ d_cleaninterval=std::min(d_cleaninterval, s_maxcleaninterval);
+
+ DLOG(L<<"new cleaninterval: "<<d_cleaninterval<<endl);
+ } else {
+ d_cleanskipped = false;
+ }
+
+ d_nextclean += d_cleaninterval;
+ d_lastclean=now;
+ cleanup();
+ }
+}
--- /dev/null
+/*
+ * This file is part of PowerDNS or dnsdist.
+ * Copyright -- PowerDNS.COM B.V. and its contributors
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * In addition, for the avoidance of any doubt, permission is granted to
+ * link this program with OpenSSL and to (re)distribute the binaries
+ * produced as the result of such linking.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ */
+#ifndef AUTH_PACKETCACHE_HH
+#define AUTH_PACKETCACHE_HH
+
+#include <string>
+#include <map>
+#include "dns.hh"
+#include <boost/version.hpp>
+#include "namespaces.hh"
+using namespace ::boost::multi_index;
+
+#include <boost/multi_index/hashed_index.hpp>
+
+#include "dnspacket.hh"
+#include "lock.hh"
+#include "packetcache.hh"
+
+/** This class performs 'whole packet caching'. Feed it a question packet and it will
+ try to find an answer. If you have an answer, insert it to have it cached for later use.
+ Take care not to replace existing cache entries. While this works, it is wasteful. Only
+ insert packets that where not found by get()
+
+ Locking!
+
+ The cache itself is protected by a read/write lock. Because deleting is a two step process, which
+ first marks and then sweeps, a second lock is present to prevent simultaneous inserts and deletes.
+*/
+
+class AuthPacketCache : public PacketCache
+{
+public:
+ AuthPacketCache(size_t mapsCount=1024);
+ ~AuthPacketCache();
+
+ void insert(DNSPacket *q, DNSPacket *r, uint32_t maxTTL); //!< We copy the contents of *p into our cache. Do not needlessly call this to insert questions already in the cache as it wastes resources
+
+ bool get(DNSPacket *p, DNSPacket *q); //!< We return a dynamically allocated copy out of our cache. You need to delete it. You also need to spoof in the right ID with the DNSPacket.spoofID() method.
+
+ void cleanup(); //!< force the cache to preen itself from expired packets
+ uint64_t purge();
+ uint64_t purge(const std::string& match); // could be $ terminated. Is not a dnsname!
+ uint64_t purgeExact(const DNSName& qname); // no wildcard matching here
+
+ uint64_t size() const { return *d_statnumentries; };
+
+ void setMaxEntries(uint64_t maxEntries)
+ {
+ d_maxEntries = maxEntries;
+ }
+ void setTTL(uint32_t ttl)
+ {
+ d_ttl = ttl;
+ }
+private:
+
+ struct CacheEntry
+ {
+ mutable string value;
+ DNSName qname;
+
+ mutable time_t created{0};
+ mutable time_t ttd{0};
+ uint32_t hash{0};
+ uint16_t qtype{0};
+ bool tcp{false};
+ };
+
+ struct HashTag{};
+ struct NameTag{};
+ struct SequenceTag{};
+ typedef multi_index_container<
+ CacheEntry,
+ indexed_by <
+ hashed_non_unique<tag<HashTag>, member<CacheEntry,uint32_t,&CacheEntry::hash> >,
+ ordered_non_unique<tag<NameTag>, member<CacheEntry,DNSName,&CacheEntry::qname>, CanonDNSNameCompare >,
+ sequenced<tag<SequenceTag>>
+ >
+ > cmap_t;
+
+ struct MapCombo
+ {
+ pthread_rwlock_t d_mut;
+ cmap_t d_map;
+ };
+
+ vector<MapCombo> d_maps;
+ MapCombo& getMap(const DNSName& name)
+ {
+ return d_maps[name.hash() % d_maps.size()];
+ }
+
+ bool getEntryLocked(cmap_t& map, uint32_t hash, const DNSName &qname, uint16_t qtype, bool tcp, time_t now, string& entry);
+ void cleanupIfNeeded();
+
+ AtomicCounter d_ops{0};
+ AtomicCounter *d_statnumhit;
+ AtomicCounter *d_statnummiss;
+ AtomicCounter *d_statnumentries;
+
+ uint64_t d_maxEntries{0};
+ time_t d_lastclean; // doesn't need to be atomic
+ unsigned long d_nextclean{4096};
+ unsigned int d_cleaninterval{4096};
+ uint32_t d_ttl{0};
+ bool d_cleanskipped{false};
+
+ static const unsigned int s_mincleaninterval=1000, s_maxcleaninterval=300000;
+};
+
+#endif /* AUTH_PACKETCACHE_HH */
--- /dev/null
+/*
+ * This file is part of PowerDNS or dnsdist.
+ * Copyright -- PowerDNS.COM B.V. and its contributors
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * In addition, for the avoidance of any doubt, permission is granted to
+ * link this program with OpenSSL and to (re)distribute the binaries
+ * produced as the result of such linking.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ */
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include "auth-querycache.hh"
+#include "logger.hh"
+#include "statbag.hh"
+#include "cachecleaner.hh"
+extern StatBag S;
+
+const unsigned int AuthQueryCache::s_mincleaninterval, AuthQueryCache::s_maxcleaninterval;
+
+extern StatBag S;
+
+AuthQueryCache::AuthQueryCache(size_t mapsCount): d_lastclean(time(nullptr))
+{
+ d_maps.resize(mapsCount);
+ for(auto& mc : d_maps) {
+ pthread_rwlock_init(&mc.d_mut, 0);
+ }
+
+ S.declare("query-cache-hit","Number of hits on the query cache");
+ S.declare("query-cache-miss","Number of misses on the query cache");
+ S.declare("query-cache-size", "Number of entries in the query cache");
+ S.declare("deferred-cache-inserts","Amount of cache inserts that were deferred because of maintenance");
+ S.declare("deferred-cache-lookup","Amount of cache lookups that were deferred because of maintenance");
+
+ d_statnumhit=S.getPointer("query-cache-hit");
+ d_statnummiss=S.getPointer("query-cache-miss");
+ d_statnumentries=S.getPointer("query-cache-size");
+}
+
+AuthQueryCache::~AuthQueryCache()
+{
+ try {
+ vector<WriteLock*> locks;
+ for(auto& mc : d_maps) {
+ locks.push_back(new WriteLock(&mc.d_mut));
+ }
+ for(auto wl : locks) {
+ delete wl;
+ }
+ }
+ catch(...) {
+ }
+}
+
+// called from ueberbackend
+bool AuthQueryCache::getEntry(const DNSName &qname, const QType& qtype, vector<DNSZoneRecord>& value, int zoneID)
+{
+ cleanupIfNeeded();
+
+ time_t now = time(nullptr);
+ uint16_t qt = qtype.getCode();
+ auto& mc = getMap(qname);
+ {
+ TryReadLock rl(&mc.d_mut);
+ if(!rl.gotIt()) {
+ S.inc("deferred-cache-lookup");
+ return false;
+ }
+
+ return getEntryLocked(mc.d_map, qname, qt, value, zoneID, now);
+ }
+}
+
+void AuthQueryCache::insert(const DNSName &qname, const QType& qtype, const vector<DNSZoneRecord>& value, uint32_t ttl, int zoneID)
+{
+ cleanupIfNeeded();
+
+ if(!ttl)
+ return;
+
+ time_t now = time(nullptr);
+ CacheEntry val;
+ val.created = now;
+ val.ttd = now + ttl;
+ val.qname = qname;
+ val.qtype = qtype.getCode();
+ val.drs = value;
+ val.zoneID = zoneID;
+
+ auto& mc = getMap(val.qname);
+
+ {
+ TryWriteLock l(&mc.d_mut);
+ if(!l.gotIt()) {
+ S.inc("deferred-cache-inserts");
+ return;
+ }
+
+ bool inserted;
+ cmap_t::iterator place;
+ tie(place, inserted) = mc.d_map.insert(val);
+
+ if (!inserted) {
+ mc.d_map.replace(place, val);
+ }
+ else {
+ (*d_statnumentries)++;
+ }
+ }
+}
+
+bool AuthQueryCache::getEntryLocked(cmap_t& map, const DNSName &qname, uint16_t qtype, vector<DNSZoneRecord>& value, int zoneID, time_t now)
+{
+ auto& idx = boost::multi_index::get<HashTag>(map);
+ auto iter = idx.find(tie(qname, qtype, zoneID));
+
+ if (iter == idx.end())
+ return false;
+
+ if (iter->ttd < now)
+ return false;
+
+ value = iter->drs;
+ return true;
+}
+
+map<char,uint64_t> AuthQueryCache::getCounts()
+{
+ uint64_t queryCacheEntries=0, negQueryCacheEntries=0;
+
+ for(auto& mc : d_maps) {
+ ReadLock l(&mc.d_mut);
+
+ for(cmap_t::const_iterator iter = mc.d_map.begin() ; iter != mc.d_map.end(); ++iter) {
+ if(iter->drs.empty())
+ negQueryCacheEntries++;
+ else
+ queryCacheEntries++;
+ }
+ }
+ map<char,uint64_t> ret;
+
+ ret['!']=negQueryCacheEntries;
+ ret['Q']=queryCacheEntries;
+ return ret;
+}
+
+/* clears the entire cache. */
+uint64_t AuthQueryCache::purge()
+{
+ d_statnumentries->store(0);
+
+ return purgeLockedCollectionsVector(d_maps);
+}
+
+uint64_t AuthQueryCache::purgeExact(const DNSName& qname)
+{
+ auto& mc = getMap(qname);
+ uint64_t delcount = purgeExactLockedCollection(mc, qname);
+
+ *d_statnumentries -= delcount;
+
+ return delcount;
+}
+
+/* purges entries from the querycache. If match ends on a $, it is treated as a suffix */
+uint64_t AuthQueryCache::purge(const string &match)
+{
+ uint64_t delcount = 0;
+
+ if(ends_with(match, "$")) {
+ delcount = purgeLockedCollectionsVector(d_maps, match);
+ *d_statnumentries -= delcount;
+ }
+ else {
+ delcount = purgeExact(DNSName(match));
+ }
+
+ return delcount;
+}
+
+void AuthQueryCache::cleanup()
+{
+ uint64_t maxCached = d_maxEntries;
+ uint64_t cacheSize = *d_statnumentries;
+ uint64_t totErased = 0;
+
+ totErased = pruneLockedCollectionsVector(d_maps, maxCached, cacheSize);
+
+ *d_statnumentries -= totErased;
+ DLOG(L<<"Done with cache clean, cacheSize: "<<*d_statnumentries<<", totErased"<<totErased<<endl);
+}
+
+/* the logic:
+ after d_nextclean operations, we clean. We also adjust the cleaninterval
+ a bit so we slowly move it to a value where we clean roughly every 30 seconds.
+
+ If d_nextclean has reached its maximum value, we also test if we were called
+ within 30 seconds, and if so, we skip cleaning. This means that under high load,
+ we will not clean more often than every 30 seconds anyhow.
+*/
+
+void AuthQueryCache::cleanupIfNeeded()
+{
+ if (d_ops++ == d_nextclean) {
+ time_t now = time(nullptr);
+ int timediff = max((int)(now - d_lastclean), 1);
+
+ DLOG(L<<"cleaninterval: "<<d_cleaninterval<<", timediff: "<<timediff<<endl);
+
+ if (d_cleaninterval == s_maxcleaninterval && timediff < 30) {
+ d_cleanskipped = true;
+ d_nextclean += d_cleaninterval;
+
+ DLOG(L<<"cleaning skipped, timediff: "<<timediff<<endl);
+
+ return;
+ }
+
+ if(!d_cleanskipped) {
+ d_cleaninterval=(int)(0.6*d_cleaninterval)+(0.4*d_cleaninterval*(30.0/timediff));
+ d_cleaninterval=std::max(d_cleaninterval, s_mincleaninterval);
+ d_cleaninterval=std::min(d_cleaninterval, s_maxcleaninterval);
+
+ DLOG(L<<"new cleaninterval: "<<d_cleaninterval<<endl);
+ } else {
+ d_cleanskipped = false;
+ }
+
+ d_nextclean += d_cleaninterval;
+ d_lastclean=now;
+ cleanup();
+ }
+}
--- /dev/null
+/*
+ * This file is part of PowerDNS or dnsdist.
+ * Copyright -- PowerDNS.COM B.V. and its contributors
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * In addition, for the avoidance of any doubt, permission is granted to
+ * link this program with OpenSSL and to (re)distribute the binaries
+ * produced as the result of such linking.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ */
+#ifndef AUTH_QUERYCACHE_HH
+#define AUTH_QUERYCACHE_HH
+
+#include <string>
+#include <map>
+#include "dns.hh"
+#include <boost/version.hpp>
+#include "namespaces.hh"
+using namespace ::boost::multi_index;
+
+#include <boost/multi_index/hashed_index.hpp>
+
+#include "dns.hh"
+#include "dnspacket.hh"
+#include "lock.hh"
+
+class AuthQueryCache : public boost::noncopyable
+{
+public:
+ AuthQueryCache(size_t mapsCount=1024);
+ ~AuthQueryCache();
+
+ void insert(const DNSName &qname, const QType& qtype, const vector<DNSZoneRecord>& content, uint32_t ttl, int zoneID);
+
+ bool getEntry(const DNSName &qname, const QType& qtype, vector<DNSZoneRecord>& entry, int zoneID);
+
+ size_t size() { return *d_statnumentries; } //!< number of entries in the cache
+ void cleanup(); //!< force the cache to preen itself from expired querys
+ uint64_t purge();
+ uint64_t purge(const std::string& match); // could be $ terminated. Is not a dnsname!
+ uint64_t purgeExact(const DNSName& qname); // no wildcard matching here
+
+ map<char,uint64_t> getCounts();
+
+ void setMaxEntries(uint64_t maxEntries)
+ {
+ d_maxEntries = maxEntries;
+ }
+private:
+
+ struct CacheEntry
+ {
+ DNSName qname;
+ mutable vector<DNSZoneRecord> drs;
+ mutable time_t created{0};
+ mutable time_t ttd{0};
+ uint16_t qtype{0};
+ int zoneID{-1};
+ };
+
+ struct HashTag{};
+ struct NameTag{};
+ struct SequenceTag{};
+ typedef multi_index_container<
+ CacheEntry,
+ indexed_by <
+ hashed_unique<tag<HashTag>, composite_key<CacheEntry,
+ member<CacheEntry,DNSName,&CacheEntry::qname>,
+ member<CacheEntry,uint16_t,&CacheEntry::qtype>,
+ member<CacheEntry,int, &CacheEntry::zoneID> > > ,
+ ordered_non_unique<tag<NameTag>, member<CacheEntry,DNSName,&CacheEntry::qname>, CanonDNSNameCompare >,
+ sequenced<tag<SequenceTag>>
+ >
+ > cmap_t;
+
+
+ struct MapCombo
+ {
+ pthread_rwlock_t d_mut;
+ cmap_t d_map;
+ };
+
+ vector<MapCombo> d_maps;
+ MapCombo& getMap(const DNSName& qname)
+ {
+ return d_maps[qname.hash() % d_maps.size()];
+ }
+
+ bool getEntryLocked(cmap_t& map, const DNSName &content, uint16_t qtype, vector<DNSZoneRecord>& entry, int zoneID, time_t now);
+ void cleanupIfNeeded();
+
+ AtomicCounter d_ops{0};
+ AtomicCounter *d_statnumhit;
+ AtomicCounter *d_statnummiss;
+ AtomicCounter *d_statnumentries;
+
+ uint64_t d_maxEntries{0};
+ time_t d_lastclean; // doesn't need to be atomic
+ unsigned long d_nextclean{4906};
+ unsigned int d_cleaninterval{4096};
+ bool d_cleanskipped{false};
+
+ static const unsigned int s_mincleaninterval=1000, s_maxcleaninterval=300000;
+};
+
+#endif /* AUTH_QUERYCACHE_HH */
#ifndef PDNS_CACHECLEANER_HH
#define PDNS_CACHECLEANER_HH
+#include "lock.hh"
+
// this function can clean any cache that has a getTTD() method on its entries, and a 'sequence' index as its second index
// the ritual is that the oldest entries are in *front* of the sequence collection, so on a hit, move an item to the end
// on a miss, move it to the beginning
moveCacheItemToFrontOrBack(collection, iter, false);
}
+template <typename T> uint64_t pruneLockedCollectionsVector(vector<T>& maps, uint64_t maxCached, uint64_t cacheSize)
+{
+ time_t now = time(nullptr);
+ uint64_t totErased = 0;
+ uint64_t toTrim = 0;
+ uint64_t lookAt = 0;
+
+ // two modes - if toTrim is 0, just look through 10% of the cache and nuke everything that is expired
+ // otherwise, scan first 5*toTrim records, and stop once we've nuked enough
+ if (maxCached && cacheSize > maxCached) {
+ toTrim = cacheSize - maxCached;
+ lookAt = 5 * toTrim;
+ } else {
+ lookAt = cacheSize / 10;
+ }
+
+ for(auto& mc : maps) {
+ WriteLock wl(&mc.d_mut);
+ auto& sidx = boost::multi_index::get<2>(mc.d_map);
+ uint64_t erased = 0, lookedAt = 0;
+ for(auto i = sidx.begin(); i != sidx.end(); lookedAt++) {
+ if(i->ttd < now) {
+ i = sidx.erase(i);
+ erased++;
+ } else {
+ ++i;
+ }
+
+ if(toTrim && erased > toTrim / maps.size())
+ break;
+
+ if(lookedAt > lookAt / maps.size())
+ break;
+ }
+ totErased += erased;
+ }
+
+ return totErased;
+}
+
+template <typename T> uint64_t purgeLockedCollectionsVector(vector<T>& maps)
+{
+ uint64_t delcount=0;
+
+ for(auto& mc : maps) {
+ WriteLock wl(&mc.d_mut);
+ delcount += mc.d_map.size();
+ mc.d_map.clear();
+ }
+
+ return delcount;
+}
+
+template <typename T> uint64_t purgeLockedCollectionsVector(vector<T>& maps, const std::string& match)
+{
+ uint64_t delcount=0;
+ string prefix(match);
+ prefix.resize(prefix.size()-1);
+ DNSName dprefix(prefix);
+ for(auto& mc : maps) {
+ WriteLock wl(&mc.d_mut);
+ auto& idx = boost::multi_index::get<1>(mc.d_map);
+ auto iter = idx.lower_bound(dprefix);
+ auto start = iter;
+
+ for(; iter != idx.end(); ++iter) {
+ if(!iter->qname.isPartOf(dprefix)) {
+ break;
+ }
+ delcount++;
+ }
+ idx.erase(start, iter);
+ }
+
+ return delcount;
+}
+
+template <typename T> uint64_t purgeExactLockedCollection(T& mc, const DNSName& qname)
+{
+ uint64_t delcount=0;
+ WriteLock wl(&mc.d_mut);
+ auto& idx = boost::multi_index::get<1>(mc.d_map);
+ auto range = idx.equal_range(qname);
+ if(range.first != range.second) {
+ delcount += distance(range.first, range.second);
+ idx.erase(range.first, range.second);
+ }
+
+ return delcount;
+}
+
#endif
ArgvMap theArg;
StatBag S; //!< Statistics are gathered across PDNS via the StatBag class S
-PacketCache PC; //!< This is the main PacketCache, shared across all threads
+AuthPacketCache PC; //!< This is the main PacketCache, shared across all threads
+AuthQueryCache QC;
DNSProxy *DP;
DynListener *dl;
CommunicatorClass Communicator;
::arg().set("setuid","If set, change user id to this uid for more security")="";
::arg().set("setgid","If set, change group id to this gid for more security")="";
- ::arg().set("max-cache-entries", "Maximum number of cache entries")="1000000";
+ ::arg().set("max-cache-entries", "Maximum number of entries in the query cache")="1000000";
+ ::arg().set("max-packet-cache-entries", "Maximum number of entries in the packet cache")="1000000";
::arg().set("max-signature-cache-entries", "Maximum number of signatures cache entries")="";
::arg().set("max-ent-entries", "Maximum number of empty non-terminals in a zone")="100000";
::arg().set("entropy-source", "If set, read entropy from this file")="/dev/urandom";
S.declare("qsize-q","Number of questions waiting for database attention", getQCount);
- S.declare("deferred-cache-inserts","Amount of cache inserts that were deferred because of maintenance");
- S.declare("deferred-cache-lookup","Amount of cache lookups that were deferred because of maintenance");
-
- S.declare("query-cache-hit","Number of hits on the query cache");
- S.declare("query-cache-miss","Number of misses on the query cache");
-
S.declare("dnsupdate-queries", "DNS update packets received.");
S.declare("dnsupdate-answers", "DNS update packets successfully answered.");
S.declare("dnsupdate-refused", "DNS update packets that are refused.");
continue;
}
}
-
+
if(distributor->isOverloaded()) {
if(logDNSQueries)
L<<"Dropped query, backends are overloaded"<<endl;
DNSPacket::s_udpTruncationThreshold = std::max(512, ::arg().asNum("udp-truncation-threshold"));
DNSPacket::s_doEDNSSubnetProcessing = ::arg().mustDo("edns-subnet-processing");
+ PC.setTTL(::arg().asNum("cache-ttl"));
+ PC.setMaxEntries(::arg().asNum("max-packet-cache-entries"));
+ QC.setMaxEntries(::arg().asNum("max-cache-entries"));
+
stubParseResolveConf();
if(!::arg()["chroot"].empty()) {
#ifndef COMMON_STARTUP_HH
#define COMMON_STARTUP_HH
-#include "packetcache.hh"
+#include "auth-packetcache.hh"
+#include "auth-querycache.hh"
#include "utility.hh"
#include "arguments.hh"
#include "communicator.hh"
extern ArgvMap theArg;
extern StatBag S; //!< Statistics are gathered across PDNS via the StatBag class S
-extern PacketCache PC; //!< This is the main PacketCache, shared across all threads
+extern AuthPacketCache PC; //!< This is the main PacketCache, shared across all threads
+extern AuthQueryCache QC;
extern DNSProxy *DP;
extern DynListener *dl;
extern CommunicatorClass Communicator;
d=orig.d;
d_isQuery = orig.d_isQuery;
+ d_hash = orig.d_hash;
}
void DNSPacket::setRcode(int v)
d_ednsrcode=extRCode;
};
uint8_t getEDNSRCode() const { return d_ednsrcode; };
+ uint32_t getHash() const { return d_hash; };
+ void setHash(uint32_t hash) { d_hash = hash; };
+
//////// DATA !
DNSName qdomain; //!< qname of the question 4 - unsure how this is used
// WARNING! This is really 12 bits
uint16_t d_ednsrcode;
+ uint32_t d_hash{0};
+
bool d_compress; // 1
bool d_tsigtimersonly;
bool d_wantsnsid;
#include "dns_random.hh"
extern StatBag S;
-extern PacketCache PC;
DNSProxy::DNSProxy(const string &remote)
{
if(sendmsg(i->second.outsock, &msgh, 0) < 0)
L<<Logger::Warning<<"dnsproxy.cc: Error sending reply with sendmsg (socket="<<i->second.outsock<<"): "<<strerror(errno)<<endl;
- PC.insert(&q, &p, true);
i->second.created=0;
}
}
d_sor=d_content.size(); // this will remind us where to stuff the record size
}
-void DNSPacketWriter::addOpt(uint16_t udpsize, int extRCode, int Z, const vector<pair<uint16_t,string> >& options)
+void DNSPacketWriter::addOpt(uint16_t udpsize, int extRCode, int Z, const vector<pair<uint16_t,string> >& options, uint8_t version)
{
uint32_t ttl=0;
EDNS0Record stuff;
stuff.extRCode=extRCode;
- stuff.version=0;
+ stuff.version=version;
stuff.Z=htons(Z);
static_assert(sizeof(EDNS0Record) == sizeof(ttl), "sizeof(EDNS0Record) must match sizeof(ttl)");
/** Shorthand way to add an Opt-record, for example for EDNS0 purposes */
typedef vector<pair<uint16_t,std::string> > optvect_t;
- void addOpt(uint16_t udpsize, int extRCode, int Z, const optvect_t& options=optvect_t());
+ void addOpt(uint16_t udpsize, int extRCode, int Z, const optvect_t& options=optvect_t(), uint8_t version=0);
/** needs to be called after the last record is added, but can be called again and again later on. Is called internally by startRecord too.
The content of the vector<> passed to the constructor is inconsistent until commit is called.
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
-#include "packetcache.hh"
+#include "auth-caches.hh"
+#include "auth-querycache.hh"
+#include "auth-packetcache.hh"
#include "utility.hh"
#include "dynhandler.hh"
#include "statbag.hh"
string DLPurgeHandler(const vector<string>&parts, Utility::pid_t ppid)
{
- extern PacketCache PC;
DNSSECKeeper dk;
ostringstream os;
int ret=0;
if(parts.size()>1) {
for (vector<string>::const_iterator i=++parts.begin();i<parts.end();++i) {
- ret+=PC.purge(*i);
+ ret+=purgeAuthCaches(*i);
if(!boost::ends_with(*i, "$"))
dk.clearCaches(DNSName(*i));
else
}
}
else {
- ret=PC.purge();
+ ret = purgeAuthCaches();
dk.clearAllCaches();
}
string DLCCHandler(const vector<string>&parts, Utility::pid_t ppid)
{
- extern PacketCache PC;
- map<char,int> counts=PC.getCounts();
+ extern AuthPacketCache PC;
+ extern AuthQueryCache QC;
+ map<char,uint64_t> counts=QC.getCounts();
+ uint64_t packetEntries = PC.size();
ostringstream os;
bool first=true;
- for(map<char,int>::const_iterator i=counts.begin();i!=counts.end();++i) {
+ for(map<char,uint64_t>::const_iterator i=counts.begin();i!=counts.end();++i) {
if(!first)
os<<", ";
first=false;
os<<"negative queries: ";
else if(i->first=='Q')
os<<"queries: ";
- else if(i->first=='p')
- os<<"packets: ";
else
os<<"unknown: ";
os<<i->second;
}
+ os<<"packets: "<<packetEntries;
return os.str();
}
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
-#include "packetcache.hh"
+#include "auth-caches.hh"
#include "utility.hh"
#include <errno.h>
#include "communicator.hh"
// do this via the FindNS class, d_fns
for(auto& di : cmdomains) {
- extern PacketCache PC;
- PC.purgeExact(di.zone);
+ purgeAuthCachesExact(di.zone);
queueNotifyDomain(di, B);
di.backend->setNotified(di.id, di.serial);
}
+++ /dev/null
-/*
- * This file is part of PowerDNS or dnsdist.
- * Copyright -- PowerDNS.COM B.V. and its contributors
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of version 2 of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * In addition, for the avoidance of any doubt, permission is granted to
- * link this program with OpenSSL and to (re)distribute the binaries
- * produced as the result of such linking.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
- */
-#ifdef HAVE_CONFIG_H
-#include "config.h"
-#endif
-#include "utility.hh"
-#include "packetcache.hh"
-#include "logger.hh"
-#include "arguments.hh"
-#include "statbag.hh"
-#include <map>
-#include <boost/algorithm/string.hpp>
-
-const unsigned int PacketCache::s_mincleaninterval, PacketCache::s_maxcleaninterval;
-
-extern StatBag S;
-
-PacketCache::PacketCache()
-{
- d_ops=0;
- d_maps.resize(1024);
- for(auto& mc : d_maps) {
- pthread_rwlock_init(&mc.d_mut, 0);
- }
-
- d_ttl=-1;
-
- d_lastclean=time(0);
- d_cleanskipped=false;
- d_nextclean=d_cleaninterval=4096;
-
- S.declare("packetcache-hit");
- S.declare("packetcache-miss");
- S.declare("packetcache-size");
-
- d_statnumhit=S.getPointer("packetcache-hit");
- d_statnummiss=S.getPointer("packetcache-miss");
- d_statnumentries=S.getPointer("packetcache-size");
-}
-
-PacketCache::~PacketCache()
-{
- try {
- // WriteLock l(&d_mut);
- vector<WriteLock*> locks;
- for(auto& mc : d_maps) {
- locks.push_back(new WriteLock(&mc.d_mut));
- }
- for(auto wl : locks) {
- delete wl;
- }
- }
- catch(...) {
- }
-}
-
-
-
-int PacketCache::get(DNSPacket *p, DNSPacket *cached)
-{
- extern StatBag S;
-
- if(d_ttl<0)
- getTTLS();
-
- cleanupIfNeeded();
-
- if(!d_ttl) {
- (*d_statnummiss)++;
- return 0;
- }
-
- if(ntohs(p->d.qdcount)!=1) // we get confused by packets with more than one question
- return 0;
-
- string value;
- bool haveSomething;
- {
- auto& mc=getMap(p->qdomain);
- TryReadLock l(&mc.d_mut); // take a readlock here
- if(!l.gotIt()) {
- S.inc("deferred-cache-lookup");
- return 0;
- }
-
- uint16_t maxReplyLen = p->d_tcp ? 0xffff : p->getMaxReplyLen();
- haveSomething=getEntryLocked(p->qdomain, p->qtype, PacketCache::PACKETCACHE, value, -1, maxReplyLen, p->d_dnssecOk, p->hasEDNS());
- }
- if(haveSomething) {
- (*d_statnumhit)++;
- if(cached->noparse(value.c_str(), value.size()) < 0)
- return 0;
- cached->spoofQuestion(p); // for correct case
- cached->qdomain=p->qdomain;
- cached->qtype=p->qtype;
- return 1;
- }
-
- // cerr<<"Packet cache miss for '"<<p->qdomain<<"'"<<endl;
- (*d_statnummiss)++;
- return 0; // bummer
-}
-
-void PacketCache::getTTLS()
-{
- d_ttl=::arg().asNum("cache-ttl");
-}
-
-
-void PacketCache::insert(DNSPacket *q, DNSPacket *r, unsigned int maxttl)
-{
- if(d_ttl < 0)
- getTTLS();
-
- if(ntohs(q->d.qdcount)!=1) {
- return; // do not try to cache packets with multiple questions
- }
-
- if(q->qclass != QClass::IN) // we only cache the INternet
- return;
-
- uint16_t maxReplyLen = q->d_tcp ? 0xffff : q->getMaxReplyLen();
- unsigned int ourttl = d_ttl;
- if(maxttl<ourttl)
- ourttl=maxttl;
- insert(q->qdomain, q->qtype, PacketCache::PACKETCACHE, r->getString(), ourttl, -1,
- maxReplyLen, q->d_dnssecOk, q->hasEDNS());
-}
-
-// universal key appears to be: qname, qtype, kind (packet, query cache), optionally zoneid
-void PacketCache::insert(const DNSName &qname, const QType& qtype, CacheEntryType cet, const string& value, unsigned int ttl, int zoneID,
- unsigned int maxReplyLen, bool dnssecOk, bool EDNS)
-{
- cleanupIfNeeded();
-
- if(!ttl)
- return;
-
- //cerr<<"Inserting qname '"<<qname<<"', cet: "<<(int)cet<<", qtype: "<<qtype.getName()<<", ttl: "<<ttl<<", maxreplylen: "<<maxReplyLen<<", hasEDNS: "<<EDNS<<endl;
- CacheEntry val;
- val.created=time(0);
- val.ttd=val.created+ttl;
- val.qname=qname;
- val.qtype=qtype.getCode();
- val.value=value;
- val.ctype=cet;
- val.maxReplyLen = maxReplyLen;
- val.dnssecOk = dnssecOk;
- val.zoneID = zoneID;
- val.hasEDNS = EDNS;
-
- auto& mc = getMap(val.qname);
-
- TryWriteLock l(&mc.d_mut);
- if(l.gotIt()) {
- bool success;
- cmap_t::iterator place;
- tie(place, success)=mc.d_map.insert(val);
-
- if(!success)
- mc.d_map.replace(place, val);
- else
- (*d_statnumentries)++;
- }
- else
- S.inc("deferred-cache-inserts");
-}
-
-void PacketCache::insert(const DNSName &qname, const QType& qtype, CacheEntryType cet, const vector<DNSZoneRecord>& value, unsigned int ttl, int zoneID)
-{
- cleanupIfNeeded();
-
- if(!ttl)
- return;
-
- //cerr<<"Inserting qname '"<<qname<<"', cet: "<<(int)cet<<", qtype: "<<qtype.getName()<<", ttl: "<<ttl<<", maxreplylen: "<<maxReplyLen<<", hasEDNS: "<<EDNS<<endl;
- CacheEntry val;
- val.created=time(0);
- val.ttd=val.created+ttl;
- val.qname=qname;
- val.qtype=qtype.getCode();
- val.drs=value;
- val.ctype=cet;
- val.maxReplyLen = 0;
- val.dnssecOk = false;
- val.zoneID = zoneID;
- val.hasEDNS = false;
-
- auto& mc = getMap(val.qname);
-
- TryWriteLock l(&mc.d_mut);
- if(l.gotIt()) {
- bool success;
- cmap_t::iterator place;
- tie(place, success)=mc.d_map.insert(val);
-
- if(!success)
- mc.d_map.replace(place, val);
- else
- (*d_statnumentries)++;
- }
- else
- S.inc("deferred-cache-inserts");
-}
-
-
-/* clears the entire packetcache. */
-int PacketCache::purge()
-{
- int delcount=0;
- for(auto& mc : d_maps) {
- WriteLock l(&mc.d_mut);
- delcount+=mc.d_map.size();
- mc.d_map.clear();
- }
- d_statnumentries->store(0);
- return delcount;
-}
-
-int PacketCache::purgeExact(const DNSName& qname)
-{
- int delcount=0;
- auto& mc = getMap(qname);
-
- WriteLock l(&mc.d_mut);
- auto range = mc.d_map.equal_range(tie(qname));
- if(range.first != range.second) {
- delcount+=distance(range.first, range.second);
- mc.d_map.erase(range.first, range.second);
- }
- *d_statnumentries-=delcount;
- return delcount;
-}
-
-/* purges entries from the packetcache. If match ends on a $, it is treated as a suffix */
-int PacketCache::purge(const string &match)
-{
- if(ends_with(match, "$")) {
- int delcount=0;
- string prefix(match);
- prefix.resize(prefix.size()-1);
- DNSName dprefix(prefix);
- for(auto& mc : d_maps) {
- WriteLock l(&mc.d_mut);
- cmap_t::const_iterator iter = mc.d_map.lower_bound(tie(dprefix));
- auto start=iter;
-
- for(; iter != mc.d_map.end(); ++iter) {
- if(!iter->qname.isPartOf(dprefix)) {
- break;
- }
- delcount++;
- }
- mc.d_map.erase(start, iter);
- }
- *d_statnumentries-=delcount;
- return delcount;
- }
- else {
- return purgeExact(DNSName(match));
- }
-}
-// called from ueberbackend
-bool PacketCache::getEntry(const DNSName &qname, const QType& qtype, CacheEntryType cet, vector<DNSZoneRecord>& value, int zoneID)
-{
- if(d_ttl<0)
- getTTLS();
-
- cleanupIfNeeded();
-
- auto& mc=getMap(qname);
-
- TryReadLock l(&mc.d_mut); // take a readlock here
- if(!l.gotIt()) {
- S.inc( "deferred-cache-lookup");
- return false;
- }
-
- return getEntryLocked(qname, qtype, cet, value, zoneID);
-}
-
-
-bool PacketCache::getEntryLocked(const DNSName &qname, const QType& qtype, CacheEntryType cet, string& value, int zoneID,
- unsigned int maxReplyLen, bool dnssecOK, bool hasEDNS)
-{
- uint16_t qt = qtype.getCode();
- //cerr<<"Lookup for maxReplyLen: "<<maxReplyLen<<endl;
- auto& mc=getMap(qname);
- // cmap_t::const_iterator i=mc.d_map.find(tie(qname, qt, cet, zoneID, maxReplyLen, dnssecOK, hasEDNS));
-
- auto& idx = boost::multi_index::get<UnorderedNameTag>(mc.d_map);
- auto range=idx.equal_range(tie(qname, qt, cet, zoneID));
-
- if(range.first == range.second)
- return false;
- time_t now=time(0);
- for(auto iter = range.first ; iter != range.second; ++iter) {
- if(maxReplyLen == iter->maxReplyLen && dnssecOK == iter->dnssecOk && hasEDNS == iter->hasEDNS ) {
- if(iter->ttd > now) {
- value = iter->value;
- return true;
- }
- }
- }
-
- return false;
-}
-
-bool PacketCache::getEntryLocked(const DNSName &qname, const QType& qtype, CacheEntryType cet, vector<DNSZoneRecord>& value, int zoneID)
-{
- uint16_t qt = qtype.getCode();
- //cerr<<"Lookup for maxReplyLen: "<<maxReplyLen<<endl;
- auto& mc=getMap(qname);
- auto& idx = boost::multi_index::get<UnorderedNameTag>(mc.d_map);
- auto i=idx.find(tie(qname, qt, cet, zoneID));
- if(i==idx.end())
- return false;
-
- time_t now=time(0);
- if(i->ttd > now) {
- value = i->drs;
- return true;
- }
- return false;
-}
-
-
-map<char,int> PacketCache::getCounts()
-{
- int packets=0, queryCacheEntries=0, negQueryCacheEntries=0;
-
- for(auto& mc : d_maps) {
- ReadLock l(&mc.d_mut);
-
- for(cmap_t::const_iterator iter = mc.d_map.begin() ; iter != mc.d_map.end(); ++iter) {
- if(iter->ctype == PACKETCACHE)
- packets++;
- else if(iter->ctype == QUERYCACHE) {
- if(iter->value.empty())
- negQueryCacheEntries++;
- else
- queryCacheEntries++;
- }
- }
- }
- map<char,int> ret;
-
- ret['!']=negQueryCacheEntries;
- ret['Q']=queryCacheEntries;
- ret['p']=packets;
- return ret;
-}
-
-
-void PacketCache::cleanup()
-{
- unsigned int maxCached = ::arg().asNum("max-cache-entries");
- unsigned long cacheSize = *d_statnumentries;
-
- // two modes - if toTrim is 0, just look through 10% of the cache and nuke everything that is expired
- // otherwise, scan first 5*toTrim records, and stop once we've nuked enough
- unsigned int toTrim = 0, lookAt = 0;
- if(maxCached && cacheSize > maxCached) {
- toTrim = cacheSize - maxCached;
- lookAt = 5 * toTrim;
- } else {
- lookAt = cacheSize / 10;
- }
-
- DLOG(L<<"Starting cache clean, cacheSize: "<<cacheSize<<", lookAt: "<<lookAt<<", toTrim: "<<toTrim<<endl);
-
- time_t now = time(0);
- unsigned int totErased = 0;
- for(auto& mc : d_maps) {
- WriteLock wl(&mc.d_mut);
- auto& sidx = boost::multi_index::get<SequenceTag>(mc.d_map);
- unsigned int erased = 0, lookedAt = 0;
- for(auto i = sidx.begin(); i != sidx.end(); lookedAt++) {
- if(i->ttd < now) {
- i = sidx.erase(i);
- erased++;
- } else {
- ++i;
- }
-
- if(toTrim && erased > toTrim / d_maps.size())
- break;
-
- if(lookedAt > lookAt / d_maps.size())
- break;
- }
- totErased += erased;
- }
- *d_statnumentries -= totErased;
-
- DLOG(L<<"Done with cache clean, cacheSize: "<<*d_statnumentries<<", totErased"<<totErased<<endl);
-}
-
-/* the logic:
- after d_nextclean operations, we clean. We also adjust the cleaninterval
- a bit so we slowly move it to a value where we clean roughly every 30 seconds.
-
- If d_nextclean has reached its maximum value, we also test if we were called
- within 30 seconds, and if so, we skip cleaning. This means that under high load,
- we will not clean more often than every 30 seconds anyhow.
-*/
-
-void PacketCache::cleanupIfNeeded()
-{
- if (d_ops++ == d_nextclean) {
- int timediff = max((int)(time(0) - d_lastclean), 1);
-
- DLOG(L<<"cleaninterval: "<<d_cleaninterval<<", timediff: "<<timediff<<endl);
-
- if (d_cleaninterval == s_maxcleaninterval && timediff < 30) {
- d_cleanskipped = true;
- d_nextclean += d_cleaninterval;
-
- DLOG(L<<"cleaning skipped, timediff: "<<timediff<<endl);
-
- return;
- }
-
- if(!d_cleanskipped) {
- d_cleaninterval=(int)(0.6*d_cleaninterval)+(0.4*d_cleaninterval*(30.0/timediff));
- d_cleaninterval=std::max(d_cleaninterval, s_mincleaninterval);
- d_cleaninterval=std::min(d_cleaninterval, s_maxcleaninterval);
-
- DLOG(L<<"new cleaninterval: "<<d_cleaninterval<<endl);
- } else {
- d_cleanskipped = false;
- }
-
- d_nextclean += d_cleaninterval;
- d_lastclean=time(0);
- cleanup();
- }
-}
#ifndef PACKETCACHE_HH
#define PACKETCACHE_HH
-#include <string>
-#include <utility>
-#include <map>
-#include <map>
-#include "dns.hh"
-#include <boost/version.hpp>
-#include "namespaces.hh"
-using namespace ::boost::multi_index;
-
-#include "namespaces.hh"
-#include <boost/multi_index/hashed_index.hpp>
-#include "dnspacket.hh"
-#include "lock.hh"
-#include "statbag.hh"
-
-/** This class performs 'whole packet caching'. Feed it a question packet and it will
- try to find an answer. If you have an answer, insert it to have it cached for later use.
- Take care not to replace existing cache entries. While this works, it is wasteful. Only
- insert packets that where not found by get()
-
- Locking!
-
- The cache itself is protected by a read/write lock. Because deleting is a two step process, which
- first marks and then sweeps, a second lock is present to prevent simultaneous inserts and deletes.
-*/
+#include "ednsoptions.hh"
+#include "misc.hh"
+#include "iputils.hh"
class PacketCache : public boost::noncopyable
{
-public:
- PacketCache();
- ~PacketCache();
- enum CacheEntryType { PACKETCACHE, QUERYCACHE};
-
- void insert(DNSPacket *q, DNSPacket *r, unsigned int maxttl=UINT_MAX); //!< We copy the contents of *p into our cache. Do not needlessly call this to insert questions already in the cache as it wastes resources
-
- void insert(const DNSName &qname, const QType& qtype, CacheEntryType cet, const string& value, unsigned int ttl, int zoneID=-1,
- unsigned int maxReplyLen=512, bool dnssecOk=false, bool EDNS=false);
-
- void insert(const DNSName &qname, const QType& qtype, CacheEntryType cet, const vector<DNSZoneRecord>& content, unsigned int ttl, int zoneID=-1);
-
- int get(DNSPacket *p, DNSPacket *q); //!< We return a dynamically allocated copy out of our cache. You need to delete it. You also need to spoof in the right ID with the DNSPacket.spoofID() method.
- bool getEntry(const DNSName &qname, const QType& qtype, CacheEntryType cet, string& entry, int zoneID=-1,
- unsigned int maxReplyLen=512, bool dnssecOk=false, bool hasEDNS=false);
- bool getEntry(const DNSName &qname, const QType& qtype, CacheEntryType cet, vector<DNSZoneRecord>& entry, int zoneID=-1);
-
-
- int size() { return *d_statnumentries; } //!< number of entries in the cache
- void cleanupIfNeeded();
- void cleanup(); //!< force the cache to preen itself from expired packets
- int purge();
- int purge(const std::string& match); // could be $ terminated. Is not a dnsname!
- int purgeExact(const DNSName& qname); // no wildcard matching here
-
- map<char,int> getCounts();
-private:
- bool getEntryLocked(const DNSName &content, const QType& qtype, CacheEntryType cet, string& entry, int zoneID=-1,
- unsigned int maxReplyLen=512, bool dnssecOk=false, bool hasEDNS=false);
- bool getEntryLocked(const DNSName &content, const QType& qtype, CacheEntryType cet, vector<DNSZoneRecord>& entry, int zoneID=-1);
-
-
- struct CacheEntry
+protected:
+ static uint32_t canHashPacket(const std::string& packet, bool skipECS=true)
{
- CacheEntry() { qtype = ctype = 0; zoneID = -1; dnssecOk=false; hasEDNS=false; created=0; ttd=0; maxReplyLen=512;}
-
- DNSName qname;
- string value;
- vector<DNSZoneRecord> drs;
- time_t created;
- time_t ttd;
-
- uint16_t qtype;
- uint16_t ctype;
- int zoneID;
- unsigned int maxReplyLen;
-
- bool dnssecOk;
- bool hasEDNS;
- };
-
- void getTTLS();
-
- struct UnorderedNameTag{};
- struct SequenceTag{};
- typedef multi_index_container<
- CacheEntry,
- indexed_by <
- ordered_unique<
- composite_key<
- CacheEntry,
- member<CacheEntry,DNSName,&CacheEntry::qname>,
- member<CacheEntry,uint16_t,&CacheEntry::qtype>,
- member<CacheEntry,uint16_t, &CacheEntry::ctype>,
- member<CacheEntry,int, &CacheEntry::zoneID>,
- member<CacheEntry,unsigned int, &CacheEntry::maxReplyLen>,
- member<CacheEntry,bool, &CacheEntry::dnssecOk>,
- member<CacheEntry,bool, &CacheEntry::hasEDNS>
- >,
- composite_key_compare<CanonDNSNameCompare, std::less<uint16_t>, std::less<uint16_t>, std::less<int>,
- std::less<unsigned int>, std::less<bool>, std::less<bool> >
- >,
- hashed_non_unique<tag<UnorderedNameTag>, composite_key<CacheEntry,
- member<CacheEntry,DNSName,&CacheEntry::qname>,
- member<CacheEntry,uint16_t,&CacheEntry::qtype>,
- member<CacheEntry,uint16_t, &CacheEntry::ctype>,
- member<CacheEntry,int, &CacheEntry::zoneID> > > ,
- sequenced<tag<SequenceTag>>
- >
- > cmap_t;
-
-
- struct MapCombo
- {
- pthread_rwlock_t d_mut;
- cmap_t d_map;
- };
-
- vector<MapCombo> d_maps;
- MapCombo& getMap(const DNSName& qname)
- {
- return d_maps[qname.hash() % d_maps.size()];
+ uint32_t ret = 0;
+ ret=burtle((const unsigned char*)packet.c_str() + 2, 10, ret); // rest of dnsheader, skip id
+ size_t packetSize = packet.size();
+ size_t pos = 12;
+ const char* end = packet.c_str() + packetSize;
+ const char* p = packet.c_str() + pos;
+
+ for(; p < end && *p; ++p) { // XXX if you embed a 0 in your qname we'll stop lowercasing there
+ const unsigned char l = dns_tolower(*p); // label lengths can safely be lower cased
+ ret=burtle(&l, 1, ret);
+ } // XXX the embedded 0 in the qname will break the subnet stripping
+
+ struct dnsheader* dh = (struct dnsheader*)packet.c_str();
+ const char* skipBegin = p;
+ const char* skipEnd = p;
+ /* we need at least 1 (final empty label) + 2 (QTYPE) + 2 (QCLASS)
+ + OPT root label (1), type (2), class (2) and ttl (4)
+ + the OPT RR rdlen (2)
+ = 16
+ */
+ if(skipECS && ntohs(dh->arcount)==1 && (pos+16) < packetSize) {
+ char* optionBegin = nullptr;
+ size_t optionLen = 0;
+ /* skip the final empty label (1), the qtype (2), qclass (2) */
+ /* root label (1), type (2), class (2) and ttl (4) */
+ int res = getEDNSOption((char*) p + 14, end - (p + 14), EDNSOptionCode::ECS, &optionBegin, &optionLen);
+ if (res == 0) {
+ skipBegin = optionBegin;
+ skipEnd = optionBegin + optionLen;
+ }
+ }
+ if (skipBegin > p) {
+ // cerr << "Hashing from " << (p-packet.c_str()) << " for " << skipBegin-p << "bytes, end is at "<< end-packet.c_str() << endl;
+ ret = burtle((const unsigned char*)p, skipBegin-p, ret);
+ }
+ if (skipEnd < end) {
+ // cerr << "Hashing from " << (skipEnd-packet.c_str()) << " for " << end-skipEnd << "bytes, end is at " << end-packet.c_str() << endl;
+ ret = burtle((const unsigned char*) skipEnd, end-skipEnd, ret);
+ }
+
+ return ret;
}
-
- AtomicCounter d_ops;
- time_t d_lastclean; // doesn't need to be atomic
- unsigned long d_nextclean;
- unsigned int d_cleaninterval;
- bool d_cleanskipped;
- AtomicCounter *d_statnumhit;
- AtomicCounter *d_statnummiss;
- AtomicCounter *d_statnumentries;
-
- int d_ttl;
-
- static const unsigned int s_mincleaninterval=1000, s_maxcleaninterval=300000;
};
-
-
#endif /* PACKETCACHE_HH */
for(const auto& rr: r->getRRS()) {
if(rr.scopeMask) {
- noCache=1;
+ noCache=true;
break;
}
}
addRRSigs(d_dk, B, authSet, r->getRRS());
r->wrapup(); // needed for inserting in cache
- if(!noCache)
+ if(!noCache && p->couldBeCached())
PC.insert(p, r, r->getMinTTL()); // in the packet cache
}
catch(DBException &e) {
#include "dnsbackend.hh"
#include "ueberbackend.hh"
#include "arguments.hh"
-#include "packetcache.hh"
+#include "auth-packetcache.hh"
+#include "auth-querycache.hh"
#include "zoneparser-tng.hh"
#include "signingpipe.hh"
#include "dns_random.hh"
#endif
StatBag S;
-PacketCache PC;
+AuthPacketCache PC;
+AuthQueryCache QC;
namespace po = boost::program_options;
po::variables_map g_vm;
//cerr<<"Backend: "<<::arg()["launch"]<<", '" << ::arg()["gmysql-dbname"] <<"'" <<endl;
S.declare("qsize-q","Number of questions waiting for database attention");
-
- S.declare("deferred-cache-inserts","Amount of cache inserts that were deferred because of maintenance");
- S.declare("deferred-cache-lookup","Amount of cache lookups that were deferred because of maintenance");
- S.declare("query-cache-hit","Number of hits on the query cache");
- S.declare("query-cache-miss","Number of misses on the query cache");
::arg().set("max-cache-entries", "Maximum number of cache entries")="1000000";
::arg().set("cache-ttl","Seconds to store packets in the PacketCache")="20";
::arg().set("negquery-cache-ttl","Seconds to store negative query results in the QueryCache")="60";
return 0;
}
-loadMainConfig(g_vm["config-dir"].as<string>());
+ loadMainConfig(g_vm["config-dir"].as<string>());
#ifdef HAVE_LIBSODIUM
if (sodium_init() == -1) {
return qname==rname && rtype == qtype && rclass == qclass;
}
-uint32_t RecursorPacketCache::canHashPacket(const std::string& origPacket)
-{
- // return 42; // should still work if you do this!
- uint32_t ret=0;
- ret=burtle((const unsigned char*)origPacket.c_str() + 2, 10, ret); // rest of dnsheader, skip id
- const char* end = origPacket.c_str() + origPacket.size();
- const char* p = origPacket.c_str() + 12;
-
- for(; p < end && *p; ++p) { // XXX if you embed a 0 in your qname we'll stop lowercasing there
- const unsigned char l = dns_tolower(*p); // label lengths can safely be lower cased
- ret=burtle(&l, 1, ret);
- } // XXX the embedded 0 in the qname will break the subnet stripping
-
- struct dnsheader* dh = (struct dnsheader*)origPacket.c_str();
- const char* skipBegin = p;
- const char* skipEnd = p;
- /* we need at least 1 (final empty label) + 2 (QTYPE) + 2 (QCLASS)
- + OPT root label (1), type (2), class (2) and ttl (4)
- + the OPT RR rdlen (2)
- = 16
- */
- if(ntohs(dh->arcount)==1 && (p+16) < end) {
- char* optionBegin = nullptr;
- size_t optionLen = 0;
- /* skip the final empty label (1), the qtype (2), qclass (2) */
- /* root label (1), type (2), class (2) and ttl (4) */
- int res = getEDNSOption((char*) p + 14, end - (p + 14), EDNSOptionCode::ECS, &optionBegin, &optionLen);
- if (res == 0) {
- skipBegin = optionBegin;
- skipEnd = optionBegin + optionLen;
- }
- }
- if (skipBegin > p) {
- //cout << "Hashing from " << (p-origPacket.c_str()) << " for " << skipBegin-p << "bytes, end is at "<< end-origPacket.c_str() << endl;
- ret = burtle((const unsigned char*)p, skipBegin-p, ret);
- }
- if (skipEnd < end) {
- //cout << "Hashing from " << (skipEnd-origPacket.c_str()) << " for " << end-skipEnd << "bytes, end is at " << end-origPacket.c_str() << endl;
- ret = burtle((const unsigned char*) skipEnd, end-skipEnd, ret);
- }
- return ret;
-}
-
bool RecursorPacketCache::checkResponseMatches(std::pair<packetCache_t::index<HashTag>::type::iterator, packetCache_t::index<HashTag>::type::iterator> range, const std::string& queryPacket, const DNSName& qname, uint16_t qtype, uint16_t qclass, time_t now, std::string* responsePacket, uint32_t* age, RecProtoBufMessage* protobufMessage)
{
for(auto iter = range.first ; iter != range.second ; ++ iter) {
bool RecursorPacketCache::getResponsePacket(unsigned int tag, const std::string& queryPacket, const DNSName& qname, uint16_t qtype, uint16_t qclass, time_t now,
std::string* responsePacket, uint32_t* age, uint32_t* qhash, RecProtoBufMessage* protobufMessage)
{
- *qhash = canHashPacket(queryPacket);
+ *qhash = canHashPacket(queryPacket, true);
const auto& idx = d_packetCache.get<HashTag>();
auto range = idx.equal_range(tie(tag,*qhash));
bool RecursorPacketCache::getResponsePacket(unsigned int tag, const std::string& queryPacket, time_t now,
std::string* responsePacket, uint32_t* age, uint32_t* qhash, RecProtoBufMessage* protobufMessage)
{
- *qhash = canHashPacket(queryPacket);
+ *qhash = canHashPacket(queryPacket, true);
const auto& idx = d_packetCache.get<HashTag>();
auto range = idx.equal_range(tie(tag,*qhash));
auto iter = range.first;
for( ; iter != range.second ; ++iter) {
- if(iter->d_type != qtype || iter->d_class != qclass)
- continue;
- // this only happens on insert which is relatively rare and does not need to be super fast
- DNSName respname(iter->d_packet.c_str(), iter->d_packet.length(), sizeof(dnsheader), false, 0, 0, 0);
- if(qname != respname)
+ if(iter->d_type != qtype || iter->d_class != qclass || iter->d_name != qname)
continue;
+
moveCacheItemToBack(d_packetCache, iter);
iter->d_packet = responsePacket;
iter->d_ttd = now + ttl;
#include <boost/tuple/tuple_comparison.hpp>
#include <boost/multi_index/sequenced_index.hpp>
+#include "packetcache.hh"
+
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
you can use a query as a key too. But query and answer must compare as identical!
This precludes doing anything smart with EDNS directly from the packet */
-class RecursorPacketCache
+class RecursorPacketCache: public PacketCache
{
public:
RecursorPacketCache();
return d_ttd;
}
};
- uint32_t canHashPacket(const std::string& origPacket);
+
typedef multi_index_container<
Entry,
indexed_by <
namespaces.hh \
nsecrecords.cc \
opensslsigners.cc opensslsigners.hh \
+ packetcache.hh \
pdns_recursor.cc \
pdnsexception.hh \
protobuf.cc protobuf.hh \
--- /dev/null
+../packetcache.hh
\ No newline at end of file
#include "packethandler.hh"
#include "qtype.hh"
#include "dnspacket.hh"
-#include "packetcache.hh"
+#include "auth-caches.hh"
+#include "statbag.hh"
#include "dnsseckeeper.hh"
#include "base64.hh"
#include "base32.hh"
#include "dns_random.hh"
#include "backends/gsql/ssql.hh"
-extern PacketCache PC;
extern StatBag S;
pthread_mutex_t PacketHandler::s_rfc2136lock=PTHREAD_MUTEX_INITIALIZER;
// Purge the records!
string zone(di.zone.toString());
zone.append("$");
- PC.purge(zone);
+ purgeAuthCaches(zone);
L<<Logger::Info<<msgPrefix<<"Update completed, "<<changedRecords<<" changed records committed."<<endl;
} else {
#include "config.h"
#endif
#include <boost/algorithm/string.hpp>
-#include "packetcache.hh"
+#include "auth-packetcache.hh"
#include "utility.hh"
#include "dnssecinfra.hh"
#include "dnsseckeeper.hh"
#include "namespaces.hh"
#include "signingpipe.hh"
#include "stubresolver.hh"
-extern PacketCache PC;
+extern AuthPacketCache PC;
extern StatBag S;
/**
#include "iputils.hh"
#include "nameserver.hh"
#include "statbag.hh"
-#include "packetcache.hh"
+#include "auth-packetcache.hh"
+#include "auth-querycache.hh"
#include "arguments.hh"
#include <utility>
extern StatBag S;
BOOST_AUTO_TEST_SUITE(packetcache_cc)
-BOOST_AUTO_TEST_CASE(test_PacketCacheSimple) {
- PacketCache PC;
+BOOST_AUTO_TEST_CASE(test_AuthQueryCacheSimple) {
+ AuthQueryCache QC;
+ QC.setMaxEntries(1000000);
- ::arg().set("max-cache-entries", "Maximum number of cache entries")="1000000";
- ::arg().set("cache-ttl","Seconds to store packets in the PacketCache")="20";
- ::arg().set("negquery-cache-ttl","Seconds to store negative query results in the QueryCache")="60";
- ::arg().set("query-cache-ttl","Seconds to store query results in the QueryCache")="20";
+ vector<DNSZoneRecord> records;
- S.declare("deferred-cache-inserts","Amount of cache inserts that were deferred because of maintenance");
- S.declare("deferred-cache-lookup","Amount of cache lookups that were deferred because of maintenance");
+ BOOST_CHECK_EQUAL(QC.size(), 0);
+ QC.insert(DNSName("hello"), QType(QType::A), records, 3600, 1);
+ BOOST_CHECK_EQUAL(QC.size(), 1);
+ BOOST_CHECK_EQUAL(QC.purge(), 1);
+ BOOST_CHECK_EQUAL(QC.size(), 0);
-
- BOOST_CHECK_EQUAL(PC.size(), 0);
- PC.insert(DNSName("hello"), QType(QType::A), PacketCache::QUERYCACHE, "something", 3600, 1);
- BOOST_CHECK_EQUAL(PC.size(), 1);
- PC.purge();
- BOOST_CHECK_EQUAL(PC.size(), 0);
-
- int counter=0;
+ uint64_t counter=0;
try {
for(counter = 0; counter < 100000; ++counter) {
DNSName a=DNSName("hello ")+DNSName(std::to_string(counter));
BOOST_CHECK_EQUAL(DNSName(a.toString()), a);
- PC.insert(a, QType(QType::A), PacketCache::QUERYCACHE, "something", 3600, 1);
- if(!PC.purge(a.toString()))
- BOOST_FAIL("Could not remove entry we just added to packet cache!");
- PC.insert(a, QType(QType::A), PacketCache::QUERYCACHE, "something", 3600, 1);
+ QC.insert(a, QType(QType::A), records, 3600, 1);
+ if(!QC.purge(a.toString()))
+ BOOST_FAIL("Could not remove entry we just added to the query cache!");
+ QC.insert(a, QType(QType::A), records, 3600, 1);
}
- BOOST_CHECK_EQUAL(PC.size(), counter);
-
- int delcounter=0;
+ BOOST_CHECK_EQUAL(QC.size(), counter);
+
+ uint64_t delcounter=0;
for(delcounter=0; delcounter < counter/100; ++delcounter) {
DNSName a=DNSName("hello ")+DNSName(std::to_string(delcounter));
- PC.purge(a.toString());
+ BOOST_CHECK_EQUAL(QC.purge(a.toString()), 1);
}
-
- BOOST_CHECK_EQUAL(PC.size(), counter-delcounter);
-
- int matches=0;
+
+ BOOST_CHECK_EQUAL(QC.size(), counter-delcounter);
+
+ uint64_t matches=0;
vector<DNSZoneRecord> entry;
- int expected=counter-delcounter;
+ int64_t expected=counter-delcounter;
for(; delcounter < counter; ++delcounter) {
- if(PC.getEntry(DNSName("hello ")+DNSName(std::to_string(delcounter)), QType(QType::A), PacketCache::QUERYCACHE, entry, 1)) {
+ if(QC.getEntry(DNSName("hello ")+DNSName(std::to_string(delcounter)), QType(QType::A), entry, 1)) {
matches++;
}
}
BOOST_CHECK_EQUAL(matches, expected);
- // BOOST_CHECK_EQUAL(entry, "something");
+ BOOST_CHECK_EQUAL(entry.size(), records.size());
}
catch(PDNSException& e) {
cerr<<"Had error: "<<e.reason<<endl;
}
-static PacketCache* g_PC;
+static AuthQueryCache* g_QC;
+static AtomicCounter g_QCmissing;
-static void *threadMangler(void* a)
+static void *threadQCMangler(void* a)
try
{
+ vector<DNSZoneRecord> records;
unsigned int offset=(unsigned int)(unsigned long)a;
for(unsigned int counter=0; counter < 100000; ++counter)
- g_PC->insert(DNSName("hello ")+DNSName(std::to_string(counter+offset)), QType(QType::A), PacketCache::QUERYCACHE, "something", 3600, 1);
+ g_QC->insert(DNSName("hello ")+DNSName(std::to_string(counter+offset)), QType(QType::A), records, 3600, 1);
return 0;
}
catch(PDNSException& e) {
throw;
}
-AtomicCounter g_missing;
-
-static void *threadReader(void* a)
+static void *threadQCReader(void* a)
try
{
unsigned int offset=(unsigned int)(unsigned long)a;
vector<DNSZoneRecord> entry;
for(unsigned int counter=0; counter < 100000; ++counter)
- if(!g_PC->getEntry(DNSName("hello ")+DNSName(std::to_string(counter+offset)), QType(QType::A), PacketCache::QUERYCACHE, entry, 1)) {
- g_missing++;
+ if(!g_QC->getEntry(DNSName("hello ")+DNSName(std::to_string(counter+offset)), QType(QType::A), entry, 1)) {
+ g_QCmissing++;
}
return 0;
}
catch(PDNSException& e) {
- cerr<<"Had error in threadReader: "<<e.reason<<endl;
+ cerr<<"Had error in threadQCReader: "<<e.reason<<endl;
throw;
}
+BOOST_AUTO_TEST_CASE(test_QueryCacheThreaded) {
+ try {
+ AuthQueryCache QC;
+ QC.setMaxEntries(1000000);
+ g_QC=&QC;
+ pthread_t tid[4];
+ for(int i=0; i < 4; ++i)
+ pthread_create(&tid[i], 0, threadQCMangler, (void*)(i*1000000UL));
+ void* res;
+ for(int i=0; i < 4 ; ++i)
+ pthread_join(tid[i], &res);
+
+ BOOST_CHECK_EQUAL(QC.size() + S.read("deferred-cache-inserts"), 400000);
+ BOOST_CHECK_SMALL(1.0*S.read("deferred-cache-inserts"), 10000.0);
+
+ for(int i=0; i < 4; ++i)
+ pthread_create(&tid[i], 0, threadQCReader, (void*)(i*1000000UL));
+ for(int i=0; i < 4 ; ++i)
+ pthread_join(tid[i], &res);
+
+ BOOST_CHECK(S.read("deferred-cache-inserts") + S.read("deferred-cache-lookup") >= g_QCmissing);
+ // BOOST_CHECK_EQUAL(S.read("deferred-cache-lookup"), 0); // cache cleaning invalidates this
+ }
+ catch(PDNSException& e) {
+ cerr<<"Had error: "<<e.reason<<endl;
+ throw;
+ }
+
+}
+
+static AuthPacketCache* g_PC;
+static AtomicCounter g_PCmissing;
+
+static void *threadPCMangler(void* a)
+try
+{
+ unsigned int offset=(unsigned int)(unsigned long)a;
+ for(unsigned int counter=0; counter < 100000; ++counter) {
+ vector<uint8_t> pak;
+ DNSName qname = DNSName("hello ")+DNSName(std::to_string(counter+offset));
+
+ DNSPacketWriter pw(pak, qname, QType::A);
+ DNSPacket q(true);
+ q.parse((char*)&pak[0], pak.size());
+
+ pak.clear();
+ DNSPacketWriter pw2(pak, qname, QType::A);
+ pw2.startRecord(qname, QType::A, 16, 1, DNSResourceRecord::ANSWER);
+ pw2.xfrIP(htonl(0x7f000001));
+ pw2.commit();
+
+ DNSPacket r(false);
+ r.parse((char*)&pak[0], pak.size());
+
+ /* this step is necessary to get a valid hash */
+ DNSPacket cached(false);
+ g_PC->get(&q, &cached);
+
+ g_PC->insert(&q, &r, 10);
+ }
+
+ return 0;
+}
+ catch(PDNSException& e) {
+ cerr<<"Had error: "<<e.reason<<endl;
+ throw;
+ }
+
+static void *threadPCReader(void* a)
+try
+{
+ unsigned int offset=(unsigned int)(unsigned long)a;
+ vector<DNSZoneRecord> entry;
+ for(unsigned int counter=0; counter < 100000; ++counter) {
+ vector<uint8_t> pak;
+ DNSName qname = DNSName("hello ")+DNSName(std::to_string(counter+offset));
+
+ DNSPacketWriter pw(pak, qname, QType::A);
+ DNSPacket q(true);
+ q.parse((char*)&pak[0], pak.size());
+ DNSPacket r(false);
+
+ if(!g_PC->get(&q, &r)) {
+ g_PCmissing++;
+ }
+ }
+ return 0;
+}
+catch(PDNSException& e) {
+ cerr<<"Had error in threadPCReader: "<<e.reason<<endl;
+ throw;
+}
BOOST_AUTO_TEST_CASE(test_PacketCacheThreaded) {
try {
- PacketCache PC;
+ AuthPacketCache PC;
+ PC.setMaxEntries(1000000);
+ PC.setTTL(20);
+
g_PC=&PC;
pthread_t tid[4];
- for(int i=0; i < 4; ++i)
- pthread_create(&tid[i], 0, threadMangler, (void*)(i*1000000UL));
+ for(int i=0; i < 4; ++i)
+ pthread_create(&tid[i], 0, threadPCMangler, (void*)(i*1000000UL));
void* res;
for(int i=0; i < 4 ; ++i)
pthread_join(tid[i], &res);
-
- BOOST_CHECK_EQUAL(PC.size() + S.read("deferred-cache-inserts"), 400000);
- BOOST_CHECK_SMALL(1.0*S.read("deferred-cache-inserts"), 10000.0);
- for(int i=0; i < 4; ++i)
- pthread_create(&tid[i], 0, threadReader, (void*)(i*1000000UL));
+ BOOST_CHECK_EQUAL(PC.size() + S.read("deferred-packetcache-inserts"), 400000);
+ BOOST_CHECK_SMALL(1.0*S.read("deferred-packetcache-inserts"), 10000.0);
+
+ for(int i=0; i < 4; ++i)
+ pthread_create(&tid[i], 0, threadPCReader, (void*)(i*1000000UL));
for(int i=0; i < 4 ; ++i)
pthread_join(tid[i], &res);
- BOOST_CHECK(S.read("deferred-cache-inserts") + S.read("deferred-cache-lookup") >= g_missing);
- // BOOST_CHECK_EQUAL(S.read("deferred-cache-lookup"), 0); // cache cleaning invalidates this
+/*
+ cerr<<"Misses: "<<S.read("packetcache-miss")<<endl;
+ cerr<<"Hits: "<<S.read("packetcache-hit")<<endl;
+ cerr<<"Deferred inserts: "<<S.read("deferred-packetcache-inserts")<<endl;
+ cerr<<"Deferred lookups: "<<S.read("deferred-packetcache-lookup")<<endl;
+*/
+ BOOST_CHECK_EQUAL(g_PCmissing + S.read("packetcache-hit"), 400000);
+ BOOST_CHECK_GT(S.read("deferred-packetcache-inserts") + S.read("deferred-packetcache-lookup"), g_PCmissing);
}
catch(PDNSException& e) {
cerr<<"Had error: "<<e.reason<<endl;
throw;
}
-
+
}
bool g_stopCleaning;
try
{
while(!g_stopCleaning) {
- g_PC->cleanup();
+ g_QC->cleanup();
}
return 0;
}
catch(PDNSException& e) {
- cerr<<"Had error in threadReader: "<<e.reason<<endl;
+ cerr<<"Had error in cacheCleaner: "<<e.reason<<endl;
throw;
}
-BOOST_AUTO_TEST_CASE(test_PacketCacheClean) {
+BOOST_AUTO_TEST_CASE(test_QueryCacheClean) {
try {
- PacketCache PC;
+ AuthQueryCache QC;
+ QC.setMaxEntries(10000);
+ vector<DNSZoneRecord> records;
for(unsigned int counter = 0; counter < 1000000; ++counter) {
- PC.insert(DNSName("hello ")+DNSName(std::to_string(counter)), QType(QType::A), PacketCache::QUERYCACHE, "something", 1, 1);
+ QC.insert(DNSName("hello ")+DNSName(std::to_string(counter)), QType(QType::A), records, 1, 1);
}
sleep(1);
-
- g_PC=&PC;
- pthread_t tid[4];
- ::arg().set("max-cache-entries")="10000";
+ g_QC=&QC;
+ pthread_t tid[4];
- pthread_create(&tid[0], 0, threadReader, (void*)(0*1000000UL));
- pthread_create(&tid[1], 0, threadReader, (void*)(1*1000000UL));
- pthread_create(&tid[2], 0, threadReader, (void*)(2*1000000UL));
+ pthread_create(&tid[0], 0, threadQCReader, (void*)(0*1000000UL));
+ pthread_create(&tid[1], 0, threadQCReader, (void*)(1*1000000UL));
+ pthread_create(&tid[2], 0, threadQCReader, (void*)(2*1000000UL));
// pthread_create(&tid[2], 0, threadMangler, (void*)(0*1000000UL));
pthread_create(&tid[3], 0, cacheCleaner, 0);
pthread_join(tid[3], &res);
}
catch(PDNSException& e) {
- cerr<<"Had error in threadReader: "<<e.reason<<endl;
+ cerr<<"Had error in test_QueryCacheClean: "<<e.reason<<endl;
throw;
}
}
-BOOST_AUTO_TEST_CASE(test_PacketCachePacket) {
+BOOST_AUTO_TEST_CASE(test_AuthPacketCache) {
try {
::arg().setSwitch("no-shuffle","Set this to prevent random shuffling of answers - for regression testing")="off";
- PacketCache PC;
+ AuthPacketCache PC;
+ PC.setTTL(20);
+ PC.setMaxEntries(100000);
+
vector<uint8_t> pak;
- vector<pair<uint16_t,string > > opts;
+ DNSPacket q(true), differentIDQ(true), ednsQ(true), ednsVersion42(true), ednsDO(true), ecs1(true), ecs2(true), ecs3(true);
+ DNSPacket r(false), r2(false);
- DNSPacketWriter pw(pak, DNSName("www.powerdns.com"), QType::A);
- DNSPacket q(true), r(false), r2(false);
- q.parse((char*)&pak[0], pak.size());
+ {
+ DNSPacketWriter pw(pak, DNSName("www.powerdns.com"), QType::A);
+ q.parse((char*)&pak[0], pak.size());
- pak.clear();
- DNSPacketWriter pw2(pak, DNSName("www.powerdns.com"), QType::A);
- pw2.startRecord(DNSName("www.powerdns.com"), QType::A, 16, 1, DNSResourceRecord::ANSWER);
- pw2.xfrIP(htonl(0x7f000001));
- pw2.commit();
+ differentIDQ.parse((char*)&pak[0], pak.size());
+ differentIDQ.setID(4242);
- r.parse((char*)&pak[0], pak.size());
+ pw.addOpt(512, 0, 0);
+ pw.commit();
+ ednsQ.parse((char*)&pak[0], pak.size());
+
+ pak.clear();
+ }
+
+ DNSPacketWriter::optvect_t opts;
+ EDNSSubnetOpts ecsOpts;
+ {
+ DNSPacketWriter pw(pak, DNSName("www.powerdns.com"), QType::A);
+ pw.addOpt(512, 0, 0, DNSPacketWriter::optvect_t(), 42);
+ pw.commit();
+ ednsVersion42.parse((char*)&pak[0], pak.size());
+ pak.clear();
+ }
+
+ {
+ DNSPacketWriter pw(pak, DNSName("www.powerdns.com"), QType::A);
+ pw.addOpt(512, 0, EDNSOpts::DNSSECOK);
+ pw.commit();
+ ednsDO.parse((char*)&pak[0], pak.size());
+ pak.clear();
+ }
+
+ {
+ ecsOpts.source = Netmask(ComboAddress("192.0.2.1"), 32);
+ opts.push_back(make_pair(EDNSOptionCode::ECS, makeEDNSSubnetOptsString(ecsOpts)));
+ DNSPacketWriter pw(pak, DNSName("www.powerdns.com"), QType::A);
+ pw.addOpt(512, 0, 0, opts);
+ pw.commit();
+ ecs1.parse((char*)&pak[0], pak.size());
+ pak.clear();
+ opts.clear();
+ }
+
+ {
+ DNSPacketWriter pw(pak, DNSName("www.powerdns.com"), QType::A);
+ ecsOpts.source = Netmask(ComboAddress("192.0.2.2"), 32);
+ opts.push_back(make_pair(EDNSOptionCode::ECS, makeEDNSSubnetOptsString(ecsOpts)));
+ pw.addOpt(512, 0, 0, opts);
+ pw.commit();
+ ecs2.parse((char*)&pak[0], pak.size());
+ pak.clear();
+ opts.clear();
+ }
+
+ {
+ DNSPacketWriter pw(pak, DNSName("www.powerdns.com"), QType::A);
+ ecsOpts.source = Netmask(ComboAddress("192.0.2.3"), 16);
+ opts.push_back(make_pair(EDNSOptionCode::ECS, makeEDNSSubnetOptsString(ecsOpts)));
+ pw.addOpt(512, 0, 0, opts);
+ pw.commit();
+ ecs3.parse((char*)&pak[0], pak.size());
+ pak.clear();
+ opts.clear();
+ }
+
+ {
+ DNSPacketWriter pw(pak, DNSName("www.powerdns.com"), QType::A);
+ pw.startRecord(DNSName("www.powerdns.com"), QType::A, 16, 1, DNSResourceRecord::ANSWER);
+ pw.xfrIP(htonl(0x7f000001));
+ pw.commit();
+
+ r.parse((char*)&pak[0], pak.size());
+ }
+
+ /* this call is required so the correct hash is set into q->d_hash */
+ BOOST_CHECK_EQUAL(PC.get(&q, &r2), false);
PC.insert(&q, &r, 3600);
+ BOOST_CHECK_EQUAL(PC.size(), 1);
+
+ BOOST_CHECK_EQUAL(PC.get(&q, &r2), true);
+ BOOST_CHECK_EQUAL(r2.qdomain, r.qdomain);
+
+ /* different QID, still should match */
+ BOOST_CHECK_EQUAL(PC.get(&differentIDQ, &r2), true);
+ BOOST_CHECK_EQUAL(r2.qdomain, r.qdomain);
+
+ /* with EDNS, should not match */
+ BOOST_CHECK_EQUAL(PC.get(&ednsQ, &r2), false);
+ /* inserting the EDNS-enabled one too */
+ PC.insert(&ednsQ, &r, 3600);
+ BOOST_CHECK_EQUAL(PC.size(), 2);
+
+ /* different EDNS versions, should not match */
+ BOOST_CHECK_EQUAL(PC.get(&ednsVersion42, &r2), false);
+
+ /* EDNS DO set, should not match */
+ BOOST_CHECK_EQUAL(PC.get(&ednsDO, &r2), false);
- BOOST_CHECK_EQUAL(PC.get(&q, &r2), 1);
+ /* EDNS Client Subnet set, should not match
+ since not only we don't skip the actual option, but the
+ total EDNS opt RR is still different. */
+ BOOST_CHECK_EQUAL(PC.get(&ecs1, &r2), false);
+
+ /* inserting the version with ECS Client Subnet set,
+ it should NOT replace the existing EDNS one. */
+ PC.insert(&ecs1, &r, 3600);
+ BOOST_CHECK_EQUAL(PC.size(), 3);
+
+ /* different subnet of same size, should NOT match
+ since we don't skip the option */
+ BOOST_CHECK_EQUAL(PC.get(&ecs2, &r2), false);
BOOST_CHECK_EQUAL(r2.qdomain, r.qdomain);
- PC.purge("www.powerdns.com");
- BOOST_CHECK_EQUAL(PC.get(&q, &r2), 0);
+ /* different subnet of different size, should NOT match. */
+ BOOST_CHECK_EQUAL(PC.get(&ecs3, &r2), false);
+
+ BOOST_CHECK_EQUAL(PC.purge("www.powerdns.com"), 3);
+ BOOST_CHECK_EQUAL(PC.get(&q, &r2), false);
+ BOOST_CHECK_EQUAL(PC.size(), 0);
PC.insert(&q, &r, 3600);
- BOOST_CHECK_EQUAL(PC.get(&q, &r2), 1);
+ BOOST_CHECK_EQUAL(PC.size(), 1);
+ BOOST_CHECK_EQUAL(PC.get(&q, &r2), true);
BOOST_CHECK_EQUAL(r2.qdomain, r.qdomain);
- PC.purge("com$");
- BOOST_CHECK_EQUAL(PC.get(&q, &r2), 0);
+ BOOST_CHECK_EQUAL(PC.purge("com$"), 1);
+ BOOST_CHECK_EQUAL(PC.get(&q, &r2), false);
+ BOOST_CHECK_EQUAL(PC.size(), 0);
PC.insert(&q, &r, 3600);
- BOOST_CHECK_EQUAL(PC.get(&q, &r2), 1);
+ BOOST_CHECK_EQUAL(PC.size(), 1);
+ BOOST_CHECK_EQUAL(PC.get(&q, &r2), true);
BOOST_CHECK_EQUAL(r2.qdomain, r.qdomain);
- PC.purge("powerdns.com$");
- BOOST_CHECK_EQUAL(PC.get(&q, &r2), 0);
+ BOOST_CHECK_EQUAL(PC.purge("powerdns.com$"), 1);
+ BOOST_CHECK_EQUAL(PC.get(&q, &r2), false);
+ BOOST_CHECK_EQUAL(PC.size(), 0);
PC.insert(&q, &r, 3600);
- BOOST_CHECK_EQUAL(PC.get(&q, &r2), 1);
+ BOOST_CHECK_EQUAL(PC.size(), 1);
+ BOOST_CHECK_EQUAL(PC.get(&q, &r2), true);
BOOST_CHECK_EQUAL(r2.qdomain, r.qdomain);
- PC.purge("www.powerdns.com$");
- BOOST_CHECK_EQUAL(PC.get(&q, &r2), 0);
+ BOOST_CHECK_EQUAL(PC.purge("www.powerdns.com$"), 1);
+ BOOST_CHECK_EQUAL(PC.get(&q, &r2), false);
+ BOOST_CHECK_EQUAL(PC.size(), 0);
PC.insert(&q, &r, 3600);
- PC.purge("www.powerdns.net");
- BOOST_CHECK_EQUAL(PC.get(&q, &r2), 1);
+ BOOST_CHECK_EQUAL(PC.size(), 1);
+ BOOST_CHECK_EQUAL(PC.purge("www.powerdns.net"), 0);
+ BOOST_CHECK_EQUAL(PC.get(&q, &r2), true);
BOOST_CHECK_EQUAL(r2.qdomain, r.qdomain);
- PC.purge("net$");
- BOOST_CHECK_EQUAL(PC.get(&q, &r2), 1);
+ BOOST_CHECK_EQUAL(PC.size(), 1);
+
+ BOOST_CHECK_EQUAL(PC.purge("net$"), 0);
+ BOOST_CHECK_EQUAL(PC.get(&q, &r2), true);
BOOST_CHECK_EQUAL(r2.qdomain, r.qdomain);
- PC.purge("www.powerdns.com$");
+ BOOST_CHECK_EQUAL(PC.size(), 1);
+
+ BOOST_CHECK_EQUAL(PC.purge("www.powerdns.com$"), 1);
BOOST_CHECK_EQUAL(PC.size(), 0);
}
catch(PDNSException& e) {
- cerr<<"Had error in threadReader: "<<e.reason<<endl;
+ cerr<<"Had error in AuthPacketCache: "<<e.reason<<endl;
throw;
}
-}
+}
BOOST_AUTO_TEST_SUITE_END()
#include "config.h"
#endif
#include <boost/test/unit_test.hpp>
-#include "packetcache.hh"
+#include "auth-packetcache.hh"
+#include "auth-querycache.hh"
+#include "statbag.hh"
StatBag S;
-PacketCache PC;
-
+AuthPacketCache PC;
+AuthQueryCache QC;
#include <boost/archive/binary_iarchive.hpp>
#include <boost/archive/binary_oarchive.hpp>
-#include "packetcache.hh"
+#include "auth-querycache.hh"
#include "utility.hh"
for_each(backends.begin(),backends.end(),del);
}
-// silly Solaris fix
-#undef PC
-
// returns -1 for miss, 0 for negative match, 1 for hit
int UeberBackend::cacheHas(const Question &q, vector<DNSZoneRecord> &rrs)
{
- extern PacketCache PC;
- static AtomicCounter *qcachehit=S.getPointer("query-cache-hit");
- static AtomicCounter *qcachemiss=S.getPointer("query-cache-miss");
+ extern AuthQueryCache QC;
if(!d_cache_ttl && ! d_negcache_ttl) {
- (*qcachemiss)++;
return -1;
}
rrs.clear();
// L<<Logger::Warning<<"looking up: '"<<q.qname+"'|N|"+q.qtype.getName()+"|"+itoa(q.zoneId)<<endl;
- bool ret=PC.getEntry(q.qname, q.qtype, PacketCache::QUERYCACHE, rrs, q.zoneId); // think about lowercasing here
+ bool ret=QC.getEntry(q.qname, q.qtype, rrs, q.zoneId); // think about lowercasing here
if(!ret) {
- (*qcachemiss)++;
return -1;
}
- (*qcachehit)++;
if(rrs.empty()) // negatively cached
return 0;
void UeberBackend::addNegCache(const Question &q)
{
- extern PacketCache PC;
+ extern AuthQueryCache QC;
if(!d_negcache_ttl)
return;
// we should also not be storing negative answers if a pipebackend does scopeMask, but we can't pass a negative scopeMask in an empty set!
- PC.insert(q.qname, q.qtype, PacketCache::QUERYCACHE, vector<DNSZoneRecord>(), d_negcache_ttl, q.zoneId);
+ QC.insert(q.qname, q.qtype, vector<DNSZoneRecord>(), d_negcache_ttl, q.zoneId);
}
void UeberBackend::addCache(const Question &q, const vector<DNSZoneRecord> &rrs)
{
- extern PacketCache PC;
+ extern AuthQueryCache QC;
if(!d_cache_ttl)
return;
return;
}
- PC.insert(q.qname, q.qtype, PacketCache::QUERYCACHE, rrs, store_ttl, q.zoneId);
+ QC.insert(q.qname, q.qtype, rrs, store_ttl, q.zoneId);
}
void UeberBackend::alsoNotifies(const DNSName &domain, set<string> *ips)
#include "json.hh"
#include "webserver.hh"
#include "logger.hh"
-#include "packetcache.hh"
#include "statbag.hh"
#include "misc.hh"
#include "arguments.hh"
#include <iomanip>
#include "zoneparser-tng.hh"
#include "common_startup.hh"
-
+#include "auth-caches.hh"
using json11::Json;
extern StatBag S;
-extern PacketCache PC;
static void patchZone(HttpRequest* req, HttpResponse* resp);
static void storeChangedPTRs(UeberBackend& B, vector<DNSResourceRecord>& new_ptrs);
}
sd.db->commitTransaction();
- PC.purgeExact(rr.qname);
+ purgeAuthCachesExact(rr.qname);
}
}
}
di.backend->commitTransaction();
- PC.purgeExact(zonename);
+ purgeAuthCachesExact(zonename);
// now the PTRs
storeChangedPTRs(B, new_ptrs);
DNSName canon = apiNameToDNSName(req->getvars["domain"]);
- int count = PC.purgeExact(canon);
+ uint64_t count = purgeAuthCachesExact(canon);
resp->setBody(Json::object {
- { "count", count },
- { "result", "Flushed cache." }
+ { "count", (int) count },
+ { "result", "Flushed cache." }
});
}
corrupt-packets=0
deferred-cache-inserts=0
deferred-cache-lookup=0
+deferred-packetcache-inserts=0
+deferred-packetcache-lookup=0
dnsupdate-answers=0
dnsupdate-changes=0
dnsupdate-queries=0
key-cache-size=0
meta-cache-size=1
overload-drops=0
-packetcache-size=8
+packetcache-size=4
qsize-q=0
+query-cache-size=4
rd-queries=0
recursing-answers=0
recursing-questions=0
projects / pdns / commitdiff