Adding DNS Adblock to my Unbound Configuration
Adding DNS Adblock to my Unbound Configuration
In this post I share how I added DNS adblocking to the Unbound DNS server configuration on my OpenBSD firewall.
In a previous post I shared my experiences setting up an OpenBSD router that included Unbound as a caching (non-authoritative) DNS server. After having run with this for a couple of weeks, I decided that I wanted to try implementing something similar to a PiHole to block domains used for ads and other nefarious content.
![](/blog/adding-dns-adblock-to-unbound/firefox-error.png)
Sourcing a List of Domains
Finding a list of domains to block proved to be very easy thanks to the efforts of Steven Black and the contributors to a set of consolidated host files on GitHub. This repository contains a consolidated list of hosts from multiple curated sources. The repo includes different hosts file variants based around different categories such as gambling, social, porn, fakenews, and so on.
I picked the Unified hosts + fakenews + gambling + social variant which, as the name suggests, includes the unified ads and malware domains, along with fake news, gambling, and social domains. At time of writing, this includes 177,286 distinct domains.
The README in the repository includes a list of downloads, with links to the README and hosts file download for each variant.
/etc/hosts
file. You can read
more about it on the manpage for hosts(5).Here is a sample from the top of the hosts file.
# Title: StevenBlack/hosts
#
# This hosts file is a merged collection of hosts from reputable sources,
# with a dash of crowd sourcing via GitHub
#
# Date: 02 July 2024 03:24:15 (UTC)
# Number of unique domains: 163,646
#
# Fetch the latest version of this file: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
# Project home page: https://github.com/StevenBlack/hosts
# Project releases: https://github.com/StevenBlack/hosts/releases
#
# ===============================================================
127.0.0.1 localhost
127.0.0.1 localhost.localdomain
127.0.0.1 local
255.255.255.255 broadcasthost
::1 localhost
::1 ip6-localhost
::1 ip6-loopback
fe80::1%lo0 localhost
ff00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
0.0.0.0 0.0.0.0
# Custom host records are listed here.
# End of custom host records.
# Start StevenBlack
#=====================================
# Title: Hosts contributed by Steven Black
# http://stevenblack.com
0.0.0.0 ck.getcookiestxt.com
0.0.0.0 eu1.clevertap-prod.com
0.0.0.0 wizhumpgyros.com
0.0.0.0 coccyxwickimp.com
0.0.0.0 webmail-who-int.000webhostapp.com
0.0.0.0 010sec.com
0.0.0.0 01mspmd5yalky8.com
0.0.0.0 0byv9mgbn0.com
... 170,000+ lines ...
In order to use this file with Unbound, I needed to make some changes.
Configuring Unbound
To configure Unbound, I needed to process the hosts file I got from Steven Black’s repository into
something that Unbound can use. To configure Unbound to block a domain, we want to use a
local-zone
configuration for each of the blocked domains. Each local-zone
configuration will
configure the reply that Unbound will send if there is no local-data
matching the domain. Because
I want to block these domains, I use the refuse
option, which tells Unbound to respond with the
REFUSED
response code (number 5), which is a good response for policy decisions.
# Block my own domain
local-zone: "blakerain.com" refuse
local-zone
attribute.To create this configuration, I needed to convert each of the hosts mentioned in the downloaded
hosts file into a local-zone
attribute in Unbound’s configuration language. Taking a look at the
unified hosts file downloaded from the repository we can see that the blocked domains are all
associated with the host 0.0.0.0
. We can find all those lines by running the hosts file through
grep
:
$ grep '^0\.0\.0\.0' hosts | head
0.0.0.0 ck.getcookiestxt.com
0.0.0.0 eu1.clevertap-prod.com
0.0.0.0 wizhumpgyros.com
0.0.0.0 coccyxwickimp.com
0.0.0.0 webmail-who-int.000webhostapp.com
0.0.0.0 010sec.com
0.0.0.0 01mspmd5yalky8.com
0.0.0.0 0byv9mgbn0.com
0.0.0.0 ns6.0pendns.org
For each of these lines, we can use awk
to reformat the line into an Unbound local-zone
configuration:
$ grep '^0\.0\.0\.0' hosts | awk '{print "local-zone: \""$2"\" refuse"}' | head
local-zone: "ck.getcookiestxt.com" refuse
local-zone: "eu1.clevertap-prod.com" refuse
local-zone: "wizhumpgyros.com" refuse
local-zone: "coccyxwickimp.com" refuse
local-zone: "webmail-who-int.000webhostapp.com" refuse
local-zone: "010sec.com" refuse
local-zone: "01mspmd5yalky8.com" refuse
local-zone: "0byv9mgbn0.com" refuse
local-zone: "ns6.0pendns.org" refuse
I piped all these generated configuration lines into a new adblock.conf
file alongside my current
Unbound configuration (which lives in the /var/unbound/etc
directory). I then updated my
unbound.conf
configuration file to include the adblock.conf
file after my existing local-data
lines:
server:
# ...
# Configure some local network domains
local-data: "cyan.localdomain A 192.168.1.20"
local-data: "blue.localdomain A 192.168.1.24"
# ...
# Include the adblock domains
include: "/var/unbound/etc/adblock.conf"
Next I ran unbound-checkconf
to check the configuration, and as everything was okay, restarted
unbound with rcctl
.
# unbound-checkconf
unbound-checkconf: no errors in /var/unbound/etc/unbound.conf
# rcctl restart unbound
To check that the domain blocking works correctly, on my local machine I made a test query to
facebook.com
. As I had hoped, the server came back with REFUSED
.
$ host facebook.com
Host facebook.com not found: 5(REFUSED)
Firefox and DNS over HTTPS
A few years ago, DNS over HTTPS was introduced with the intention of preserving privacy and increasing security. Unfortunately, one of the disadvantages of this approach is that it frustrates the use of a local DNS server. Fortunately, for now, many applications still let us change whether or not they use DoH, and Firefox is one of them.
My intended outcome of this whole endeavour was primarily to block domains for ads in my browser, and thankfully Firefox still lets you configure DoH. To make sure that Firefox makes use of my local DNS server I disabled DNS over HTTPS in Firefox settings (found at the bottom of the Privacy and Security page in Firefox settings).
![](/blog/adding-dns-adblock-to-unbound/firefox-dns-over-https.png)
Automating with Shell Scripting
Now that I knew how I would populate my Unbound ad-blocking configuration, I wanted to be able to
quickly update the adblock.conf
file with the latest download from the GitHub repository. To
facilitate this, I wrote a short shell script that downloads the hosts file from GitHub for the
chosen variant and transforms the output into the Unbound configuration.
VARIANT=${VARIANT:-fakenews-gambling-social}
URL="https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/$VARIANT/hosts"
wget -qO- "$URL" | grep '^0\.0\.0\.0' | sort | awk '{print "local-zone: \""$2"\" refuse"}'
Now I can update the ad-blocking configuration on my gateway by piping the output of this shell
script into the /var/unbound/etc/adblock.conf
and then restarting Unbound:
# sh gen-adblock.sh > /var/unbound/etc/adblock.conf
# unbound-checkconf
unbound-checkconf: no errors in /var/unbound/etc/unbound.conf
# rcctl restart unbound
unbound(ok)
unbound(ok)
The particular variant that is downloaded is controlled by the VARIANT
variable, which defaults to
my preferred fakenews-gambling-social
. This can be overridden when the script is invoked by
setting the variable prior to running the script:
# VARIANT=fakenews-gambling sh gen-adblock.sh > /var/unbound/etc/adblock.conf
Adding Exceptions
The domain blocking works very nicely, however there are some domains that get mentioned in the
unified hosts file that I actually still want to access. For example, I still want to access
engineering.fb.com
, but this has been blocked by the facebook.com
domain. To allow the
engineering domain, I added a new local-zone
statement after the inclusion of the adblock.conf
file, setting the type to transparent.
server:
# ...
# Configure some local network domains
local-data: "cyan.localdomain A 192.168.1.20"
local-data: "blue.localdomain A 192.168.1.24"
# ...
# Include the adblock domains
include: "/var/unbound/etc/adblock.conf"
# Override some local zones
local-zone: "engineering.fb.com." transparent
Unfortunately this doesn’t always work for well for every domain I want to unblock, especially if
the domain already exists in the adblock.conf
file. For example, the domain code.facebook.com
is
mentioned directly in the hosts file, and so unbound-checkconf
issues a warning about a duplicate
zone configuration:
# unbound-checkconf
[...] unbound-checkconf[87279:0] warning: duplicate local-zone code.facebook.com.
To address this, I decided to update my gen-adblock.sh
script to handle my exceptions to the
ad-blocked domains. To start with I created an adblock.exceptions
file listing the domains I want
to permit, with each domain on one line:
engineering.fb.com
code.facebook.com
I want to remove any lines in the generated adblock.conf
file that contain any of the domains in
the adblock.exceptions
file. To do this, I use awk
to convert each line into a g/re/d
command
for ed
(the line editor). This means that the line code.facebook.com
will become
g/"code\.facebook\.com"/d
. When passed to ed
, this will delete any lines that match the given
regular expression.
Using ed
is quite a fun way to perform a series of edits to a file, as it can accept commands from
stdin
:
(echo 'g/"code\.facebook\.com"/d'; echo w) | ed - adblock.conf
Here we’re echoing the g/re/d
command to delete any lines that match "code.facebook.com"
. Notice
the additional echo w
to tell ed
to write the modified buffer back to the file after we have
performed the modifications.
I changed the gen-adblock.sh
script as follows:
- The script now writes the
adblock.conf
file in the/var/unbound/etc
directory rather than me having to pipe the output of the script into the file myself. - The
adblock.exceptions
file is transformed intoed
commands, and then ran against theadblock.conf
file to delete any matching lines. - The
adblock.exceptions
file is transformed intolocal-zone
statements with thetransparent
type, and written to theadblock.exceptions.conf
configuration file.
The new gen-adblock.sh
script is now as follows:
UNBOUND="/var/unbound/etc"
VARIANT=${VARIANT:-fakenews-gambling-social}
URL="https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/$VARIANT/hosts"
wget -qO- "$URL" | grep '^0\.0\.0\.0' | sort | \
awk '{print "local-zone: \""$2"\" refuse"}' > "$UNBOUND/adblock.conf"
(cat "$HOME/adblock.exceptions" | \
awk '{gsub(/\./,"\\."); print "g/\"" $0 "\"/d"}'; echo w) | ed - "$UNBOUND/adblock.conf"
cat "$HOME/adblock.exceptions" | \
awk '{print "local-zone: \""$0".\" transparent"}' > "$UNBOUND/adblock.exceptions.conf"
Next I updated my unbound.conf
file to include the adblock.exceptions.conf
file instead of
overriding the local zones.
server:
# ...
# Configure some local network domains
local-data: "cyan.localdomain A 192.168.1.20"
local-data: "blue.localdomain A 192.168.1.24"
# ...
# Include the adblock domains and the exceptions
include: "/var/unbound/etc/adblock.conf"
include: "/var/unbound/etc/adblock.exceptions.conf"
Now I can just run gen-adblock.sh
to generate the new Unbound configuration, respecting my
exceptions in the $HOME/adblock.exceptions
file. Then I can check the configuration with
unbound-checkconf
, which no longer shows duplicate local-zone
attributes:
# sh gen-adblock.sh
# unbound-checkconf
unbound-checkconf: no errors in /var/unbound/etc/unbound.conf
Conclusion
Using DNS blocking has so far worked quite nicely. The blocking helps ensure I don’t end up being
directed to something like x.com
.
I have the gen-adblock.sh
script along with my adblock.exceptions
file in my home directory on
the Raspberry Pi. When I want to update the DNS ad-blocker configuration I can just run this script
and then restart Unbound.
I’ve included the source for the gen-adblock.sh
script in the following Gist on GitHub.
![Script to generate adblocking configuration for Unbound](https://github.githubassets.com/images/modules/gists/gist-og-image.png)
Related Posts
Setting Up a Firewall with Raspberry Pi and OpenBSD
Tired with my current firewall, I have decided to switch over to a Raspberry Pi 4 running OpenBSD. In this post I describe how I did this and the problems that I ran into.
2 Jun 2024•17 min read