Simple DNS for your basic needs
Intro
Sometimes you don't need (or want) some complex solution to simple problem like DNS. Therefore this one will be the simplified version of previous guide with bind as only element.
For this one you will again need podman
. If you are (like me in this case) doing this on centOS
or similar machine, getting podman
is as simple as:
# dnf install podman
If you are on some other distro, it shouldn't be that complicated.
Building container image
You could probably find one on dockerhub
or some other image repo, but building our own is simple and makes sure we know what we are running.
The goal of this will be simple, create a dns server that will provide a custom local domain, and forward everything else to upstream of choice.
All my guides follow same folder structure to keep all the files:
/containers
├── build
│ └── bind
└── run
└── bind
└── etc
Run following commands to create it:
# mkdir -p /containers/build/bind
# mkdir -p /containers/run/bind/etc
We'll create a simple Dockerfile
in /containers/build/bind
with following content:
FROM alpine:latest
LABEL maintainer="Marvin Sinister"
RUN addgroup -S -g 2021 bind && adduser -S -u 2001 -G bind bind; \
apk add --no-cache ca-certificates bind-tools bind; \
rm -rf /var/cache/apk/*; \
mkdir /var/cache/bind;
RUN chown -R bind: /etc/bind; \
chown -R bind: /var/cache/bind;
HEALTHCHECK --interval=5s --timeout=3s --start-period=5s CMD nslookup ns.domain.tld 127.0.0.1 || exit 1
USER bind
CMD ["/bin/sh", "-c", "/usr/sbin/named -g -4"]
There is really nothing special about it, we are using latest alpine base image, installing bind, fixing some permissions and adding healthcheck. Make sure you use your domain instead of ns.domain.tld
in HEALTHCHECK
.
We can build the container now:
# podman build . -t bind
STEP 1: FROM alpine:latest
STEP 2: LABEL maintainer="Marvin Sinister"
--> Using cache 573e94441dfdbd7dcfa8e232ce7baedb9860192d749efd679b2bb8ccf64a797d
--> 573e94441df
STEP 3: RUN addgroup -S -g 2021 bind && adduser -S -u 2001 -G bind bind; apk add --no-cache ca-certificates bind-tools bind; rm -rf /var/cache/apk/*; mkdir /var/cache/bind;
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/community/x86_64/APKINDEX.tar.gz
(1/53) Installing ca-certificates (20191127-r5)
...
(53/53) Installing bind (9.16.11-r0)
Executing bind-9.16.11-r0.pre-install
Executing bind-9.16.11-r0.post-install
wrote key file "/etc/bind/rndc.key"
Executing busybox-1.32.1-r2.trigger
Executing ca-certificates-20191127-r5.trigger
OK: 41 MiB in 67 packages
--> e90abd46aba
STEP 4: RUN chown -R bind: /etc/bind; chown -R bind: /var/cache/bind;
--> c5b324d0cff
STEP 5: HEALTHCHECK --interval=5s --timeout=3s --start-period=5s CMD nslookup ns.domain.tld 127.0.0.1 || exit 1
--> 3c7e5ae2cf3
STEP 6: USER bind
--> bbb48348aa0
STEP 7: CMD ["/bin/sh", "-c", "/usr/sbin/named -g -4"]
STEP 8: COMMIT bind
--> 1bd34f66b06
1bd34f66b066f714124253631db94d50a21643bf3b8c2fbb6539862d447bee32
If you are running recent enought version of podman
you might get warnings about HEALTCHECK
not being supported. In that case, just add --format docker
to end of build command.
You should now see the image:
# podman image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
localhost/bind latest 1bd34f66b066 3 minutes ago 41.9 MB
docker.io/library/alpine latest e50c909a8df2 5 days ago 5.88 MB
Creating configurations
The main config goes into etc/named.conf
:
// This is the primary configuration file for the BIND DNS server named.
// If you are just adding zones, please do that in /etc/bind/named.conf.local
include "/etc/bind/named.conf.options";
include "/etc/bind/named.conf.local";
Next, lets do options, etc/named.conf.options
:
options {
directory "/var/cache/bind";
forwarders {
1.1.1.1;
1.0.0.1;
};
recursion yes;
allow-query { lan; };
dnssec-validation auto;
auth-nxdomain no; # conform to RFC1035
listen-on { any; };
};
Here we define our forwarders, in this case cloudflares 1.1.1.1
, allow the recursion and limit the queries to our local network (defined later).
Now, let's define local domains:
acl lan {
127.0.0.1;
};
zone "domain.tld" {
type master;
file "/etc/bind/db.domain.tld";
};
zone "122.168.192.in-addr.arpa" {
type master;
notify no;
file "/etc/bind/db.122.168.192.in-addr.arpa";
};
In this configuration we define the acl, we are only allowing localhost because bind will run in container and all calls will be from localhost from containers point of view. Next we define our domain domain.tld
and our reverse domain 122.168.192.in-addr.arpa
. The reverse zone should correspond to your ip range, in this case 192.168.122.0/24
.
Let's define the zones themself, first the forward zone domain.tld
:
; BIND data file for domain.tld zone
;
$TTL 86400
@ IN SOA ns.domain.tld. root.domain.tld. (
5 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
86400 ) ; Negative Cache TTL
;
@ IN NS ns.domain.tld.
ns IN A 192.168.122.254
; hosts
containers IN A 192.168.122.254
This is where we add our dns entries, in this case we have two, ns.domain.tld
and containers.domain.tld
both pointing to 192.168.122.254
, the address of our podman host containers.domain.tld
.
And lats, the reverse zone:
;
; BIND reverse data file for lan zone
;
$TTL 604800
@ IN SOA ns.domain.tld. root.domain.tld. (
5 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
;
@ IN NS ns.domain.tld.
254 IN PTR ns.domain.tld.
; hosts
254 IN PTR containers.domain.tld.
Similar to forward zone, the reverse zone defines the reverse pointers so you can find what hosts are behind some address.
Since bind uses DNSSEC to check the upstream repositories, we need to provide the upstream keys, you can download them from ICS website from bind.keys (at the time of writing this document). Download the file and save it as etc/bind.keys
.
Since we specified user with id 2021
in Dockerfile
we will make that user owner of those files:
# chown -R 2021 /containers/run/bind
Running the pod
While we could run this as a container, the decision was made to standardize on pods. There is no major reason to go one way or other, but standardization generally provides benefits longtime. In that spirit, let's create the pod:
# podman pod create --name dns -p '192.168.122.254:53:53/udp'
And run the bind container within:
# podman run -d --name bind --pod dns -v '/containers/run/bind/etc:/etc/bind:Z' --user 2021 localhost/bind:latest
To allow external machines access to our new dns, we need to open 53/udp
on firewall:
# firewall-cmd --add-service=dns --permanent
# firewall-cmd --reload
Once everything starts, we can try running some dns queries to check if everyting is okay, install the bind-utils
to get dig
command:
# dnf install bind-utils
And then check if you can resolv some addresses:
# dig +short @192.168.122.254 google.com
172.217.16.110
# dig +short @192.168.122.254 ns.domain.tld
192.168.122.254
And that's it, now you can point your network to 192.168.122.254
to resolv you local domain.
Infrastructure monitoring with grafana and friends
Table of content
Intro
In this guide we will look into how to configure infrastructure monitoring using the Grafana. Besides grafana itself, we'll use prometheus for metrics aggregation, node_exporter for log collection, loki for log agregation and promtail for log collection.
For this one you will obviously need podman
. If you are (like me in this case) doing this on centOS
or similar machine, getting podman
is as simple as:
# dnf install podman
If you are on some other distro, it shouldn't be that complicated.
Now that we have podman
let's talk about what exactly we are doing. We want to achieve following:
- collect metrics from local and remote machines
- collect logs from local and remote machines
- display everything in pretty dashboards
For those who are not familiar, let's go through each component.
grafana
is a web dashboard for visualizing data. It's most commonly used to visualize different metrics.
prometheus
is a monitoring system with time series database and alerting capabilities.
node_exporter
is one of many metrics exporters for prometheus, in this case exporting the metrics of node it's running on.
loki
is a log aggregation system inspired by prometheus.
promtail
is an agent to collect logs and send them to loki
.
With all this out of our way, let's get started.
Creating configuration files
Not unlike last time, we build a simple folder structure to keep all the files:
/containers
└── run
├── grafana
├── loki
└── prometheus
So there is containers
folder in root of the filesystem, that holds runtime files for our containers. We don't need to build any containers, we'll use upstream for all our needs.
grafana
The grafana
config resides in following folder structure:
grafana
├── etc
│ └── grafana
│ └── provisioning
│ ├── dashboards
│ ├── datasources
│ ├── notifiers
│ └── plugins
└── var
└── lib
└── grafana
To create it, run following commands:
# mkdir -p /containers/run/grafana/etc/grafana/provisioning/{datasources,plugins,notifiers,dashboards}
# mkdir -p /containers/run/grafana/var/lib/grafana
Here, we will keep both configuration and persistent data. To begin, let's grab the default config from git:
# cd /containers/run/grafana/etc/grafana
# wget https://raw.githubusercontent.com/grafana/grafana/master/conf/sample.ini -O grafana.ini
We will be running the container with user 2012
so let's change the ownership of that folder structure:
# chown -R 2012 /containers/run/grafana
We can now modify the config to suit our needs, since this is not a production, we won't change much or worry too much about security. We'll do the following:
[server]
# The public facing domain name used to access grafana from a browser
domain = grafana.domain.tld
# Redirect to correct domain if host header does not match domain
# Prevents DNS rebinding attacks
enforce_domain = true
[security]
# default admin password, can be changed before first start of grafana, or in profile settings
admin_password = YOUR_STRONG_PASSWORD
[log]
# Either "console", "file", "syslog". Default is console and file
# Use space to separate multiple modes, e.g. "console file"
mode = console
You should by now have a dns server, so add an entry for grafana and fill in the domain
in config. We'll force the redirect for good measure. Under security
configure the admin password. Since this runs within container we don't want it to write logs to disk, so we'll change the log mode to console
only.
You can look through the config and change whatever else you feel is important.
node_exporter
Before starting to configure prometheus, let's figure what data we want to collect. Since we want to monitor some kind of IT infrastructure, let's assume that we are interested in node metrics. To get the data into prometheus, we need some kind of exporter. To have some kind of data wihtin pormetheus, let's get the metrics from our container host.
Let's add the ibotty/prometheus-exporter repo and install the node_exporter
package:
# curl -Lo /etc/yum.repos.d/_copr_ibotty-prometheus-exporters.repo https://copr.fedorainfracloud.org/coprs/ibotty/prometheus-exporters/repo/epel-8/ibotty-prometheus-exporters-epel-8.repo
# dnf install node_exporter
Once installed, we can run and enable the service:
# systemctl start node_exporter
# systemcrl enable node_exporter
We can check if everything is okay by running:
# curl localhost:9100/metrics
# HELP go_gc_duration_seconds A summary of the GC invocation durations.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 0
go_gc_duration_seconds{quantile="0.25"} 0
go_gc_duration_seconds{quantile="0.5"} 0
go_gc_duration_seconds{quantile="0.75"} 0
go_gc_duration_seconds{quantile="1"} 0
go_gc_duration_seconds_sum 0
go_gc_duration_seconds_count 0
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
...
And a lot more metrics from the node.
prometheus
Now that we have some data to work with, let's get it into prometheus. As usual we'll start with desired folder structure:
prometheus
├── etc
│ └── prometheus
├── prometheus
│ └── data
└── var
└── lib
└── prometheus
To create it, run:
# mkdir -p /containers/run/prometheus/etc/prometheus
# mkdir -p /containers/run/prometheus/prometheus/data
# mkdir -p /containers/run/prometheus/var/lib/prometheus
Here, we will provide very simple config for prometheus to scrape the local machine in etc/prometheus/prometheus.yml
:
# A scrape configuration scraping a Node Exporter and the Prometheus server
# itself.
global:
scrape_interval: 5s
scrape_configs:
# Scrape Prometheus itself every 5 seconds.
- job_name: 'prometheus'
static_configs:
- targets: ['localhost:9090'] # Scrape the Node Exporter every 5 seconds.
- job_name: 'containers.domain.tld'
static_configs:
- targets: ['192.168.122.254:9100']
With this config, prometheus will collect metrics for itself, and for the container node it's running on. Notice that since prometheus is runnnig within container, we can't scrape the node config on localhost, but have to use the IP address of the host.
Just fix the permissions:
chown -R 2011 /containers/run/prometheus
And we are done with prometheus config.
loki
As usual, first we need a folder structure to hold all the data and config, for loki it looks like this:
loki
├── etc
│ └── loki
└── loki
And we can create it by running:
# mkdir -p /containers/run/loki/etc/loki
# mkdir -p /containers/run/loki/loki
We can get the default config from git:
# cd /containers/run/loki/etc/loki
# wget https://raw.githubusercontent.com/grafana/loki/master/cmd/loki/loki-local-config.yaml
The only thing we'll change inside is the location where loki will store chunks:
storage_config:
# ...
filesystem:
directory: /loki/chunks
Changing it from /tmp/loki/chunks
to /loki/chunks
.
The only thing left to do is to fix the ownership of the folder:
chown -R 2013 /containers/run/loki
promtail
Unlike prometheus which scrapes the metrics endpoints, loki needs logs pushed into it. For that role we'll use promtail. Since we'll push logs from host, we'll install it on our container host and configure to push logs to loki.
Promtail ships as a single binary, so we can download it from directly from github. Let's download the package and put it in /usr/local/bin
:
# cd /usr/local/bin
# wget https://github.com/grafana/loki/releases/download/v2.1.0/promtail-linux-amd64.zip
# unzip promtail-linux-amd64.zip
# mv promtail-linux-amd64 promtail
# rm promtail-linux-amd64.zip
Next, let's create an user that will run promtail:
# useradd --system promtail
# usermod -aG systemd-journal promtail
We have added the user to systemd-journal
group, so it will be able to read journal and send it to loki.
Next, we will create the config folder:
# mkdir /etc/promtail
And add congig in /etc/promtail/promtail-local-config.yaml
server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /tmp/positions.yaml
clients:
- url: http://127.0.0.1:3100/loki/api/v1/push
scrape_configs:
- job_name: journal
journal:
max_age: 12h
labels:
job: systemd-journal
relabel_configs:
- source_labels: ['__journal__systemd_unit']
target_label: 'unit'
The config is fine for our needs, it reads systemd journal and pushes it to localhost
on port 3100
.
Let's create a systemd service file for promtail in /etc/systemd/system/promtail.service
:
[Unit]
Description=Promtail service
After=network.target
[Service]
Type=simple
User=promtail
ExecStart=/usr/local/bin/promtail -config.file /etc/promtail/promtail-local-config.yaml
[Install]
WantedBy=multi-user.target
And finally update the systemd daemon files:
# systemctl daemon-reload
Podman
At last, it's time to create and start the pod. To create pod run:
# podman pod create --name mon -p '127.0.0.1:9090:9090/tcp' -p '127.0.0.1:3000:3000/tcp' -p '127.0.0.1:3100:3100/tcp'
For a tyniest bit of theory you can check last post.
We are runing three containers within mon
pod, and we map the ports for grafana (3000/tcp
), prometheus (9090/tcp
) and loki (3100/tcp
). All of them are mapped to localhost, so we will need some kind of proxy in front which we'll do later with NGINX.
First, create and start grafana:
# podman run -d --name grafana --pod mon -v '/containers/run/grafana/var/lib/grafana:/var/lib/grafana:Z' -v '/containers/run/grafana/etc/grafana:/etc/grafana:Z' --user 2012 grafana/grafana
If we try to connect to it:
# curl localhost:3000
<a href="http://grafana.domain.tld:3000/">Moved Permanently</a>.
We see that we are redirected to grafana.domain.tld
, so grafana is running and our config is used.
Next, lets do prometheus:
podman run -d --name prometheus --pod mon -v '/containers/run/prometheus/var/lib/prometheus:/var/lib/prometheus:Z' -v '/containers/run/prometheus/prometheus:/prometheus:Z' -v '/containers/run/prometheus/etc/prometheus:/etc/prometheus:Z' --user 2011 prom/prometheus --config.file=/etc/prometheus/prometheus.yml --web.route-prefix=/ --storage.tsdb.retention.time=200h --web.enable-lifecycle
And again, running:
# curl localhost:9090
<a href="/graph">Found</a>.
Shows us that something is running on that port (spoiler: it's prometheus).
And last one, loki:
# podman run -d --name loki --pod mon -v '/containers/run/loki/loki:/loki:Z' -v '/containers/run/loki/etc/loki:/etc/loki:Z' --cpus=1 --user 2013 grafana/loki -config.file=/etc/loki/loki-local-config.yaml
And aftera few moments we can check:
# curl localhost:3100/ready
ready
Telling us that loki is ready to receive data.
Web access
At this point you should have a dns server with custom domain. Add the entries for our services to your dns:
grafana.domain.tld
prometheus.domain.tld
loki.domain.tld
Also, you should have NGINX installed. If not, simply run:
# dnf install nginx
And don't forget SELINUX if you have it enabled:
# setsebool -P httpd_can_network_connect 1
Once everything is prepared, create configs new virtual hosts in /etc/nginx/conf.d/
.
grafana.conf
for grafana:
server {
listen 192.168.122.254:80;
server_name grafana.domain.tld;
root /usr/share/nginx/html;
index index.html index.htm;
location / {
proxy_pass http://127.0.0.1:3000/;
proxy_set_header Host $host;
}
access_log /var/log/nginx/grafana.access.log;
error_log /var/log/nginx/grafana.error.log;
}
prometheus.conf
for prometheus:
server {
listen 192.168.122.254:80;
server_name prometheus.domain.tld;
root /usr/share/nginx/html;
index index.html index.htm;
location / {
proxy_pass http://127.0.0.1:9090/;
proxy_set_header Host $host;
}
access_log /var/log/nginx/prometheus.access.log;
error_log /var/log/nginx/prometheus.error.log;
}
And, you guessed it, loki.conf
for loki:
server {
listen 192.168.122.254:3100;
server_name loki.domain.tld;
root /usr/share/nginx/html;
index index.html index.htm;
location / {
proxy_pass http://127.0.0.1:3100/;
proxy_set_header Host $host;
}
access_log /var/log/nginx/loki.access.log;
error_log /var/log/nginx/loki.error.log;
}
If you are following carefully, you will notice that loki is running on port 3100
, and if you have any experience with SELINUX you'll know that it loves to complain, so let's fix it by adding port 3100
to list of http ports in http_port_t
variable:
# semanage port -a -t http_port_t -p tcp 3100
And now we can (re)start and enable NGINX:
# systemctl restart nginx
# systemctl enable nginx
There is one last obstacle before we can connect to our services, the firewall:
# firewall-cmd --add-service=http --permanent
# firewall-cmd --add-service=https --permanent
# firewall-cmd --add-port=3100/tcp --permanent
# firewall-cmd --reload
At this point we should be able to access all our services. So let's do some exploration and configuration.
prometheus
Go to http://prometheus.domain.tld
in your browser, then in menu choose Status
and Targets
. You should see something similar to this:
Make sure everything is UP
and there are no errors.
gafana metrics
Now that we know that prometheus is collecting metrics, let's see some visualizations in grafana. Go to http://grafana.domain.tld/datasources/new
, login and choose prometheus.
Enter http://localhost:9090
as URL and click Save & Test
at the bottom of the page.
Next, hop to http://grafana.domain.tld/dashboard/import
.
Enter 1860
as dashboard id, and press load. On the next screen select our (only) prometheus as data source, and press Import
. You should be revarded with a dashboard similar to one on the picture below.
You can now start exploring what kind of exporters and dashboards are availalbe for prometheus and grafana.
loki
Now that we have some metrics, let's tackle the logs. If you remember we have added the promtail to our node, but we never started it, so let's do that now:
# systemctl start promtail
# systemctl enable promtail
With that sorted, let's turn to loki. To see what data we have in loki, we'll again use grafana. Add another source in grafana by visiting http://grafana.domain.tld/datasources/new
again. But this time choose Loki
and set http://localhost:3100
as URL, save with Save & Test
at the bottom.
Unfortunately i haven't found any nice dashboards to show data from loki, however, if you visit http://grafana.domain.tld/explore
and choose Loki
as source, you can query all logs shiped to loki. If you search for {job="systemd-journal"}
, you'll see everything currently in loki.
You can of course play around with creating different queries or creating a custom dashboard to suit your needs. You can also add other logs to promtail and add them to loki.
This concludes this part of infrastructure monitoring, but there is much more to be done in future.
Overcomplicated homelab DNS configuration
Table of content
Intro
In this guide we will look into how to configure an overcomplicated DNS setup using pihole
, bind
and cloudflared
, running inside a podman
pod
.
For this one you will obviously need podman
. If you are (like me in this case) doing this on centOS
or Red Hat
machine, getting podman
is as simple as:
# dnf install podman
If you are on some other distro, it shouldn't be that complicated.
Now that we have podman
let's talk about what exactly we are doing. We want to achieve following:
- custom domain(s) for home lab
- DNS over HTTPS to cloudflare
- DNS blackhole with
pihole
For those who are not familiar, let's go through each component.
pihole
is a dns blackhole, it has lists of malicious and/or unwanted addresses and discards them. You can find it at pihole.net and consider it a network wide AD blocker. It also has a web interface that you can use for configuration and tracking of dns queries.
bind
is a nameserver. It's probably most common nameserver in the world, it has many features and it's able to run ISP sized DNS servers. In this case we will just use it to provide a local domain. Speaking of domain, you need to decide what you will use, in this example i'll just use domain.tld
.
Cloudflare is a company that provides internet services related to security and performance. Similar to googles 8.8.8.8 dns, cloudflare provides their own dns server at 1.1.1.1. Since cloudflare is not an AD revenue driven corporation, I prefer them over google. cloudflared
is a daemon
that forwards UDP dns requests over HTTPS to cloudflare.
So the path of request will be as follows:
origin -> bind -> pihole -> cloudflared -> cloudflare
Building container images
Now that we have all the basics covered, let's start building the images. First, we build a simple folder structure to keep all the files:
/containers
├── build
│ ├── bind
│ └── cloudflared
└── run
├── bind
└── pihole
So there is containers
folder in root of the filesystem, that holds build and runtime files for our containers. We need to build two containers, bind
and cloudflared
. We'll start with bind
.
BIND
For bind, we create a simple Dockerfile
in /containers/build/bind
with following content:
FROM alpine:latest
LABEL maintainer="Marvin Sinister"
RUN addgroup -S -g 2001 bind && adduser -S -u 2001 -G bind bind; \
apk add --no-cache ca-certificates bind-tools bind; \
rm -rf /var/cache/apk/*; \
mkdir /var/cache/bind;
RUN chown -R bind: /etc/bind; \
chown -R bind: /var/cache/bind;
HEALTHCHECK --interval=5s --timeout=3s --start-period=5s CMD nslookup -port 5053 ns.domain.tld 127.0.0.1 || exit 1
USER bind
CMD ["/bin/sh", "-c", "/usr/sbin/named -g -4 -p 5053"]
What we are doing here is building a simple container from alpine
, creating user, installing the service and fixing some permissions. Two things to notice:
- healthcheck running
nslookup
onlocalhost
port5053
for address of our nameserver for chosen domaindomain.tld
- the bind command itself running service on port
5053
And build the container image with:
# podman build . -t bind
STEP 1: FROM alpine:latest
STEP 2: LABEL maintainer="Marvin Sinister"
--> aebbb98fb2e
STEP 3: RUN addgroup -S -g 2001 bind && adduser -S -u 2001 -G bind bind; apk add --no-cache ca-certificates bind-tools bind; rm -rf /var/cache/apk/*; mkdir /var/cache/bind;
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/community/x86_64/APKINDEX.tar.gz
(1/53) Installing ca-certificates (20191127-r5)
(2/53) Installing brotli-libs (1.0.9-r3)
...
(52/53) Installing bind-dnssec-root (9.16.11-r0)
(53/53) Installing bind (9.16.11-r0)
Executing bind-9.16.11-r0.pre-install
Executing bind-9.16.11-r0.post-install
wrote key file "/etc/bind/rndc.key"
Executing busybox-1.32.1-r2.trigger
Executing ca-certificates-20191127-r5.trigger
OK: 41 MiB in 67 packages
--> c72cd8464c8
STEP 4: RUN chown -R bind: /etc/bind; chown -R bind: /var/cache/bind;
--> dcdfc85fd12
STEP 5: HEALTHCHECK --interval=5s --timeout=3s --start-period=5s CMD nslookup -port 5053 ns.domain.tld 127.0.0.1 || exit 1
--> 0d742c04892
STEP 6: USER bind
--> 54b96184563
STEP 7: CMD ["/bin/sh", "-c", "/usr/sbin/named -g -4 -p 5053"]
STEP 8: COMMIT bind:latest
--> 3a18e54af59
3a18e54af590947fd0230193a02675f26010ab2a177e859305f0f3f98d9c22e6
If you are running recent enought version of podman
you might get warnings about HEALTCHECK
not being supported. In that case, just add --format docker
to end of build command.
Once finished we can list the images with:
# podman image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
localhost/bind latest 3a18e54af590 About a minute ago 89.2 MB
docker.io/library/alpine latest e50c909a8df2 2 days ago 5.88 MB
cloudflared
Same procedure as bind, create a Dockerfile
in /containers/build/cloudflared
:
ARG ARCH=amd64
FROM golang:alpine as gobuild
ARG GOARCH
ARG GOARM
RUN apk update; \
apk add git gcc build-base; \
go get -v github.com/cloudflare/cloudflared/cmd/cloudflared
WORKDIR /go/src/github.com/cloudflare/cloudflared/cmd/cloudflared
RUN GOARCH=${GOARCH} GOARM=${GOARM} go build ./
FROM multiarch/alpine:${ARCH}-edge
LABEL maintainer="Marvin Sinister"
ENV DNS1 1.1.1.1
ENV DNS2 1.0.0.1
RUN addgroup -S -g 2002 cloudflared && adduser -S -u 2002 -G cloudflared cloudflared; \
apk add --no-cache ca-certificates bind-tools; \
rm -rf /var/cache/apk/*;
COPY --from=gobuild /go/src/github.com/cloudflare/cloudflared/cmd/cloudflared/cloudflared /usr/local/bin/cloudflared
HEALTHCHECK --interval=5s --timeout=3s --start-period=5s CMD nslookup -po=5054 cloudflare.com 127.0.0.1 || exit 1
USER cloudflared
CMD ["/bin/sh", "-c", "/usr/local/bin/cloudflared proxy-dns --address 127.0.0.1 --port 5054 --upstream https://${DNS1}/dns-query --upstream https://${DNS2}/dns-query"]
Similar to bind, but this time we use golang:alpine
to build and multiarch\alpine:amd64-edge
to run the cloudflared daemon. And this time the health check is for cloudflared.com
address, and the port where service is running is 5054
.
And run:
# podman build . -t cloudflared:latest
STEP 1: FROM golang:alpine AS gobuild
STEP 2: ARG GOARCH
--> 793a7c5f9f9
STEP 3: ARG GOARM
--> 3bb89556acd
STEP 4: RUN apk update; apk add git gcc build-base; go get -v github.com/cloudflare/cloudflared/cmd/cloudflared
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/community/x86_64/APKINDEX.tar.gz
v3.13.1-12-g1c7edde73a [https://dl-cdn.alpinelinux.org/alpine/v3.13/main]
v3.13.1-13-g9824e6cf00 [https://dl-cdn.alpinelinux.org/alpine/v3.13/community]
OK: 13878 distinct packages available
(1/26) Installing libgcc (10.2.1_pre1-r3)
...
(26/26) Installing git (2.30.0-r0)
Executing busybox-1.32.1-r2.trigger
OK: 208 MiB in 41 packages
github.com/cloudflare/cloudflared (download)
...
github.com/cloudflare/cloudflared/cmd/cloudflared
--> 07c20f86a9d
STEP 5: WORKDIR /go/src/github.com/cloudflare/cloudflared/cmd/cloudflared
--> 92fd2da44f5
STEP 6: RUN GOARCH=${GOARCH} GOARM=${GOARM} go build ./
# github.com/mattn/go-sqlite3
sqlite3-binding.c: In function 'sqlite3SelectNew':
sqlite3-binding.c:125322:10: warning: function may return address of local variable [-Wreturn-local-addr]
125322 | return pNew;
| ^~~~
sqlite3-binding.c:125282:10: note: declared here
125282 | Select standin;
| ^~~~~~~
--> 36f3ba92883
STEP 7: FROM multiarch/alpine:amd64-edge
STEP 8: LABEL maintainer="Marvin Sinister"
--> a133bb4e290
STEP 9: ENV DNS1 1.1.1.1
--> c68bcb25fe6
STEP 10: ENV DNS2 1.0.0.1
--> ce1efac0a83
STEP 11: RUN addgroup -S -g 2002 cloudflared && adduser -S -u 2002 -G cloudflared cloudflared; apk add --no-cache ca-certificates bind-tools; rm -rf /var/cache/apk/*;
fetch http://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
(1/18) Installing fstrm (0.6.0-r1)
...
(18/18) Installing ca-certificates (20191127-r5)
Executing busybox-1.33.0-r1.trigger
Executing ca-certificates-20191127-r5.trigger
OK: 22 MiB in 38 packages
--> 3fc0d971570
STEP 12: COPY --from=gobuild /go/src/github.com/cloudflare/cloudflared/cmd/cloudflared/cloudflared /usr/local/bin/cloudflared
--> f99b87627f7
STEP 13: HEALTHCHECK --interval=5s --timeout=3s --start-period=5s CMD nslookup -po=5054 cloudflare.com 127.0.0.1 || exit 1
--> c0b3a24146e
STEP 14: USER cloudflared
--> 66804934da3
STEP 15: CMD ["/bin/sh", "-c", "/usr/local/bin/cloudflared proxy-dns --address 127.0.0.1 --port 5054 --upstream https://${DNS1}/dns-query --upstream https://${DNS2}/dns-query"]
STEP 16: COMMIT cloudflared:latest
--> 7d8b02575fe
7d8b02575fea97bc09ca94fdf85569c47d15112f3930405e8c25d477fa491aaf
After a while we have the new image:
# podman image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
localhost/cloudflared latest 7d8b02575fea 3 minutes ago 95.7 MB
localhost/bind latest 3a18e54af590 18 minutes ago 89.2 MB
docker.io/multiarch/alpine amd64-edge 901ee590dcdb 22 hours ago 25.8 MB
docker.io/library/golang alpine 54d042506068 2 days ago 308 MB
docker.io/library/alpine latest e50c909a8df2 2 days ago 5.88 MB
pihole
We will use the upstream docker images for pihole.
Creating configuration files
Now that we have images, we will create the configuration files to run services.
cloudflared
Let's begin with the simplest one. cloudflared
doesn't need any kind of configuration, we'll run it as pure ephemeral container.
pihole
There is not much we need to configure for pihole
. We will just prepare the folders for persistence. The configuration options will be provided as environment variables on runtime.
We need to create the following folder structure under /containers/run/
:
pihole/
└── etc
├── dnsmasq.d
└── pihole
And that's it for pihole for the moment.
BIND
The most configuration will be required for bind
. Here we will define our own domain.
Create the following structure under /containers/run/
:
bind/
└── etc
Here we have two options, either copy the config from existing bind
; by either running a container without mounting volumes or having existing bind
or creating them ourselves. Since it's simple config I'll just put all files here.
The main config goes into etc/named.conf
:
// This is the primary configuration file for the BIND DNS server named.
//
// Please read /usr/share/doc/bind9/README.Debian.gz for information on the
// structure of BIND configuration files in Debian, *BEFORE* you customize
// this configuration file.
//
// If you are just adding zones, please do that in /etc/bind/named.conf.local
include "/etc/bind/named.conf.options";
include "/etc/bind/named.conf.local";
And then we need to create the rest of the bunch, etc/named.conf.options
:
options {
directory "/var/cache/bind";
forwarders {
127.0.0.1 port 5054;
};
recursion yes;
allow-query { lan; };
dnssec-validation auto;
auth-nxdomain no; # conform to RFC1035
listen-on port 5053 { any; };
};
The things configured here:
- forward the unknown queries to
127.0.0.1
on port5054
which is the address ofpihole
- allow recursion
- limit queries to
lan
acl (defined later) - listen on port
5053
on all interfaces
Next, etc/named.conf.local
:
acl lan {
10.88.0.0/16;
127.0.0.1;
};
zone "domain.tld" {
type master;
file "/etc/bind/db.domain.tld";
};
zone "122.168.192.in-addr.arpa" {
type master;
notify no;
file "/etc/bind/db.122.168.192.in-addr.arpa";
};
The things configured here:
- the
lan
access control list allowing10.88.0.0/16
, which is the default address space ofpodman
and127.0.0.1
which islocalhost
- the forward zone for
domain.tld
(defined later) - the reverse zone for
192.168.122.0/24
(defined later) - any other zones (not in scope of this document)
Now we can create our own domain.tld
zone in etc/db.domain.tld
file with following content:
; BIND data file for domain.tld zone
;
$TTL 86400
@ IN SOA ns.domain.tld. root.domain.tld. (
5 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
86400 ) ; Negative Cache TTL
;
@ IN NS ns.domain.tld.
ns IN A 192.168.122.254
; hosts
containers IN A 192.168.122.254
; services
pihole IN A 192.168.122.254
Reverse domain will in this case go into db.122.168.192.in-addr.arpa
, where the numbers in file name corespod to reverse of the ip range, in this case we are doing reverse zone for 192.168.122.0/24
so the file is named 122.168.192
(droping the last 0
because it's a /24
domain).
;
; BIND reverse data file for lan zone
;
$TTL 604800
@ IN SOA ns.domain.tld. root.domain.tld. (
5 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
;
@ IN NS ns.domain.tld.
254 IN PTR ns.domain.tld.
; hosts
254 IN PTR containers.domain.tld.
A few notes:
- If your domain spans across multiple subnets, you will wnat multiple reverse zones
- Notice that we have two forward addresses (
containers.domain.tld
andpihole.domain.tld
) but only one reverse entry (containers.domain.tld
). While you can have multiple reverse (PTR
) records, you don't have to.
Since bind uses DNSSEC to check the upstream repositories, we need to provide the upstream keys, you can download them from ICS website from bind.keys (at the time of writing this document). Download the file and save it as etc/bind.keys
.
Since we specified user with id 2001
in Dockerfile
we will make that user owner of those files:
# chown -R 2001 /containers/run/bind
And the config is done. Almost there.
Podman
Finally it's time to create and start the pod. To create pod run:
# podman pod create --name dns -p '192.168.122.254:53:53/udp' -p '127.0.0.1:8080:80/tcp' -p '127.0.0.1:8443:443/tcp'
A bit of theory time. Pods are similar to containers, but within a pod, you can run multiple containers. The networks within pod is shared, therefore, you define the port mappings on pod level, and not for each container. In this case we'll run three containers within pod, and we map the following ports:
- The starndard dns port
53/udp
to our host address, so that it's available from whole network 80/tcp
and443/tcp
to our localhost. This ispihole
web interface. We are making it available only from localhost, but we can stick a reverse proxy in front later.
We also give the pod a name, in this case dns
.
And once whe have a pod running, we can start the containers:
# podman run -d --name cloudflared --pod dns --user 2002 localhost/cloudflared:latest
# podman run -d --name bind --pod dns -v '/containers/run/bind/etc:/etc/bind:Z' --user 2001 localhost/bind:latest
# podman run -d --name pihole --pod dns -v '/containers/run/pihole/etc/pihole:/etc/pihole:Z' -v '/containers/run/pihole/etc/dnsmasq.d:/etc/dnsmasq.d:Z' -e=ServerIP='192.168.122.254' -e=DNS1='127.0.0.1#5053' -e=DNS2='no' -e=IPv6='false' -e=TZ='Europe/Berlin' -e=WEBPASSWORD='MY_STRONG_PASSWORD' pihole/pihole:latest
As you can notice, we are binding the appropriate folders to each container, and providing a few extra options to pihole
, of wich password is the one you should probably change.
Once everything starts, you can try running some dns queries to check if everyting is okay, if you don't have them install bind-utils
to get dig
command:
# dnf install bind-utils
And then check if you can resolv some addresses:
# dig +short @192.168.122.254 google.com
172.217.16.110
# dig +short @192.168.122.254 ns.domain.tld
192.168.122.254
To access the dns service from outside the host, we need to open 53/udp
on firewall:
# firewall-cmd --add-service=dns
# firewall-cmd --add-service=dns --permanent
Web access for pihole
To access web interface of pihole
we need a proxy in front, in this case we'll use NGINX. First thing we need to do is install it:
# dnf install nginx
Once done, create the config file for new virtual host in /etc/nginx/conf.d/pihole.conf
:
server {
listen 192.168.122.0:80;
server_name pihole.domain.tld;
root /usr/share/nginx/html;
index index.html index.htm;
location / {
proxy_pass http://127.0.0.1:8080/;
}
access_log /var/log/nginx/pihole.access.log;
error_log /var/log/nginx/pihole.error.log;
}
And start and enable the service:
# systemctl start nginx
# systemctl enable nginx
A few more details, open the ports in firewall:
# firewall-cmd --add-service=http --permanent
# firewall-cmd --add-service=https --permanent
# firewall-cmd --reload
And allow nginx to connect to pihole on localhost
in selinux
:
# setsebool -P httpd_can_network_connect 1
You should be able to access pihole
at http://192.168.122.254
in your web browser and login using your password. You can also point other machines on your network to use it as dns server. And of course, you should add pihole to your local domain.
Adding geoip2 to NGINX
The following guide will help with seting up GeoIP2 database and logging locations of source IP adresses in NGINX access log. The guide is for debian, but should be simple to adopt for other distributions.
To start, we need some packages, the packages are available in bullseye
repo:
# apt update
# apt install libnginx-mod-http-geoip2
We also need the GeoIP database. You can download one for free at maxmind website. You need to create an account and create API key. The files can be downloaded from their webpage, but if you have the API key you can use the following links:
https://dev.maxmind.com/geoip/geoip2/geolite2/GeoLite2-Country&license_key=GEOIP2_API_KEY&suffix=tar.gz
https://dev.maxmind.com/geoip/geoip2/geolite2/GeoLite2-City&license_key=GEOIP2_API_KEY&suffix=tar.gz
Once you have those extract them and place the .mmdb
files into /etc/nginx/geoip
folder:
# ls /etc/nginx/geoip/
GeoLite2-City.mmdb GeoLite2-Country.mmdb
With the database and packages sorted, we can configure NGINX. Create a new config /etc/nginx/conf.d/geoip2.conf
:
##
# Add GEO IP support
##
geoip2 /etc/nginx/geoip/GeoLite2-Country.mmdb {
auto_reload 5m;
$geoip2_metadata_country_build metadata build_epoch;
$geoip2_data_country_code source=$remote_addr country iso_code;
$geoip2_data_country_name country names en;
}
geoip2 /etc/nginx/geoip/GeoLite2-City.mmdb {
$geoip2_data_city_name city names en;
}
# Enabling request time and GEO codes
log_format custom '$remote_addr - $remote_user [$time_local]'
' "$request" $status $body_bytes_sent'
' "$http_referer" "$http_user_agent"'
' "$request_time" "$upstream_connect_time"'
' "$geoip2_data_country_code" "$geoip2_data_country_name"'
' "$geoip2_data_city_name"';
access_log /var/log/nginx/access.log custom;
Test the config with nginx -t
and restart NGINX:
# systemctl restart nginx
After which the access log will contain lines with country codes:
# cat /var/log/nginx/access.log
198.20.99.130 - - [31/Jan/2021:11:58:00 +0100] "GET /robots.txt HTTP/1.1" 200 26 "-" "-" "0.000" "-" "NL" "Netherlands" "-"
Depending on your existing config you might end up with double logs of each request, the standard NGINX log, and the custom log with GeoIP. In that case edit /ext/nginx/nginx.conf
and remove (coment) the access_log
line.