Posts

Posted on

Initial foray into Terraform / OpenTofu

So over the last couple of weeks at work, I’ve been learning to use Terraform (well OpenTofu) to help us manage multiple deployments in Azure and AWS.

The thought being that we can have a single ‘plan’ of what a deployment should look like, and deviations will be spotted / can be alerted on.

I was tempted to try and write a contrived article showing how you could create a VM in AWS (or Azure) using Terraform, but I’m not sure I’ve got anything to add over the 101 other articles on the internet.

Vaguely useful things :

  • The tofu configuration is much quicker to write than e.g. trying to talk to AWS using it’s SDK (something I did do about 7-8 years ago)
  • You can split the config up into multiple .tf files within your working directory, the tool just merges them all together at run time
  • Having auto-complete in an editor is pretty much necessary (in my case, PHPStorm)
  • tofu is quite quick to run – it doesn’t take all that long to check the state of the known resources and the config files, which is good; unfortunately Azure often takes sometime to do something on its end…
  • I’ve yet to see any point in writing a module to try and encapsulate any of our configuration as I can’t see any need to re-use bits anywhere

I’m not sure how we’re going to go about reconciling our legacy (production) environment with a newer / shiny one built with tofu though.

Posted on

New Computer Time? Perhaps not quite yet

Well, I almost went ahead and bought a Beelink SER8 – after my current desktop (Beelink SER6 Max) kept crashing at random intervals (normally between 3 and 6 days apart).

Of course, since I threatened to replace the SER6 with something else, it’s behaved itself…. typical hardware

ALMOST as annoying as a printer.

Posted on

Animal Licensing – Paws On the Doors

Last Monday saw the launch of Paws On the Doors – Animal Licensing – which aims to have animal licensing data for England – providing the public with a single place to lookup animal licensing inspections / data.

Under the hood it’s using PHP/Laravel, TypeSense and some magic fairy dust.

It’s kind of related to ScoresOnTheDoors – UK Food Hygiene Ratings – but with a refreshed code base (we felt it was time to move on from a code base that’s probably 20 something years old and start from a cleaner slate without legacy baggage).

Posted on

atuin – console shell history search

As a random, useful, console tool … try atuin for some magical shell history searching.

I’m too lazy to try and record a semi-useful demo of it, but it’s replaced my ‘ctrl+r’ lookup thingy and has a good search interface.

Posted on

LetsEncrypt + Azure Keyvault + Application gateway

A few years ago I setup an Azure Function App to retrieve a LetsEncrypt certificate for a few $work services.

Annoyingly that silently stopped renewing stuff.

Given I’ve no idea how to update it etc or really investigate it (it’s too much of a black box) I decided to replace it with certbot etc, hopefully run through a scheduled github action.

To keep things interesting, I need to use the Route53 DNS stuff to verify domain ownership.

Random bits :

docker run --rm -e AWS_ACCESS_KEY_ID="$AWS_ACCESS_KEY_ID" \
-e AWS_SECRET_ACCESS_KEY="$AWS_SECRET_ACCESS_KEY" \
-v $(pwd)/certs:/etc/letsencrypt/ \
-u $(id -u ${USER}):$(id -g ${USER}) \
certbot/dns-route53 certonly \
--agree-tos \
--email=me@example.com \
--server https://acme-v02.api.letsencrypt.org/directory \
--dns-route53 \
--rsa-key-size 2048 \
--key-type rsa \
--keep-until-expiring \
--preferred-challenges dns \
--non-interactive \
--work-dir /etc/letsencrypt/work \
--logs-dir /etc/letsencrypt/logs \
--config-dir /etc/letsencrypt/ \
-d mydomain.example.com

Azure needs the rsa-key-size 2048 and type to be specified. I tried 4096 and it told me to f.off.

Once that’s done, the following seems to produce a certificate that keyvault will accept, and the load balancer can use, that includes an intermediate certificate / some sort of chain.

cat certs/live/mydomain.example.com/{fullchain.pem,privkey.pem} > certs/mydomain.pem 

openssl pkcs12 -in certs/mydomain.pem -keypbe NONE -cetpbe NONE -nomaciter -passout pass:something -out certs/something.pfx -export

az keyvault certificate import --vault-name my-azure-vault  -n certificate-name -f certs/something.pfx --password something

Thankfully that seems to get accepted by Azure, and when it’s applied to an application gateway listener, clients see an appropriate chain.

Posted on

ssh signed git commits

git config –global gpg.format ssh
git config –global user.signingkey ~/.ssh/id_ed25519.pub
git config –global commit.gpgsign true

Hopefully that’ll result in my github commits being signed…. and when I forget how to do it …

Posted on

Minimal WordPress Fail2ban integration

I used to have a fail2ban filter etc setup to look for POST requests to wp-login.php; but the size of the Apache log files on one server made this infeasible (it took fail2ban too long to parse/process the files). Also, doing a filter on the Apache log file looking for POST /wp-login … means you are also catching someone successfully logging in.

Perhaps this is a better approach :

Assumptions

  • You’re using PHP configured with an error_log = /var/log/php.log

    • If this isn’t configured, PHP will probably log to the webserver’s error log file (/var/log/apache2/error.log perhaps).

  • The Apache/PHP processes are able to write to the error_log file.

  • You’re using Debian or Ubuntu Linux

Add a ‘must use’ wordpress plugin

Put this in … /path/to/your/site/wp-content/mu-plugins/log-auth-failures.php

(It must be wp-content/mu-plugins … )

<?php
add_action( ‘wp_login_failed’, ‘login_failed’ );
function login_failed( $username ) {
error_log(“WORDPRESS LOGIN FAILURE {$_SERVER[‘REMOTE_ADDR’]} – user $username from ” . __FILE__);
}

(Yes, obviously you don’t have to use error_log, you could do something else, and there’s a good argument not to log $username as it’s ultimately user supplied data that might just mess things up)

Fail2ban config

Then in /etc/fail2ban/jail.d/wordpress-login.conf :

[wordpress-login]
enabled = true
filter = wordpress-login
action = iptables-multiport[name=wp, port="80,443", protocol=tcp]
logpath = /var/log/php.log
maxretry = 5

If you have PHP logging somewhere else, change the logpath appropriately.

Finally in /etc/fail2ban/filter.d/wordpress-login.conf put :

[Definition]

# PHP error_log is logging to /var/log/php.log something like :
#[31-Jan-2024 20:34:10 UTC] WORDPRESS LOGIN FAILURE 1.2.3.4  - user admin.person from /var/www/vhosts/mysite/httpdocs/wp-content/mu-plugins/log-auth-failures.php

failregex = WORDPRESS LOGIN FAILURE <HOST> - user 


ignoreregex =

Extra bonus points for making the failregex stricter, or stop including $username in the log output (which perhaps makes it vulnerable to some sort of injection attack).

There’s probably a good argument for using a different file (not the PHP error_log) so other random error messages can’t confuse fail2ban, which might also allow you to specify a more friendly date format for fail2ban etc….

Finally …

Restart fail2ban and watch it’s log file (and /var/log/php.log).

service fail2ban restart

Posted on

Upgrade some things

Well, I sort of realised I had a web server or two that were still on Debian Buster, and it was time to move to Bullseye or Bookworm. As usual the Debian upgrade procedure was mostly pretty straight forward and uneventful.

Interesting findings :

  • “hitch“, which I use as an SSL frontend to varnish, doesn’t seem to get along all that well with systemd and silently fails if your config has “daemon = on” setting in /etc/hitch/hitch.conf. Annoyingly when trying to test the configuration with “hitch -t” you will get an error like: “No x509 certificate PEM file specified for frontend ‘default’!” – the solution to that is to specify the config file – i.e : hitch -t --config /etc/hitch/hitch.conf

  • hitch hasn’t had a release in it’s packagecloud.io repository for the last 3 years; so the debian supported variant looks more appealing.

In other news, I noticed this post where someone moaned about systemd-resolved the other day – https://www.reddit.com/r/linux/comments/18kh1r5/im_shocked_that_almost_no_one_is_talking_about/ – I’ve had similar problems to the people on the thread (resolved stops working etc) so thought it was time to try and use ‘unbound‘ instead.

apt-get install unbound

and then tell /etc/resolv.conf to use 127.0.0.1 for DNS.

annoyingly, unbound-control stats isn’t quite as pretty as resolvectl statistics but oh well.

echo -e “nameserver 127.0.0.1\nnameserver 8.8.8.8\noptions timeout:4” >/etc/resolv.conf

and an /etc/unbound/unbound.conf file that looks perhaps like :

server:
interface: 127.0.0.1
access-control: 127.0.0.0/8 allow
access-control: ::1/128 allow
# The following line will configure unbound to perform cryptographic
# DNSSEC validation using the root trust anchor.
auto-trust-anchor-file: "/var/lib/unbound/root.key"
tls-cert-bundle: "/etc/ssl/certs/ca-certificates.crt"

remote-control:
control-enable: yes
# by default the control interface is is 127.0.0.1 and ::1 and port 8953
# it is possible to use a unix socket too
control-interface: /run/unbound.ctl

forward-zone:
name: "."
forward-tls-upstream: yes
forward-addr: 1.1.1.1@853#cloudflare-dns.com
forward-addr: 1.0.0.1@853#cloudflare-dns.com

(Unfortunately my ISP is shitty, and doesn’t yet give me an ipv6 address).

Looking at https://1.1.1.1/help – I do sometimes see that ‘DNS over TLS’ is “yes”…. so I guess something is right; annoyingly I don’t see anything useful from unbound’s stats (unbound-control stats) to show it’s done a secure query…

“unbound-host” (another debian package) – will helpfully tell you whether a lookup was done ‘securely’ or not – e.g.

$ unbound-host google.com -D -v
google.com has address 142.250.178.14 (insecure)
google.com has IPv6 address 2a00:1450:4009:815::200e (insecure)
google.com mail is handled by 10 smtp.google.com. (insecure)

which seems a little odd to me (I’d have thought google would support dns sec), but some domains do work – e.g.

$ unbound-host mythic-beasts.com -D -v
mythic-beasts.com has address 93.93.130.166 (secure)
mythic-beasts.com has IPv6 address 2a00:1098:0:82:1000:0:1:2 (secure)
mythic-beasts.com mail is handled by 10 mx1.mythic-beasts.com. (secure)
mythic-beasts.com mail is handled by 10 mx2.mythic-beasts.com. (secure)