Author Archives: Kirrus

Varnish weird error

Are you getting a weird error from varnish that you’re having trouble working out? Is it cryptically saying that there’s an issue with your host definition in your VCL?

Expected ID got ';'
 (program line 165), at
 ('input' Line 20 Pos 30)
 .first_byte_timeout = 120;

You need to add a time unit to your timeout definitions — ‘s’ for seconds, ‘m’ for minutes.  Stupid cryptic error is dumbly cryptic. Grrr.

ARGH! A Rant about software developers

When software developers update their software to include new configuration options, they have a bunch of options to pick from, of how to handle installs running on the old options.

For example, you could

  1.  automatically update config
  2.  document a process to gracefully update config
  3.  Give human readable errors showing what changes you need to make to conifg
  4.  Break the application with a super cryptic error message, and refuse to start until config is wiped with ‘new’ default config.

Why do developers keep picking 4? At least 2 horribly complex bits of software in the past year have decided to do that to me, which cost me a few days of head-bashing-against-desk trying to work out what was broken time. 🙁

Varnish nsca logging on systemd system with x-forwarded-for

So, you have a Varnish server running systemd, which is behind a reverse proxy for SSL like nginx, and you can’t work out how to make varnishncsa log IP addresses from a specified header? Well, it’s a bit of a pain in the neck really. You need to override the systemd service file, which is like systemd’s version of the init scripts. Due to it being systemd, this is not just a case of editing a file…

For Debian, you can use the service file below, and paste it into

Once done, you need to reload systemd’s service listing itself;
$ systemctl daemon-reload

Congratulations, you now have varnishncsa logs including the visitor’s real IP address, as specified by Nginx. Change the name in “{X-Forwarded-For}” to change the header name, for example if you want CloudFlare’s view of the client’s IP address, use “CF-Connecting-IP”

 ExecStart=/usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -D -P /run/varnishncsa/ -F '%%{X-Forwarded-For}i %%l %%u %%t "%%r" %%s %%b "%%{Referer}i" "%%{User-agent}i"'
 ExecReload=/bin/kill -HUP $MAINPID


mysql/mariadb failing to start under systemd

If SystemD MariaDB/MySQL is failing to start, make sure your logs directory is set correctly. Sometimes this is caused by failing to correctly make sure the directory is moved.

You’ll see an error log in the journal as follows;

Nov 27 18:24:55 db-a mysqld[28677]: 2017-11-27 18:24:55 140273229838208 [Note] /usr/sbin/mysqld (mysqld 10.2.10-MariaDB-10.2.10+maria~jessie-log) starting as process 28677
Nov 27 18:24:55 db-a systemd[1]: mariadb.service: main process exited, code=exited, status=1/FAILURE
Nov 27 18:24:55 db-a systemd[1]: Failed to start MariaDB database server.
Nov 27 18:24:55 db-a systemd[1]: Unit mariadb.service entered failed state.

SetEnvIf https (Tell apache we’re behind a reverse proxy handling SSL)

This is complicated, but I need to remember this, so I want to put it somewhere. You need to add this to apache config / .htaccess file to get apache to correctly set the HTTPS environment variable when the backend is secure. It trips based on the X-Forwarded-Proto header being ‘https’.

SetEnvIf X-Forwarded-Proto "https" HTTPS=on

Office music

isolation with headphones

or Togetherness by synchronised streaming?


Apps used: 

  • Logitech media server (aka squeezebox) – running on a  Raspberry Pi
  • Squeezer (Android) for remote control, queuing tracks, playlist management and synchronising the players
  • SB player (Android) to play the 

    Ps, it’s way past my bedtime! Insomnia is a pain. Goodnight all. 

    Attention span

    Ran across this interesting video the other day, thanks to YouTube recommendations. Tl;dr version, internet-addicted humans have (possibly) screwed their attention spans, thanks to unrestricted access to the internet.

    I’m not sure. I’m pretty reliant on the internet for my social support structure, and use it to (try to) help others, with what I can.

    I’ve not really found that my attention span has seriously been diminished via the internet, though I certainly do find myself checking twitter more than I probably should. I read an interesting study/article which I can’t find right now that posited that humans are designed to always seek out new information, since it’s valuable to the clan. Found yourself whiling a few hours away on Wikipedia? Same article mentioned twitter as a source of potential endless new things, leading to it’s potential addictive nature.  Whilst looking for it, I did find this more modern article, which discusses why twitter’s addictive, and how missing empathy feedback loops might be being impacting twitter’s troll problem. An article to think on another day.

    Personally, I try to read everything everyone I follow on twitter writes, which means I have to be particularly strict about who I follow. To be honest, there’s plenty of voices on twitter I’d love to read more from, but who just tweet too damn much for me to be able to keep up. I’ll end this ramble with just a few interesting people on twitter;

    • @swiftonsecurity — Computer security, microfiction
    • @jimrossingol — Game development, rt’s interesting left leaning politics
    • @dannilion — Sufferer of a particularly nasty disease leaving Danni bedbound

    Price of cow manure in Nova Scotia

    fishing cove, nova scotia

    So, I didn’t know what to blog about today, so I asked a friend, and he said that. Huh, I thought. That’s actually kinda weirdly interesting. Let’s ask the googles.

    Google tells me, you can buy 1.5 Kilograms of it in a nice plastic bag from home hardware for $3.49 (Canadian dollars). Or, I did find, you could get goat manure for free! But you’ve gotta collect it yourself.

    A bag of cow shit. Yes, really.

    TBH, I didn’t even know where Nova Scotia was, or in which country. I mean, I’d heard the name before, but couldn’t put the name to a place. Canada! Wikipedia tells me Nova Scotia is the second most-densely populated province in Canada with 17.4 inhabitants per square kilometre (45/sq mi).

    So there you go. That’s today’s blogpost, a random fact about cow manure in Canada.

    Though, it turns out that Nova Scotia is one of the places that suffered an “interesting” past due to British colonial activities, namely the forced relocation of french colonists, costing the lives of thousands, not counting the torn histories of tens of thousands.

    Whilst trying to find a good image for this post, I came across the one above, of Peggys cove. Now I wanna actually go there, just because it looks pretty! Probably never going to go, but one can dream!

    .. Please, save me from myself. Ask me interesting questions I can answer and blogpost about, help me keep keep blogging daily. I’ll try to answer (almost) anything within reason.

    I mean, I could tell you about my day, but it was mostly just normal, relatively boring to relate sysadmin stuff. I did have to help a client with a slightly gnarly DNS thing, but that was just knowing how the timeouts work, not sure there’s anything interesting to post about there.

    Is gnarly spelt gnarly or knarly? I kinda want it to be spelt knarly, but the spell checker reliably informs me it’s gnarly. Shame. I’m going to stop typing here. Bye!