Posts on blog.dataforce.org.uk

Remote LVM-on-LUKS (via ISCSI) with automatic decrypt on boot

Sun, 25 Jun 2023 15:45:24 +0000

I have recently added some iscsi-backed storage to my proxmox-based server environment, primarily as an off-server location to store backup data.

For a multitude of reasons, such as the sensitive nature of the data, the fact that the physical storage lies outside of my control, and just good security hygiene - I wanted to ensure that the data is all encrypted at rest.

I wanted to be able to use this iscsi as a storage target for proxmox allowing me to just add the volumes to VMs allowing HA, and I didn’t want to have to do encryption inside every VM incase I accidentally forgot to enable it for one of the VMs (remember, the storage is hosted external to me so I have no control over the physical access to it) so to do this I have made use of LUKS encryption on the iscsi block device that I am presented with and then I run LVM over the top of this. (LVM-on-LUKS as-opposed to LUKS-on-LVM)

Updated Theme

Tue, 08 Feb 2022 05:06:54 +0000

Once again, I have replaced the theme of this blog.

Unlike the previous theme, this one is actually one I mostly ended up designing myself rather than just finding one that I mostly liked and running with it, and given that, I figured I’d talk a little bit about the thoughts behind it and how it came to be and what else I’ve done behind the scenes.

Docker Swarm Cluster Improvements

Sat, 21 Aug 2021 22:23:00 +0000

Since my previous posts about running docker-swarm with ceph, I’ve been using this fairly extensively in production and made some changes to the setup that follows on from the previous posts.

Fun with Dell S4048 and ONIE

Tue, 03 Nov 2020 03:01:26 +0000

In $DayJob we make use of Dell S4048-ON Switches for 10G Top-of-Rack (ToR) switching and also sometimes 10G Aggregation/Core for smaller deployments. They’re fairly flexible devices with a high number of 10G ports, some 40Gs and they can do L3 ports and L2 ports. You can also run them either Stacked or in VLT mode for redundancy purposes.

In addition these things use ONIE (Open Network Install Environment) and can run different firmware images - though we almost exclusively run these with DNOS 9 which is the Force10 FTOS code that Dell acquired some time ago rather than DNOS 10.

One evening, I was tasked with an “emergency” build request. We had some kit being shipped to a remote PoP the following day and the intended routers were delayed, so we needed to get something quickly and temporarily in place to take a BGP Transit Feed and deliver VRRP to the rest of the kit. A spare S4048 we had lying around would do the job sufficiently for the time period needed. I figured it wouldn’t take too long to get the base config needed and get it ready to be shipped with the rest of the kit.

So I got the Datacenter to rack/cable/console it so that I could begin configuration then set aside some time in the evening to do the work.

As I was watching the switch boot up I noticed something odd. Turns out the last engineer who had used this device had chosen to install the OpenSwitch OPX ONIE firmware on it instead of the usual DNOS9 firmware. So much for my quick and easy config.

At this point, I could have just reloaded the device into the ONIE installer environment and installed DNOS9 and been done with it all. But, I had a fairly open evening, and I’d not yet really played about much with any of the alternative ONIE OSes, so armed with my Yak Sheers, I thought I’d have a look around.

(After all this, I then re-imaged the device onto our standard deployment image of DNOS9 and completed the required config work that I was supposed to be doing.)

Cisco XConnect L2Protocol Handling

Mon, 27 Apr 2020 03:06:10 +0000

In $DayJob we make fairly extensive use of MPLS ATOM Pseudowires (XConnects) between our various datacenter locations to enable services in different sites to talk to each other at layer2.

The way I describe this to customers is that in essence these act as a “long cable” from Point-A to Point-B. The customer gets a cable at each side to connect to their kit, but in the middle of it there is magic that routes the packets over our network rather than an actual long-cable. Packets that enter 1 side will be pushed out the other side, and vice-versa. We don’t need to know or care what these packets are, we are just transparently transporting them.

Upgrading Ceph in Docker Swarm

Mon, 29 Jul 2019 02:02:22 +0000

This post is a followup to an earlier blog bost regarding setting up a docker-swarm cluster with ceph.

I’ve been running this cluster for a while now quite happily however since setting it up, a new version of ceph has been released - nautilus - so now it’s time for some upgrades.

Note: This post is out of date now.

I would suggest looking at this post and using the docker-compose based upgrade workflow instead, up to the housekeeping part.

I’ve mostly followed https://docs.ceph.com/docs/master/releases/nautilus/#upgrading-from-mimic-or-luminous but adapted it for the fact we’re running everything in docker. I recommend that you have a read though this yourself first to have an idea of what we are doing and why.

(It’s worth noting at this point that this guide was mostly written after the fact based on command history so I may have missed something. It’s always a good idea to do this on a test cluster first, or in a maintenance window!)

Fun with TOTP Codes

Fri, 29 Mar 2019 04:20:00 +0000

This all started with a comment I overheard at work from a colleague talking about a 2FA implementation on a service they were using.

“It works fine on everything except Google Authenticator on iPhone.”

… What? This comment alone immediately piqued my interest, I stopped what I was doing, turned round, and asked him to explain.

He explained that a service he was using provided 2FA support using TOTP codes. As is normal, they provided a QR Code, you scanned it with your TOTP application (Google Authenticator or Authy or so), then you typed in the verification code - and it worked for both Google Authenticator and Authy on his Android phone, but only with Authy and not Google Authenticator on another colleagues iPhone.

This totally nerd sniped me, and I just had to take a look.

Docker Swarm with Ceph for cross-server files

Sun, 10 Feb 2019 21:36:54 +0000

I’ve been wanting to play with Docker Swarm for a while now for hosting containers, and finally sat down this weekend to do it.

Something that has always stopped me before now was that I wanted to have some kind of cross-site storage but I don’t have any kind of SAN storage available to me just standalone hosts. I’ve been able to work around this using ceph on the nodes.

Note: I’ve never used ceph before, I don’t really know what I’m doing with ceph, so this is all a bit of guesswork. I used Funky Penguin’s Geek Cookbook as a basis for some of this, though some things have changed since then, and I’m using base-centOS not AtomicHost (I tried AtomicHost, but wanted a newer-version of docker so switched away).

All my physical servers run Proxmox, and this is no exception. On 3 of these host nodes I created a new VM (1 per node) to be part of the cluster. These all have 3 disks, 1 for the base OS, 1 for Ceph, 1 for cloud-init (The non-cloud-init disks are all SCSI with individual iothreads).

Advent of Code Benchmarking

Sat, 08 Dec 2018 18:19:10 +0000

For a few years now I’ve been enjoying Eric Wastl’s Advent of Code. For those unaware, each year since 2015 Advent of Code provides a 2-part coding challenge every day from December 1st to December 25th.

In previous years, Myself and Chris have been fairly informally trying to see who was able to produce the fastest code (Me in PHP, Chris in Python). In the final week of last year to assist with this, we both made our repos run in Docker and produce time output for each day.

This allowed us to run each other’s code locally to compare fairly without needing to install the other’s dev environment, and made the testing a bit fairer as it was no longer dependant on who had the faster CPU when running their own solution. For the rest of the year this was fine and we carried on as normal. As we got to the end I remarked it would be fun to have a web interface that automatically dealt with it and showed us the scores, but there was obviously no point in doing that once the year was over. Maybe in a future year…

Fast forward to this year. Myself and Chris (and ChrisN) coded up our Day 1 solutions as normal and then some other friends started doing it for the first time. I remembered my plans from the previous year and suggested everyone should also docker-ify their repos… and so they agreed…

mdadm RAID with Proxmox

Mon, 18 Jun 2018 01:47:07 +0000

I recently acquired a new server with 2 drives that I intended to use as RAID1 for a virtualisation host for various things.

My hypervisor of choice is Proxmox (For a few reasons, Support for KVM and LXC primarily, but the fact it’s debian based is a nice bonus, and I really dislike the occasionally-braindead networking implementation from vmware which rules out ESXi)

This particular server does not have a RAID card, so I needed to use a software raid implementation. Out of the box for RAID1 on Proxmox you need to use ZFS, however To keep this box similar to others I have I wanted to use ext4 and mdadm. So we’re going have to do a bit of manual poking to get this how we need it.

This post is mostly an aide-memoire for myself for the future.

DNS Hosting - Part 3: Putting it all together

Sun, 10 Jun 2018 18:31:21 +0000

In my previous posts I discussed the history leading up to, and the eventual rewrite of my DNS hosting solution. So this post will (finally) talk briefly about how it all runs in production on MyDNSHost.

Shortly before the whole rewrite I’d found myself playing around a bit with Docker for another project, so I decided early on that I was going to make use of Docker for the main bulk of the setup to allow me to not need to worry about incompatibilities between different parts of the stack that needed different versions of things, and to update different bits at different times.

The system is split up into a number of containers (and could probably be split up into more).

DNS Hosting - Part 2: The rewrite

Sat, 20 Jan 2018 17:25:32 +0000

In my previous post about DNS Hosting I discussed the history leading up to when I decided I needed a better personal DNS hosting solution. I decided to code one myself to replace what I had been using previously.

I decided there was a few things that were needed:

Fully-Featured API
- I wanted full control over the zone data programmatically, everything should be possible via the API.
- The API should be fully documented.
Fully-Featured default web interface.
- There should be a web interface that fully implements the API. Just because there is an API shouldn’t mean it has to be used to get full functionality.
- There should exist nothing that only the default web ui can do that can’t be done via the API as well.
Multi-User support
- I also host some DNS for people who aren’t me, they should be able to manage their own DNS.
Domains should be shareable between from users
- Every user should be able to have their own account
- User accounts should be able to be granted access to domains that they need to be able to access
  - Different users should have different access levels:
    - Some just need to see the zone data
    - Some need to be able to edit it
    - Some need to be able to grant other users access
Backend Agnostic
- The authoritative data for the zones should be stored independently from the software used to serve it to allow changing it easily in future

These were the basic criteria and what I started off with when I designed MyDNSHost.

HUGO PPA

Mon, 08 Jan 2018 09:39:47 +0000

I run ubuntu on my servers, and since moving to Hugo, I wanted to make sure I was using the latest version available.

The ubuntu repos currently contain hugo version 0.15 in Xenial, and 0.25.1 in artful (And the next version, bionic only contains 0.26). The latest version of hugo (as of today) is currently 0.32.2 - so the main repos are quite a bit out of date.

So to work around this, I’ve setup an apt repo that tracks the latest release for hugo, which can be installed and used like so:

sudo wget http://packages.dataforce.org.uk/packages.dataforce.org.uk_hugo.list -O /etc/apt/sources.list.d/packages.dataforce.org.uk_hugo.list
wget -qO- http://packages.dataforce.org.uk/pubkey.gpg | sudo apt-key add -
sudo apt-get update
sudo apt-get install hugo

This repo tracks the latest hugo debs in all 4 of the architectures supported: amd64, i386, armhf and arm64 and should stay automatically up to date with the latest version.

DNS Hosting - Part 1: History

Wed, 03 Jan 2018 01:15:06 +0000

For as long as I can remember I’ve hosted my own DNS. Originally this was via cpanel on a single server that I owned and then after a while I moved to a new server away from cpanel and moved to doing everything myself (web, email, dns) - hand-editing zone files and serving them via BIND.

This was great for learning, and this worked well for a while but eventually I ended up with more servers, and a lot more domains. Manually editing DNS Zone files and reloading BIND was somewhat of a chore any time I was developing things or spinning up new services or moving things between servers - I wanted something web-based.

There wasn’t many free/cheap providers that did what I wanted (This was long before Cloudflare or Route 53), so around 2007 I did what any (in)sane person would do - I wrote a custom control panel for managing domains. and email. and it was multi-user. and it had billing support. and a half-baked ticket-system… ok, so I went a bit overboard with the plans for it. But mainly it controlled DNS and throughout it’s lifetime that was the only bit that was “completed” and fully functional.

Moving to Hugo

Sat, 30 Dec 2017 04:56:26 +0000

For a while now I’ve been thinking of moving this blog to a statically generated site rather than using wordpress.

There are a number of reasons for this:

I can version-control the content rather than relying on wordpress database backups.
It renders quicker
I don’t actually use any of the wordpress features, so it’s just bloat, and a potential security hole.

I’ve seen Chris successfully use Hugo for his site and it seems to do exactly what I want, So I took the opportunity during the Christmas break to spend some time converting my blog from Wordpress to Hugo.

Actually doing this was a multi-stage process.

…

Sun, 28 May 2017 01:34:18 +0000

It did not help.

It’s been a while…

Sat, 28 May 2016 19:59:22 +0000

It’s been a while since I updated this. Not through a lack of wanting to, more a combination of things - lack of time, lack of anything worth writing, hating the old blog theme…

So I’ve replaced the theme!

Maybe that’ll help.

It probably won’t.

Limiting the effectiveness of DNS Amplification

Sat, 03 Aug 2013 02:25:19 +0000

I recently had the misfortune of having a server I am responsible for used as a target for DNS Amplification, and thought I’d share how I countered this. (Whilst this was effective for me, your mileage may vary, but if this actually helps someone then it’s worth posting about.)

This particular server was the main recursor for the site that it was located at (And this was correctly limited not to allow open recursion), but was also authoritative for a small selection of domains. (Yes I know mixing recursors and resolvers is bad.)

The problem only came about when I needed to relocate the server to another site. In order to ensure continuity of service whilst the nameserver IP change propagated, I added some port-forwards at the old site that redirected DNS traffic to the new site. This however meant that all DNS traffic going towards the server came from an IP that was trusted for recursion. Oops.

Website Reshuffle

Sun, 23 Sep 2012 17:37:44 +0000

Over the past few weekends I’ve been (slowly) working on moving my websites around a bit so that things are all in once place, and in the case of this blog, no longer hosted on my home ADSL connection.

At the moment all non-existent pages on http://home.dataforce.org.uk/, http://dataforce.org.uk/ and http://shanemcc.co.uk/ will now redirect to the equivalent link on http://blog.dataforce.org.uk/, over time I will work on moving all public content from these sites over to here (There isn’t much, they’ve mostly been used as dumping grounds!)

After this http://home.dataforce.org.uk/ will be primarily for private things and http://dataforce.org.uk/ and http://shanemcc.co.uk/ will simply redirect here. Eventually I may look to transition this blog over to one of the raw domains (probably dataforce.org.uk). Ultimately I’m trying to do this without breaking any links that may exist to files/etc on these domains.

So if stuff breaks over the next few weeks, that’s why. Feel free to leave a comment if you notice anything or something goes missing.

GMail – apply labels to email from group members – Redux

Wed, 12 Sep 2012 01:30:18 +0000

A while ago I posted a python script that allowed automatically adding labels to GMail messages based on contact groups.

Unfortunately, a side effect of this script was that Google occasionally would lock an account out for “suspicious activity”, and for this reason I stopped using the script.

However recently I looked at Google Apps Scripts to see if this would allow me to recreate this using Google-Approved APIs, and the good news is, yes it does.

The following script implements the same behaviour as the old python script. It checks every thread from the past 2 dates (so today, and yesterday) and then for each message in the thread gets the list of groups the sender is in (if the sender is a contact, and in any groups) and then checks to see if there are labels that match the same name, if so it applies them to the message.

To get this running, create a new project on the Google Apps script page, then paste the code in.

Modify scheduledProcessInbox and processInboxAll to include a label prefix if desired (eg contacts/) and then enable the desired schedule (click on the clock icon in the toolbar). Once this has been scheduled you can run an initial pass over the inbox using processInboxAll() - however this is limited to the last 500 threads.

The code can now be found here on github

Any questions/comments/bugs please leave them here or on github.

Microsoft Lync on Linux

Fri, 20 Apr 2012 11:55:26 +0000

Update: This post still gets a lot of search traffic hits, but is now over a year old, and I no longer have a need to use Lync, so haven’t needed to keep this working.

I believe that the Ubuntu repos now contain new enough versions of SIPE that the deb mentioned here shouldn’t be needed any more, but that the rest of the instructions should still be valid.

Update 2: I need to use LYNC again. Pidgin from the default Ubuntu repos does indeed now appear to work just fine with a custom user agent. In addition, I’ve also had some success with “WYNC” which works pretty well but has a few minor issues of it’s own.

Recently at work we have started using Lync internally. Whilst this is great for the Windows and Mac users among us, not so much for those of us running on Linux.

However, it turns out that it is possible to get basic Lync support working quite easily. I can see people, talk to people, people can talk to me – I can send files to people, but people can’t send file to me. I’ve not tried any video/voice stuff but I suspect it doesn’t work.

It’s done using “sipe” – basically an open source implementation of the Extended SIP/SIMPLE protocol Lync uses for chat.

Update iptables on Endian Community Firewall (EFW) 2.4.0

Sun, 12 Feb 2012 09:37:18 +0000

Compiling ip6tables on Endian Community Firewall (EFW) 2.4.0

Unfortunately the version of ip6tables available at the time of fedora core 3 doesn’t support the ‘state’ or ‘comment’ modules for use with firewall rules. So in order to get these, I decided to compile iptables 1.4.12.2 for Endian.

IPv6 with Endian Community Firewall (EFW) 2.4.0

Sun, 12 Feb 2012 09:15:56 +0000

First post in over a year! Oops.

For a while now, my home ADSL provider (EntaNET) has provided me with an IPv6 allocation, but I’ve never really used it (Its been on my to-do list for some time) primarily due to the fact that it is unsupported by Endian which I use for my home router/firewall.

However the other day after being asked about IPv6 at my day job, I decided I wanted to get this working, and decided to document it here in case it can assist anyone else in future. (I also finally got round to completing the Hurricane Electric IPv6 Certification up to sage level)

There’s a few things worth noting before we continue here.

I use a Draytek Vigor 120 for my adsl modem - this is a PPPoA to PPPoE bridge. This means that my Endian box uses PPPoE to get its Internet connection, and directly receives an IPv4 address via the PPP session. There is no “PPP Half-Bridge” tricks here (such as where Modem does authentication, then DHCPs the address to Endian).
Due to Endian lacking support for IPv6 you will need to use SSH to configure this, and any Endian upgrades will probably reverse a fair chunk of it. (Also, some reconfigurations may also undo things) - so with this in mind the rest of this guide assumes you are familiar with SSH and have successfully logged in as root to the Endian box (SSH can be enabled under the “System” section and “SSH Access”).
Due to previous requirements, my Endian server is not “pure” in that I have additional packages installed that made this easier. Notably, a complete build environment. This won’t be needed here.
This was all done without writing it down, so this documentation is based on my recollection and attempts at replicating various parts on a VirtualBox VM (which can’t do PPPoE…). If I’ve missed anything, please let me know in the comments.
This was done with EFW 2.4.0 and may not work in the latest 2.5.1 version.
I have only had this running for a few days, so there may be some unforeseen issues with this.

With this in mind, we continue to the actual important stuff!

Happy New Year

Sat, 01 Jan 2011 00:50:03 +0000

A tale of two monitors

Sun, 21 Nov 2010 19:20:56 +0000

So, a while back (just under 3 years ago) I obtained 2 of Hyundai’s W240D monitors. These monitors had (I believe) PVA panels and worked fine for most of their life so far.

A while back they both developed a problem, as evident in the video below:

So, as they were still under warranty I contacted Hyundai/RepairTech and arranged for these to be repaired. Hyundai sent the parts to RepairTech, who collected the units, repaired them and sent them back.

Greasemonkey script for hp.com forums

Fri, 24 Sep 2010 12:23:51 +0000

If you’ve ever visited the hp.com forums you’ll know that any links in the post get enclosed by a call to “javascript:openExternal(’’)” in the href rather than doing it properly in onClick. Amongst other things, this breaks the ability to middle click to open links in new tabs.

This finally annoyed me enough today and as a result, I now use the following greasemonkey script:

// ==UserScript==
// @name           Stupid HP.COM Links
// @namespace      http://shanemcc.co.uk/
// @include        *hp.com*
// ==/UserScript==

var a = document.getElementsByTagName("A");
for (var i = 0; i &lt; a.length; i++){
	var href = a[i].href;
	href = href.replace(/javascript:openExternal\('([^']+)'\)/i, '$1');

	a[i].href = href;
}

This will make the links no longer have the call to openExternal around them, and thus make them middle-click friendly.

Ubuntu on HP Compaq Mini 311c-1030SA

Tue, 10 Aug 2010 01:41:53 +0000

I recently purchased a HP Compaq Mini 311c-1030SA with Nvidia ION and built in 3G, unfortunately the 3G card is a “UN2400” which isn’t supported right out of the box as it requires proprietary firmware.

This post is mostly notes for myself on getting the UN2400 3G card inside it working enough to use.

This post assumes that the netbook is running Ubuntu maverick ~~(which is currently in alpha but seems to work just fine)~~ as it has gobi_loader as a package and a kernel which supports it.

Sending SMS with a Huawei E220

Fri, 06 Aug 2010 09:15:09 +0000

Today I decided to play with an old Huawei E220 I have lying around.

After getting it setup and recognised in Linux by following the first 5 steps from http://ubuntuforums.org/showthread.php?p=3656717

After this, restarting udev (restart udev) and replugging the device makes it ready to use.

WordPress

Sun, 27 Jun 2010 19:13:10 +0000

I have recently decided to switch this site to use wordpress rather than the custom code that was here before.

This will allow me to edit/post to the site using my phone, which will probably allow me to update it more often.

I am working on migrating everything from the old site to the new site and trying to find a theme I like. Hopefully I will have everything working soon. Should anything be missing that you would like brought back leave a comment and let me know and I’ll see what I can do.

Unfortunately for anyone reading this using RSS, I’m afraid the recent posts will duplicate themselves as the IDs in the RSS feed will have changed.

Ident Server

Wed, 03 Mar 2010 22:53:00 +0000

I recently encountered a problem on a server that I manage where by the oidentd server didn’t seem to be working.

Manual tests worked, but connecting to IRC Servers didn’t.

I tried switching oidentd with ident2 and the same problem.

After switching back, and a bit of debugging later it appeared that the problem was that the IRC Servers were expecting spaces in the ident reply, whereas oidentd wasn’t giving them.

I then quickly threw together an xinet.d-powered ident server with support for spoofing.

GitWeb Hacking.

Fri, 27 Mar 2009 02:52:27 +0000

Recently I setup gitweb on one of my servers to allow a web-based frontend to any git projects which the users of the server place in their ~/git/ directory.

After playing about with it, I noticed that it allowed for placing a README.html file in the git config directory to allow extra info to be shown on the summary view, managed to get it to pull the README.html file from the actual repository itself, and not the config directory, thus allowing the README.html to be versioned along with everything else, and not require the user to edit it on the server, but rather just edit it locally and push it.

New Phone – T-Mobile G1.

Thu, 11 Dec 2008 01:56:12 +0000

Recently I acquired a T-Mobile G1 to replace my old T-Mobile MDA Vario 2 (HTC Hermes).

All I can say about this phone is that it is quite awesome. I no longer need to run an exchange server to keep my contacts/calendar synced somewhere as the G1 syncs everything to Google Mail/Calendar.

Its a really good phone and I recommend it to anyone who is thinking of getting a new phone, the integration with Google is especially useful, and the full-html (including CSS and javascript) is very nice.

JDesktopPane Replacement

Sat, 09 Aug 2008 03:46:49 +0000

As as I mentioned before I’ve been recently converting an old project to Java.

This old project was an MDI application, and when creating the UI for the conversion, I found the default JDesktopPane to be rather crappy. Google revealed others thought the same, one of the results that turned up was: http://www.javaworld.com/javaworld/jw-05-2001/jw-0525-mdi.html

So, I created DFDesktopPane based on this code, with some extra changes:

Frames can’t end up with a negative x/y
Respond to resize events of the JViewport parent
Iconified icons move themselves to remain inside the desktop at all times.
Handles maximised frames correctly (desktop doesn’t scroll, option to hide/remove titlebar)

My modified JDesktopPane can be found as here part of my dflibs Google code project.

Other useful things can be found here, take a look and leave any feedback either here or on the project issue tracker

GMail – apply labels to email from group members

Tue, 05 Aug 2008 06:03:27 +0000

NOTE: The information in this article has been superceeded by this one.

As Noted by Chris recently on IRC, Google Mail lacks a feature in its ability to automatically label/filter messages - you can’t do it based on emails from people in a contact group, short of adding a filter with all their email address on it.

At the time it was mentioned this didn’t affect me, however later when I got round to adding loads of labels/filters in gmail (yay for, nicely coloured inbox!) to nicely separate things for me I also ran into this problem, so came up with the following python script that does it for me.

It checks messages, sees if the sender is in the contacts, then checks each group to see if there is a label with that group name that is not already set, then checks to see if the contact is in the group, and finally sets the label if everything matches up.

MD5

Mon, 28 Jul 2008 02:55:09 +0000

I was recently looking at converting an old application from VB6 to Java that used MD5 in its output files as hashes for validation.

The first thing I did was to make a java class that read in the file and checked the hashes, I tried it on a few files and it worked fine, then I found a file that it failed on.

Now, this app wrote all the files using the exact same function, so it seemed odd that 1 of them wouldn’t parse and the rest would.

When I looked at the file closer, I found that this one contained some symbols in the output that the others didn’t - I eventually figured out that the symbol that was causing the problem was the pound sign (£).

Without going into too much detail, this presented a major problem, the string in question was used as part of the password validation for the app (the output files are encrypted using the password as a key), and the java code was getting different results than the old VB6 code, and was unable to decode the file as a result.

So, this sparked my curiosity a bit, the VB6 code I was using wasn’t a built in, it was code I’d gotten elsewhere and used, so I assumed it was faulty code (not that this helped me much, as I needed to get the exact same output, but ignoring that).

“Dr. Horrible’s Sing-Along Blog – Act 3” a Dissapointment

Sun, 20 Jul 2008 14:26:23 +0000

Despite all the “This was amazing” “fantastic” reviews I seem to find for this everywhere, I found myself disappointed after watching it.

The first 2 acts were funny and very rewatchable, there was gold bars that became gold liquid, the “Bad Horse” letters and phone calls, Captain Hammer and his “Hammer”, they made me laugh and alongside the humour was a kick ass sound track (my personal favourite being “Its a brand new day”).

Act 3 on the other hand was a complete change in direction, the songs weren’t as good, it wasn’t really all that funny (Infact I think the only bit I laughed at was him stopping his song to correct the spelling of his name), it suddenly became all serious. All in all I found it a rather disappointing, and somewhat obvious, end and a let down to an otherwise awesome show.

Despite this I’m still going to buy the DVD (and hopefully the OST if one comes out), as I approve of the idea of a web-streamed show (I think there was a push to get a firefly season 2 done in this way at one point) and would like to see more of them.

Yakuake on OS X – Almost

Thu, 17 Jul 2008 07:18:03 +0000

For a while now (pretty much since I’ve been using Linux) I’ve been using yakuake, and I’ve been looking for something similar for OS X (at the moment I tend to ssh from my desktop to my OS X machine to do anything console like).

I found visor which wraps terminal.app, but overall this is a poor replacement primarily for the lack of tab support (I use tabs a lot in yakuake, at the moment on my desktop I have ~15 open)

I tried to get yakuake working on OS X a while ago and failed (it wouldn’t compile) so gave up, recently however I decided to try again and have made better progress.

This is what I did, its probably not the best way of doing it (for example, the initial 3GB download could probably be reduced).

Virtualbox Bridging

Wed, 11 Jun 2008 10:56:17 +0000

Edit: This is now pretty much unneeded, the new version of VirtualBox seems to handle this all nicely on its own.

As I mentioned in my last post, One of the useful advantages of the network boot setup is that I can use it to quickly install virtual machines.

Now a few things:

My Desktop is a lot more powerful than my server, so I run the virtual machines on it.
I use virtualbox rather than vmware.
All the network boot stuff is on my server not my desktop (obviously)

So in order to allow this, virtualbox needed to be setup to bridge to my existing adapter, this was quite straight forward, pretty much exactly as the manual said.

PXE Goodness

Tue, 10 Jun 2008 01:52:22 +0000

So as you may or may not know from time to time I have the joy of fixing computers for various people. A lot of these fixes result in a reinstall of windows and away.

This is a rather easy enough job, I have a KVM switch that I attach to the machine, pop a windows CD in (I used to have an unattended CD but don’t any more), answer a few questions and then occasionally switch the KVM over to see if the install died or so.

Now this is all well and good except for 2 problems:

It means I need to keep (or remember to bring) windows CDs at home (where I do most of my jobs)
I recently had a machine to fix that had a non-working CD Drive

Now, the first one isn’t so much of a problem, but the second one was.

OpenID

Sun, 02 Mar 2008 21:13:31 +0000

Recently I finally finished adding support for AuthGate to be an OpenID Provider.

This prompted Chris to develop Poidsy, which I have also implemented into AuthGate. This allows people to login to AuthGate (and thus also here, and other AuthGate powered sites) using OpenID instead of having to register with AuthGate itself.

Email Woes

Sat, 18 Aug 2007 01:10:19 +0000

On a daily basis, I get around 800 emails to my email accounts, of which most of it is spam.

Now as good as thunderbird is at detecting spam, even it fails at a lot of the spam I receive, leaving me with around 100-200 spam per day that gets into my inbox.

I’ve dealt with and accepted this for over a year now, before a discussion on IRC made me decide to do something about it. (When I say discussion, I mean Chris pasted one line showing how good the UTD-Hosting mail server was at preventing junk getting to him)

So, I recently (Today and yesterday) started prodding my postfix config to help with the problem.

More DMDirc

Mon, 11 Jun 2007 03:32:24 +0000

DMDirc 0.4 has now been released :)

DMDirc

Sat, 03 Mar 2007 06:40:42 +0000

I (Along with Chris ‘MD87’ Smith and Greg ‘Greboid’ Holmes have recently started working on DMDirc (again, only this time its in Java to help towards the original aim of the project to be a decent, truly cross platform IRC Client)

Its coming along quite well :)

You can track the progress, (and download the current release which is 0.1) at the Google Code Project or the Project Website

Lemme know what you think.

Linux Desktop

Wed, 31 Jan 2007 00:48:55 +0000

With the release of windows vista, comes the start-of-the-end for Windows XP. with its EOL (End-Of-Life) date now set at January 30th 2008 (that’s less than a year away), people (by people I mean windows users) who are unable to upgrade to vista (due to Lack of computing power or so) or don’t want it (its crap, proprietary, riddled with DRM and probably bugs - Microsoft are already producing SP1!) need to start looking for alternatives, unless they want to stay using an unsupported (this means no more bug/security fixes) Operating System.

Imo, The best alternative is some derivative of Linux. (Although there is others such as MacOS x86 although its not supported on non-mac hardware, FreeBSD but I don’t think its desktop oriented, and others such as beOS or so)

As of Saturday 27/1/07 I have started using KUbuntu Linux as the main OS on my desktop, as a trial to see how well I can get by without my “trusty” windows installation.

Delphi/FreePascal MySQL.pas

Sat, 30 Dec 2006 14:00:23 +0000

Due to a recent need in a project of mine for mySQL access from delphi/freepascal I have adapted the version of mysql.pas from http://www.fichtner.net/delphi/mysql.delphi.phtml to load both libmysql.dll (on svn) and libmysqlclient.so (Usually located in /usr/lib/mysql/).

I also created a wrapper class for it (TSQL in SQL.pas)

Downloads can be found here: [http://blog.dataforce.org.uk/viewcvs/misc/MySQL/]

Any queries/questions should be left in the comments.

(This has been tested, and compiled on Freepascal on Linux (1.9.8) and windows (2.0.0) and in Delphi (6/7/Turbo) on windows.)

Captcha Woes

Fri, 22 Dec 2006 03:22:22 +0000

Even the most complex captchas can be bypassed, if not tested thoroughly enough.

Some time ago, I implemented a captcha on my comments form for news posts, to stop spam bots. My captcha is quite complicated, and sometimes generates images that even humans strugle on, yet for some reason I managed to get 10788 spam comments!

I immediately tested my captcha - tried with no value, a wrong value, a right value, only the right one worked.

Then I tried in a different window, open 2 tabs to the same comment, submit the 2nd one with no data and get told the correct captcha, then try using that on the first window, this also didn’t work - however, it told me the correct captcha was “”, upon hitting back and trying with no captcha, it worked.

Turns out I had forgot to make "" an invalid captcha when testing if the values were correct, seeing as when you submit the captcha, it clears the captcha session, a blank value WAS indeed the same value that was stored in the session! The spam bots were just not accepting the session, and thus had a blank captcha - and by them not filling in the captcha, and not accepting the session, they were able to submit comments. This has now been fixed, and a blank captcha will now give an error of “Captcha Timeout”.

Fortunately, 3 Simple SQL Queries pruned the lot :) 1 of which pruned 7900+. Silly bots being so similar!