Will Dowling's blog

  1. New website

    Mon 01 April 2013

    Another thing that has been on my shitlist for quite a while now - is this site.

    The original system for this website was custom PERL/PHP loading articles straight from a MySQL database - I would use phpmyadmin as an editor of sorts and all was good, for a time.

    In 2005 I wanted more flexibility on what kind of entries I could post. I wanted to move to a format including the main article content, as well as a micro-blogging feed. This lasted for a few years until around 2007-8 when social media started being the outlet for this kind of dribble.

    But the move to Drupal was my main choice, it was a compromise between getting things done and having some control over the content. That and I never quite figured out how to get Xaraya to do what I wanted.

    One of the things that really bothered me though is that themeing for Drupal is a pain in the butt (ain't nobody got time for that). Back in the day my website used to be both designed and built end-to-end by yours truly. Moving to Drupal had the unintended side-effect of making this too-hard(tm) to do and carry over between upgrades.

    Add on top the difficulty of reformatting content and managing comment spam and I'm finally shitted off to do something about it...

    Without further ado...

    Tada! Here's the new site... pretty, huh?

    The site itself is statically generated HTML, driven by a nifty piece of software called Pelican.

    Pelican takes a path full of files and converts them from their source format (in this case, Markdown) and generates static html pages from them using templates.

    I've been looking at a setup like this for a while, as I've been keen to get my website content in something presentation-agnostic like docbook/markdown for quite a while now. One of my mates has also done this with his website recently, with great success.

    Out of the box, Pelican hits 90% of what I want from it.. but to convert the site over I had to write some modules.

    URL Redirection

    My old site used a horrible URL scheme, generating content at URIs like: /node/###. The way I have the new site uses a much more sensible scheme of /YYYY/SHORTNAME.

    That being said, it would be nice to make sure any links on social media or elsewhere pointing to my site (hah), continue to work. Something that Pelican doesn't support.

    So the first module hooks each article and looks for a metadata tag oldurl. If present, it will add it to the site's global .htaccess file.

    Page media

    One thing that seems particularly broken is page media, ie: images or downloads attached to a page. In Pelican's standard model, these files need to be stored in a special folder in your content root called static, where they will be copied into the final output folder. Files copied from this location can be linked to using the standard |filename|blah.jpg tag in Pelican's template engine.

    This doesn't make a lot of sense to me, as I'm using my own structure in my content hierachy:

    - content/
         - YYYY/ 
              - MMDD_SHORT/
                   - *.md
                   - (other files)
    

    I'd like to keep the files for each post alongside the post itself. And in case I add new source formats or whatnot later, I don't want to blind copy things either. And I want the |filename|blah.jpg syntax to work too, so Pelican needs to know that there's a file with that name being copied.

    This ended up being fairly tricky, at first I wrote a generator plugin to scan the content folders and find any file that wouldn't be picked up by other generators. However this got tricky, because without actually parsing the content alongside it, I wouldn't know where to put the file (eg: ipv6.png on my IPv6 Article should be output to /2013/IPv6/ipv6.png).

    Itch: Scratched.

    And that's about it. I'm reasonably happy with the way it's turned out. The sites content is now tracked in a git repository, and looks a hell of a lot better (even if the styling is blatently ripped from svbtle).

    I'll try and remember to push the two python modules above onto my github, as well as the changes I'm contributing back to core also.

    The logo is likely to change, as there's an ongoing discussion about how poor it looks currently - when that's finalised I'll generate some matching busines cards and move onto the next project in the pipeline!

  2. IPv6

    Fri 22 March 2013

    image

    One of the big things on my todo list for a while now has been to get as many services on my VPS running on IPv6 as possible.

    Getting up to speed on v6 is important to me for both personal and professional reasons - however, an important part of testing this is having somewhere to test from.

    So for the first part of my challenge, I needed to get IPv6 working at home. Fortunately my ISP has IPv6 available via a tunnel-broker service available two hops away, so it's almost as good as having it native and far superior to tunneling to HE.net for their otherwise excellent v6 service.

    Having recently purchased an awesome Juniper SRX110H-VA for home, I've got a great platform for getting IPv6 up and running.

    Here's a somewhat cutdown configuration with all the bits needed to get it working.

    interfaces {
        ...
        ip-0/0/0 {
            unit 0 {
                description "iiNet 6RD";
                tunnel {
                    source 203.59.XXX.XX;
                    destination XXX.X.XXX.XX;
                }
                family inet6 {
                    address 2001:4479:cb3b:XXXX::1/64;
                }
            }
        }
        pp0 {
            unit 0 {
                ...
                description "ISP IPv4 Interface";
                family inet {
                    filter {
                        input fix-6in4;
                    }
                    ...
                }
            }
        }
        vlan {
            unit 1 {
                description "Home clients";
                family inet {
                    address 10.0.0.254/24;
                }
                family inet6 {
                    address 2001:4479:cb3b:XXXX::2/64;
                }
            }
        }
    }
    security {
        forwarding-options {
            family {
                inet6 {
                    mode flow-based;
                }
            }
        }
        zones {
            security-zone world {
                interfaces {
                    ip-0/0/0.0 {
                        host-inbound-traffic {
                            system-services {
                                ssh;
                                ping;
                            }
                        }
                    }
                }
                ...
            }
        }
    }
    firewall {
        family inet {
            filter fix-6in4 {
                term term1 {
                    from {
                        destination-address {
                            203.59.XXX.XX/32;
                        }
                        protocol 41;
                    }
                    then packet-mode;
                }
                term term2 {
                    then accept;
                }
            }
        }
    }
    routing-options {
        rib inet6.0 {
            static {
                route ::/0 next-hop ip-0/0/0.0;
            }
        }
    }
    protocols {
        router-advertisement {
            interface vlan.1 {
                prefix 2001:4479:cb3b:XXXX::/64;
            }
        }
    }
    

    OK, maybe that wasn't such a quick snippet, and you'll want to be putting some zone rules to control what traffic can ingress onto home network - as all machines will autoconfigure their own public IPv6 address. Death to NAT, yeeeeeeah baby.

    Anyway, after a lot of messing wondering why things weren't working - I realised my Ubiquiti WiFi bridges weren't letting v6 traffic across the links. This is easily fixed by turning on it's WDS which makes it act like a proper Layer 2 bridge.

    So now I can IPv6 at home, and have been able to validate my configuration on my VPS. As of the time of writing, the following services are exposed and configured for IPv6...

    • DNS
    • SMTP & Submission
    • HTTP/SSL
    • IMAP/TLS
    • SSH
    • XMPP

    Which is everything! Huzzah. Over the last few days I've been tracking and it looks like between 10-50% of my traffic is being delivered via IPv6, which isn't too suprising.

    I suspect that my home connection will be doing somewhere near 80% IPv6 traffic, though I've yet to graph that. I also need to get clients performing native IPv6 queries - currently they're getting their DNS settings via DHCPv4.

    And that's one off the list... I've learnt a bit about v6, but excepting my WiFi issues it's not as scary as everyone makes out (either that, or I've got a bit of a dunning-krueger thing going on, which wouldn't suprise me).

  3. Half-Baked Sunday: Left 4 Dead Map

    Sun 24 February 2013

    image

    I'm quite notorious for picking up an idea, half-completing it and putting it back down again.

    It's pretty frustrating for me, let alone other people - and probably one of the reasons I don't share a lot of what I'm working on; but I still think it's a great way to scratch an itch and move on.

    Anyway in the interest of getting out there a bit more, I'll try to share things I come across in my project boneyard...

    For the first Half-Baked Sunday: a Left 4 Dead map I was working on in 2010... The map is of the iconic Perth Concert Hall here in Perth and is based on some drawings for it.

    The idea was to put together a quick Left 4 Dead campaign starting in the old tunnels under St Georges terrace, breaking through the main auditorium and out to escape on a boat on the river.

    I'd still like to complete this - if only because I had such a good start to it. However trying to draw up a map in Hammer based off drawings is a major pain in the butt, and was in fact the motivation to write Jocular.

    Jocular was a tool to take a set of 2d lines and extrude them into 3d walls - with control points in the map to handle alignment/scaling of multiple floors (ie: stitching extruded segments together).

    Whilst Jocular was working as a tool, at some point it scope creeped to include a GUI editor as well - which then came down to a decision on what vector and 3d library to use (Jocular was written in .NET).

    It got a bit too hard and put on the backburner - if I approach this again I will consider writing it in Python as a Blender3D import tool and use the features available there.

    ... And that's it for the first Half-Baked Sunday, hopefully I'll be able to share some of my stalled projects in future weeks :)

    image image image

  4. 2012 Recap

    Mon 04 February 2013

    image

    Whilst I'm usually loathe to use the new year as a milestone on which to measure things - 2013 is promising to be a good year once I get some things sorted out.

    But first, lets cap off some of the highlights of 2012 since my last posts...

    image

    Personal Scrum - I've been giving "Personal" Scrum a go for tasking. The image is a cumulative total of tasks (green is complete!). It helps keep on top of me and when I'm not motivated to work on projects (I'm still making some kind of progress - huzzah!)

    image

    USA, Canada, Japan! - I was lucky enough to take a month off work and travel around the USA (mostly) for a month, as well as a little bit of Japan and Canada. We met up with Hurricane Sandy though and missed our epic week in NYE - so we'll just have to go again :)

    image image

    Projects - As usual, I'm dissapointed how little progress I've made on my forever projects](http://www.dev.gd/20130122-the-joys-of-having-a-forever-project.html), but maybe thats okay this year - I've played with some new things and done some forays into things outside my comfort zone.

    Some things I've worked on in 2012 - none complete but all explored a bit to ever so slightly scratch the itch:

    • das player - a continuous streaming music player from multiple cloud music sources.
    • jocular - a tool for importing, editing and extruding and stitching 2d geometries into 3d space.
    • super secret project - using general purpose graph database neo4j, creating richer data for a popular dataset, for use with other tools.

    As well as some other bits and pieces exploring some other areas.

    image

    Startup Weekend - Visited in a previous post, the inaugural Startup Weekend was a fantastic opportunity to network and explore the lean process in an intense 54 hour period.

  5. Startup Weekend Perth

    Sun 09 September 2012

    image

    Startup Weekend is a worldwide event that allows business people, designers and developers to come together and explore ideas for startups.

    I was lucky enough to take part in Perth's first Startup Weekend event this weekend.

    Somewhat unsurprisingly (depending on your point of view), the inaugural event sold out rather quickly, making a new Australian record (I believe).

    Whilst writing this, we're in the final hours of the weekend and everyone is getting their pitch ready and putting the final polish on their ideas. There are some really great ideas here, and people passionate about them.

    There are a few special things that have caught my eye and are worth a mention:

    • Gainspiration - The pitch for this is a resource for introverted people to explore and build their social skills in a supportive, collaborative environment. This is by far the least conceited and most worthwhile pitch I saw. The guys seems to be struggling with their business model and execution, but it's something I can really get behind.
    • Threadster - This is probably the clearest vision that was pitched, and seems to have a strong team behind it executing the vision.

    I'll call it now, but I think the winner will be Threadster, though there are a few guys with their heads down who might have some suprises.

    Whilst the event seems geared towards online startups and coming up with ideas, I hope the organisers mix up the blend of skillsets in favour of business/designers. This might be slightly biased on my experience....

    ... about which I've got a lot to write, but it will take a while to get down - all things considered though the weekend is definately worthwhile and I look forward to attending the next event and lending my support to the team if needed.

continue   →