Going to Tokyo (A very unplanned holiday)

frau-klein is going to Tokyo for work and I'm tagging along. It's a badly planned (well unplanned) holiday, considering we were going to go to Dresden originally.

Some useful links for preparing on a short notice for i.e. hotels and plane tickets

Some guides/reading material that I ended up getting

  • Lonely Planet - Toyko Encounter
  • AA Citypack guide and foldout map

Now that I have a plane ticket and a hotel, I need an itinery for the trip.

Posted
Backing up a Flickr account

It just occured to me in the last few days that I should probably do a complete backup of my flickr account. I came across these apps which seem useful but not all of them are cross platform apps.

Migratr seems to be the most useful as it tries to grab album/set information, it appears to be a windows only application.

Planning my trip to Tokyo
  • Thursday - be jetlagged, going to arrive in the afternoon, so I will probably just wander around the local area at night.
  • Friday - Shinjuku and Shibuya - need to kill some time on my own.
  • Saturday - Tsukiji Fishmarket, Ginza, Imperial Gardens then to Shinjuku for dinner (as it suggests in the Lonely planet)
  • Sunday - Harajuku & Aoyama - surrounding areas, which ever day it is when all the dressed up action is going to be.
  • Monday - Ueno - museums around Ueno, Parks and Temples.
  • Tuesday - Odaiba - to see all those crazy buildings.
  • Wednesday - Akasaka & Roppongi - surrounding area, Roppongi Hills.
  • Thursday - Ghibli museum at Mitaka and surrounding areas.
  • Friday - Akihabara, Electric town and surrounding areas.
Migrating my development folder from linux to OSX

One of the things that I don't like too much about OSX is the case-insensitive and type sensitive defaults. It's certainly nice and easy to use for most people, but if you are wanting to write shell scripts that you want to run elsewhere it can be a bit of a pain. I prefer to have case and type sensitive file system for development. "Disk Utility" is pretty handy for creating disk images with the properties that I need.

Patching 'fsl_sub' for SLURM

I recently patched 'fsl_sub', which is apart of fsl, to allow it to batch submit tasks to our computer clusters. I'd submit the patch to the authors of the software but I couldn't figure out where to send the patch to. I really didn't need to sign up to another mailing list. So here it is.

--- fsl_sub.orig        2010-06-11 13:03:35.279077000 +0100
+++ fsl_sub     2010-06-11 13:04:33.409821000 +0100
@@ -100,6 +100,10 @@
     fi
 fi

+if [ "x$SLURM_JOB_ID" != "x" ] ; then
+       METHOD=SLURM
+fi
+

 ###########################################################################
 # The following auto-decides what cluster queue to use. The calling
@@ -123,6 +127,11 @@
        queue=verylong.q
     fi
     #echo "Estimated time was $1 mins: queue name is $queue"
+
+    # if slurm environment is detected use the compute partition, change this to suit
+    if [ $METHOD = SLURM ] ; then
+           queue=compute
+    fi
 }


@@ -200,7 +209,7 @@
 # change. It also sets up the basic emailing control.
 ###########################################################################

-queue=long.q
+queue=compute
 mailto=`whoami`@fmrib.ox.ac.uk
 MailOpts="n"

@@ -364,6 +373,40 @@
        ;;

 ###########################################################################
+# SLURM method
+# this is a very naive way of doing things, its just to simply fire off all
+# the tasks individually to the resource manager
+###########################################################################
+
+       SLURM)
+               if [ $verbose -eq 1 ] ; then
+                       echo "Starting Slurm submissions..." >&2
+               fi
+               _SRMRAND=$RANDOM
+               _SRMNAME=$JobName$SRMRAND
+               echo "========================" >> sbatch.log
+               echo "= Starting submissions =" >> sbatch.log
+               echo "========================" >> sbatch.log
+               date >> sbatch.log
+while read line
+do
+        if [ "x$line" != "x" ] ; then
+sbatch -J $_SRMNAME -o "slurm-log-$_SRMNAME-%j-%N.out" -t 01:00:00 -p $queue -n 1 <<EOF
+#!/bin/sh
+echo 
+echo $SLURM_JOB_NAME
+echo $SLURM_JOB_ID
+echo $SLURM_JOB_NODELIST
+echo
+date
+echo
+$line
+EOF
+        fi
+done < $taskfile >> sbatch.log 2>&1
+       ;;
+
+###########################################################################
 # Don't change the following - this runs the commands directly if a
 # cluster is not being used.
 ###########################################################################

It's not the prettiest thing around, its quick and dirty and it spams the queue system pretty good. It's cut down a job which did take 5-6days to about 8-9hrs. That is it used to run on 1cpu, now it runs on 24-32cpus at a time.

Setting up a FreeRADIUS server on Scientificlinux

The following notes are a summary of HOWTO: Incremental Setup of FreeRADIUS Server for EAP Authentications

Prerequisites

  • Scientificlinux 5.x
  • A working LDAP directory - I'm pointing to a LDAP directory that I already have on our network.
  • FreeRADIUS - I just installed this from the SL5.x repositories

Testing

Step 1 - test on the localhost

Debugging the server

/usr/sbin/radiusd -X

Testing the setup, first add this to your /etc/raddb/users file

dummyuser User-Password := "dummypasswd"

Note that I am using version 1.1.x, in newer FreeRADIUS versions you will need to use Cleartext-Password

Then do

radtest dummyuser dummypasswd 127.0.0.1 1812 testing123

The testing123 part is a default shared secret password. The above steps are just used to see if the basic setup works or not.

Step 2 - test from a remote host

Now edit /etc/raddb/clients.conf and add in your host

client 226.134.191.112 {
secret = testing123
shortname = 226.134.191.112
}

Restart the radiusd process then do this from 226.134.191.112

radtest dummyuser dummypasswd 226.134.191.113 1812 testing123

where 226.134.191.113 is the radius server. It should complete sucessfully

Step 3 - EAP (with TLS) and PEAP configuration

Prerequisites

  • SSL keys and certs for the radius server, see the eap.conf for comments.
  • Step 1 and Step 2 works.

at minimum you will want to have this type of configuration in your eap.conf file.

eap {
    default_eap_type = md5
    timer_expire     = 60
    ignore_unknown_eap_types = no
    cisco_accounting_username_bug = no
    md5 {
    }
    leap {
    }
    gtc {
        auth_type = PAP
    }
    tls {
        private_key_password = whatever
        private_key_file = ${raddbdir}/certs/cert-srv.pem
        certificate_file = ${raddbdir}/certs/cert-srv.pem
        CA_file = ${raddbdir}/certs/demoCA/cacert.pem
        dh_file = ${raddbdir}/certs/dh
        random_file = ${raddbdir}/certs/random
    }
    peap {
        default_eap_type = mschapv2
    }
    mschapv2 {
    }
}

For testing you may want to just comment out the group=radiusd and user=radiusd options in radiusd.conf

Step 4 - test with WPA Supplicant or similar tool

I used the eapol_test program from wpa_supplicant with this configuration file

network={
    eap=PEAP
    eapol_flags=0
    key_mgmt=IEEE8021X
    identity="dummyuser"
    password="dummypasswd"
    #ca_cert="/home/jtang/ca.crt"
    phase2="auth=MSCHAPV2"
    anonymous_identity="anonymous"
}

run like this

eapol_test -c radeap.conf  -a226.134.191.113 -p1812 -stesting123 -r1

eapol_test was built with

wget http://hostap.epitest.fi/releases/wpa_supplicant-0.5.10.tar.gz
tar xvf wpa_supplicant-0.5.10.tar.gz
cd wpa_supplicant-0.5.10/
cp defconfig .config
make eapol_test

Making FreeRADIUS look up LDAP

Add this to your /etc/raddb/dictionary file

VALUE           Auth-Type               Local                   0
VALUE           Auth-Type               System                  1
VALUE           Auth-Type               SecurID                 2
VALUE           Auth-Type               Crypt-Local             3
VALUE           Auth-Type               Reject                  4
VALUE           Auth-Type               LDAP                    5

In eap.conf in the PEAP section set

default_eap_type = peap

In radiusd.conf in the LDAP section set at least this

server = "ldap.tchpc.tcd.ie"
identity = "cn=admin,dc=tchpc,dc=tcd,dc=ie"
password = myadminpassword
basedn = "ou=people,dc=tchpc,dc=tcd,dc=ie"
#filter = "(posixAccount)(uid=%u))"
filter = "(uid=%u)"
# access_attr = "dialupAccess"
password_attribute = userPassword
set_auth_type = yes

then in the authorize section uncomment the ldap option and comment out the files module.

Macports - Portfile - mr - a Multiple Repository management tool

Since I now have a mac for work, I've been moving my old work files and creating some packages for macports. Here's the first of the lot that I use regularly, it's for mr - a Multiple Repository management tool

# -*- coding: utf-8; mode: tcl; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- 
# vim:fenc=utf-8:ft=tcl:et:sw=4:ts=4:sts=4
# $Id$
PortSystem      1.0
name            mr
version         0.48
categories      devel
maintainers     tchpc.tcd.ie:jtang
description     a Multiple Repository management tool
homepage        http://kitenet.net/~joey/code/mr/
master_sites    http://ftp.de.debian.org/debian/pool/main/m/mr/
distfiles       mr_${version}${extract.suffix}
distname        mr
depends_lib     port:perl5.8

use_configure       no

checksums           md5     3d3770f7d4b1bb4f92cdfa9c57090dfe \
                    sha1    bdaac495252c562f708e9ddda47b3f54424eb07a \
                    rmd160  ee55d5c9ec37719c5c56d98ed314d4693dd0e102


build.args    PERL_PATH="${prefix}/bin/perl" DESTDIR=${destroot} prefix=${prefix}
destroot.destdir  DESTDIR=${destroot} prefix=${prefix}

build.target      build

configure {}

destroot {
     xinstall -m 755 -d ${destroot}${prefix}/bin
     xinstall -m 755 ${worksrcpath}/mr ${destroot}${prefix}/bin
     xinstall -m 755 ${worksrcpath}/webcheckout ${destroot}${prefix}/bin

     xinstall -m 755 -d ${destroot}${prefix}/share/man/man1
     xinstall -m 755 ${worksrcpath}/mr.1 ${destroot}${prefix}/share/man/man1
     xinstall -m 755 ${worksrcpath}/webcheckout.1 ${destroot}${prefix}/share/man/man1
}

dwm or pristine-tar is on my hit list next.

Bookmark and Share