Twitter Weekly Updates for 2011-02-06

  • Upgraded my iphone4 to iOS 4.2.1 and re-jailbroken; exceedingly quick and easy process – thanks to greenpois0n etc. #
  • oooh, looks like Squeeze is very close… good luck #debian #
  • “@GeneHunt: My girlfriend says I'm ignorant. ….. I've absolutely no idea what she's talking about.” #
  • What sort of idiot walks around in the dark & head-butts a corner. Oh wait … #PleaseDoNotBruise http://twitpic.com/3wjig4 #
  • Nearly midnight is perhaps the wrong time to think about upgrading and re-jailbreaking my iphone. Until tomorrow…. #
  • http://www.youtube.com/watch?v=oJagxe-Gvpw 'World's worst hacker'…. #
  • Hopefully I'll manage to do the Aberystwyth -> Kington bike ride this year at least, and maybe next year London -> Paris or something abroad #
  • 10th anniversary Dyfi Enduro registration opens on 10/02/2011. Almost regretting being mtb-less now http://t.co/vA4QJpQ #
  • Interesting experimentation with 4square's API this evening #PHP #4sq #
  • Wish 'normal' people would stop using the disabled / baby change toilet. Lazy … Grr. Grr. #
  • Year of the Rabbit. I bet Ann Summers will make full use of this for marketing …. #
  • Yes apple, I'm obviously going to read those 50+ pages of t&c's you've asked me to agree to on my phone. #
  • Call to back UK's hi-tech talent – http://www.bbc.co.uk/news/technology-12326688 (software / games ) #
  • “@scottsigler: Time for Rocket Fuel. Recipe: equal parts strong coffee, sugar and Red Bull…" @stormysan @aypok u r wimps in comparison #
  • Happy mailman day. #
  • Note to self: when leaving windows open to air place of residence avoid leaving the fan heater turned on. #fail #
  • I wonder what significance there is to the tiles on my bathroom floor becoming loose. #
  • Looks like I have a sea faring new employee … http://twitpic.com/3v4var #

The Post Office

Earlier today, I went to the Post Office in Bromsgrove, to buy some Euros.

The conversation I had with the minion behind the counter is paraphrased as follows :

  • Me: Hi, I’d like to buy $x euros, please?
  • Her: How do you wish to pay?
  • Me: By card? <<waves debit card at her>>
  • Her: We’ll need proof of ID (Passport etc)
  • Me: <<sighs>> Why?
  • Her: It’s an anti-fraud thing, you don’t think like a criminal do you?
  • Me: Errr? <<WTF?>>
  • Her: If you’ve stolen a card, the first thing you do is try and withdraw money using it … blah blah blah fraud blah blah blah ….
  • Me: But, it needs a PIN number to be used…?
  • Her: It’s ok, <<gestures at the card reader infront of me>> – that acts like a cash point. You can withdraw the right amount of cash to pay for your euros using it…
  • Me: <<WTF?>>
  • Her: Now, just put your card in the reader and type in your PIN when requested….
  • Me: <<types in pin>>
  • Her: <<hands over euros>>

I know I’m often a bit dim, but I’m failing to understand the ‘process’. Wasn’t I meant to have proof of ID to buy euros using my debit card?

How is me “withdrawing” cash for her, any different to me paying by card – especially when the process from my point of view is IDENTICAL.

PHP Serialization & igbinary

Recently I’ve been trying to cache more and more stuff – mostly to speed things up. All was well, while I was storing relatively small numbers of data – because (as you’ll see below) my approach was a little flawed.

Random background – I use Zend_Cache, in a sort of wrapped up local ‘Cache’ object, because I’m lazy. This uses Zend_Cache_Backend_File for storage of data, and makes sure e.g. different sites (dev/demo/live) have their own unique storage location – and also that nothing goes wrong if e.g. a maintenance script is run by a different user account.

My naive approach was to do e.g.

$cached_data = $cache->load('lots_of_stuff');
if(!empty($cached_data)) {
   if(isset($cached_data[$key])) {
       return $value;
   }
}
else {
    // calculate $value
    $cached_data[$key] = $value;
    $cache->save($cached_data, $cache_key);
}
return $value;

The big problem with this is that the $cached_data array tends to grow quite large; and PHP spends too long unserializing/serializing. The easy solution for that is to use more than one cache key. Problem mostly solved.

However, if the site is performing a few thousand calculations, speed of [de]serialisation is still gong to be an issue – even if the data involved is in small packets. I’d already profiled the code with xdebug/kcachegrind and could see PHP was spending a significant amount of time performing serialisation – and then remembered a presentation I’d seen (http://ilia.ws/files/zendcon_2010_hidden_features.pdf – see slides 14/15/16 I think) at PHPBarcelona covering Igbinary (https://github.com/phadej/igbinary)

Once you install the extension –

phpize
./configure
make
cp igbinary.so /usr/lib/somewhere
#add .ini file to /etc/php5/conf.d/

You’ll have access to igbinary_serialize() and igbinary_unserialize() (I think ‘make install’ failed for me, hence the manual cp etc).

I did a random performance test based on this and it seems to be somewhat quicker than other options (json_encode/serialize) – this was using PHP 5.3.5 on a 64bit platform. Each approach used the same data structure (a somewhat nested array); the important things to realise are that igbinary is quickest and uses less disk space.

JSON (json_encode/json_decode):

  • JSON encoded in 2.18 seconds
  • JSON decoded in 9.83 seconds
  • serialized “String” size : 13993

Native PHP :

  • PHP serialized in 2.91 seconds
  • PHP unserialized in 6.43 seconds
  • serialized “String” size : 20769

Igbinary :

  • WIN igbinary serialized in 1.60 seconds
  • WIN igbinrary unserialized in 4.77 seconds
  • WIN serialized “String” Size : 4467

The performance testing bit is related to this Stackoverflow comment I made on what seemed a related post

Twitter Weekly Updates for 2011-01-30

  • Hhh: “@madeupstats: Xprts hv wrnd by currnt trnds vwls cld bcm xtnct by 2050.” #
  • Bromsgrove 3rd hockey vs khansa(?) (Warwick). We lost 3-1 🙁 we were winning 1-0 at half time 🙂 #
  • RT: @GeekStats: ~20% of Facebook users state their relationship status. 40% = "single"; 3% = "it's complicated" http://bit.ly/gnuFiH #
  • It seems my turbo trainer doesn't wake toddlers up. This is good. #ExercisingInSecret #
  • “@StormySan: Apparently the phrase 'Like a red rag to a bull' isn't appropriate to describe anger during PMS.” <- good thing Lyne in today #
  • Beware Bromsgrove; I am slowly claiming back mayorship of the town centre. Competition will be squashed….. #4sq #
  • My random PHP serialization performance note – http://bit.ly/hug5bY – igbinary vs serialize vs json_encode (result: igbinary is best). #
  • Batman and superman are patrolling Bromsgrove today. Have no fear citizens. #
  • Now, onwards to meet @zookx … Interesting that they're in a previous office of ours. Still – No excuse for failing to find them. #
  • My minions accepted their payrise without fuss. Perhaps I'll try a negative rise next time and see if they notice. #evil #employer #
  • I'm always slightly surprised when I come across a non supported / non LTS ubuntu release being used in production. #
  • This morning has been mostly Linux sysadmin (virtual host setup, backup jobs, updating stuff). Sometimes the boss let's me code. #pray #
  • Yes, in the end it's beautiful, so beautiful…. You are a puppet. I am a puppet. We are all puppets. #guessTheSong #
  • .@bhamsouthpolice – “Don’t forget that our 24 hour tweet-a-thon starts tomorrow morning at 7.00am. ….” #
  • Customer: can you backup server, here's the root u/p. Me: err I have a few questions first …. #Linux #support #sysadmin #
  • “@StormySan: Ahhh, the first Monkeys call of 2011. Good start to the day :)” LOL-telephony #phonespam #
  • I'm such a gentleman, lending women money so they can get home from a night out. #StretchingTheFactsALittle #
  • “@madeupstats: Let's hope the Daily Mail doesn't get hold of this: almost half of Britain's schools perform below average.” Rotfl. #
  • “@StormySan: !in_array($bitch, $99problems) #technologysongs

Twitter Weekly Updates for 2011-01-23

  • Slave labour / someone likes cleaning. http://twitpic.com/3spmzo #
  • I <3 chocolate rice crispy cake. #
  • Turbo trainer found. Now what about the bike ? (volare elite esto mag something.) #
  • Bromsgrove 3rds vs West Brom – lost 2-0. I played the second half. Meh. #
  • Must learn to not leave home without my hockey stick when going to play hockey…. #
  • It appears my body is unwilling to give up it's blood easily. #
  • Blood donation o'clock. #
  • "I will feel better having you there; that is what a relationship is; we average our misery" #house #
  • We can get FTTC 'Internet' – 39/9 mbps down/up (~3x faster, ~2x price than now) but with BTInternet – think i'll stick with BeThere for now #
  • Browsing a Samba share from OSX seems exceptionally painful, especially if OpenOffice is involved. Grr… #
  • I've had enough of Dublin Core, OpenGraph and EGMS meta tags for this morning. Next ticket please. #
  • Ah. The yellow pages is finally letter
    box size. Took "them" long enough to figure that one out ! #
  • Who stole my minions and replaced them with muppets that listen to rap music all the time? #AliensAreHere #
  • I was a bit surprised to see piles of 'snow' still in Tesco's car park earlier (Redditch). #

Javascript Linting…

Suffice to say, my minions write a quantity of Javascript. And testing it isn’t all that easy. While rummaging the internet, I came across @NeilCrosby‘s FrontEndTestSuite which aims to automate e.g. w3c validator checks and so on – there will be more on that later I suspect once I get it working.

Anyway, the first part I wanted to do is to run a Javascript linter on things…. as I haven’t really come across these before.

I’ve found two approaches :

Install JavascriptLint

see http://www.javascriptlint.com/download.htm

Build instructions for the lazy – uncompress/extract the files from the archive, then :

  1. cd jsl-0.3.0/src
  2. make -f Makefile.ref
  3. cp Linux_All_DBG.OBJ/jsl /usr/local/bin

Usage looks a bit like :

jsl -process $file

As per the ‘help’ documentation, it returns different exit codes depending how things went (e.g. 0 – everything good; 1 – warnings etc).

Aside from the annoying compile step, this seemed the easiest to setup, and friendliest to use from the command line.

Mozilla’s Rhino & JSLint.js

This is the approach expected by Neil’s TestSuite above (more soon, perhaps).

Download the Mozilla Rhino thing – for me this is a simple ‘apt-get install rhino‘ YMMV.

  1. export CLASSPATH=/usr/share/java/js.jar
  2. java org.mozilla.javascript.tools.shell.Main /path/to/some/javascript.js

Again, this will give some sort of return error code if it can’t parse it – but it’s not yet running through jslint… which is what we really want.

Firstly, download JSLint.js via https://github.com/douglascrockford/JSLint/blob/master/fulljslint.js (click on the ‘raw’ button)…

  1. (Requires CLASSPATH thing from above)
  2. java org.mozilla.javascript.tools.shell.Main
  3. load(‘fulljslint.js’);
  4. to_test = readFile(‘/path/to/javascript/file/to/test.js’);
  5. result = JSLINT(to_test, null);

If ‘result’ is ‘false’ then you can inspect the errors via JSLINT.errors.

Next up, getting frontend-test-suite running, or something based upon it….

Twitter Weekly Updates for 2011-01-16

  • Playing for the other team…. #hockey #
  • Nails filed. Breakfast eaten. Hello Sunday…. #
  • Only 3 doors left to open on @rowangoodwin's advent calendar. #festiveSpirit 🙂 #
  • Bromsgrove 3rds 1:7 hampton Arden. #hockey 🙁 #
  • Yawn. Yawn. Yawn. #
  • Neighbours – STFU. Kthxbai. #
  • Not sure why Bromsgrove council feel the need to build a new leisure centre – the current one is fine and only just been refurbished #
  • I can't help but think the Bromsgrove standard is a little optimistic with it's 'cinema in Bromsgove' headline. #
  • I'll soon be deaf unless these burglar alarms STFU. #Bromsgrove #electric #fail #
  • I should pull my finger out and run, before hockey training. Need to meet some zombies while armed with hockey stick -> awesome evening 🙂 #
  • The parenting manual probably says:
    "Rule 57: Do not feed toddlers mince pies after 7pm.". Oh well. @RowanGoodwin had two… #
  • Off to the sidemoor SHED meeting. Community allotment(s) and err something. #
  • RT @GeneHunt The girlfriend's mother was taken ill. I acted swiftly and rushed off to find a pen and paper to write for an ambulance. #
  • Interesting presentation by @akrabat concerning the changes coming in with ZF2… nice to see many new faces at #phpwm meeting this evening. #
  • I'm at fat fighters. #
  • Ditched Microsoft's crash prone OSX remote desktop client; replaced with CoRD; thanks @cordapp #
  • I nominate @GeneHunt for a Shorty Award in #humor because he makes me smile, and glad I'm a man 🙂 #
  • RT @GeneHunt My ex used to get upset if I used her toothbrush. If anyone knows a better way to get dog crap off my boots, I'm all ears. #
  • RT @GeneHunt I love defenceless animals, especially in a good gravy. #
  • Sleepless in Bromsgrove #NewMovieNames #

Upgrading Cassandra 0.6 to 0.7 …

For one project we use Cassandra as a distributed backend message store (for an email archive, which by it’s nature is always going to grow in size); we choose to use Cassandra as it offered the ability to replicate data over a number of servers – giving us scalability and redundancy. Also, for the project in question, we only ever retrieve an email based on it’s message-id – which happens to be unique (hopefully) and forms a good key 🙂

Anyway, we’ve been using Cassandra 0.6.x for some time, through the Debian packages the project makes available. All was well, until this afternoon when I saw an upgrade to 0.7 was available… now, I knew 0.7 was a long awaited upgrade (as it would allow us to create new keyspaces etc on the fly; apparently…)… and I thought

“No doubt they [the package maintainers] will have either a big warning message, or some automatic migration from 0.6 to 0.7”

I was wrong.

Upon restart of Cassandra (and chown -R cassandra:cassandra /var/lib/cassandra) 0.7, everything appeared fine – except it had no idea where our Keyspace was – but did give an error message like :

“DatabaseDescriptor.java (line 439) Found table data in data directories. Consider using JMX to call org.apache.cassandra.service.StorageService.loadSchemaFromYaml().” in /var/log/cassandra/system.log

Rummaging through the online docs showed that we’d need fire up a “jconsole” thing to fix it. Unfortunately it running on a remote server, so this wasn’t so easy. The easiest solution seemed to be to download Cassandra locally, copy the remote server’s storage-conf.xml file locally and then run the included ‘bin/config-convertor’ –

bin/config-converter conf/storage-conf.xml conf/cassandra.yaml

This YAML file could then be copied to the remote server (/etc/cassandra/cassandra.yaml); then restart the Cassandra service, and you’re ready to connect via jconsole and perform the ‘migration’ to your pre-existing schema…

ssh -L 8080:localhost:8080 user@remote.server

<<start jconsole, and point at localhost:8080; no authentication required>>

And click :

MBeans -> org.apache.cassandra.db -> StorageService -> Operations -> loadSchemaFromYaml

Once this was done, we found that running ‘show keyspaces‘ from within the ‘cassandra-cli’ client showed what we needed (our well named ‘Keyspace1’).

Then we just needed to upgrade our pycassa version so the client connected properly, and everything started to work….

Twitter Weekly Updates for 2011-01-09

  • "behave yourself would you, no homework! Watch sone porn!" #supernatural #
  • RT @GeneHunt Gene's 4th law of physics: The more streamlined a woman is, the greater resistance she offers. #
  • RT @GeneHunt There's only one sort of 'man bag' a man should carry and that's the one God gave him. #
  • RT @dick_turpin ipad for the elderly http://ping.fm/h3f2t #
  • Twitter OSX client seems ok; once you realise apple+N is for a new tweet. #appstore #twitter #
  • Snow. Go away. Don't even think about falling. I want to play hockey tomorrow. That is all. #
  • RT @joshprice Top tip: Pay for your homeopathic medicine with a glass of water that's had a $100 note dipped into it #
  • Too many tweets.
    Not enough time.
    'Ctrl-a – mark as read' type behaviour by me. #
  • Dripping tap 1 : 0 David. #Grr #
  • No, I know all I am and we are not to blame. …. Prepare yourself for the subjugation. #zto #
  • RT @wonkothesane http://imgur.com/fDAQd – language barrier <- nice 🙂 #
  • Do all taps take the same sized washer? #dripdripdrip #
  • Thunderbird 2 in action !!
    http://www.bbc.co.uk/news/technology-12110386 #
  • My minions seem to be boasting about how much caffeine they need to drink to get through the day. Clearly they've had too much time off… #
  • Today it seems @rowangoodwin is sleeping in. Why couldn't he do this before when we didn't have to get up? #
  • Rowan is very cute and cuddly lying asleep on my shoulder. #snoreSnore #

Twitter Weekly Updates for 2011-01-02

  • When my mother complains I'm really quiet on the phone what she really means is that her phone handset volume is turned down very low. #
  • "They were saying on radio 4 that there's a new iPhone germ" said my mother. #
  • RT @carolinegoodwin so it seems my nephew doesn't want a bath & possesses lungs capable of informing all of this situation. Dad 1:0 toddler #
  • Wetherspoons. You need more staff. Especially on a busy evening. #
  • Last run, shower and shave of the year complete. #randomcrap #
  • A modern curse: "May all your presents be delivered by City Link" <- @aypok !! #
  • Bad asda has no LCD tvs left. I predict a day driving now. Grr. #
  • Tron. In 2D. Because I'm a Luddite. #
  • STBXW is dressed like her MIL. #
  • RT @KevlinHenney RT @sf105 Why Companies Should Insist that Employees Take Naps http://is.gd/jIWFK <- interesting. #
  • RT @webmonkeyash Gingerbread house complete and ready to eat! http://flic.kr/p/95H45J <- Nom Nom looks good and yummy. #
  • Bromsgrove dog owners – snow does not remove your responsibility to clean up after your dog. Dog shit is not a grit substitute. #
  • Pudding #1 lost due to baby meddling. #
  • RT @gadgetlab How zip ties can help you bike through a blizzard http://bit.ly/fbLcXs <- too late now ! 🙂 #
  • My iPod Shuffle is dead; any recommendations for equivalent players? Needs to be small, decent battery life; iTunes sync would be nice 🙂 #
  • The rain sounds very nice and relaxing. Hopefully we'll be snow and ice free soon 🙂 #
  • http://sickipedia.org/ – plenty of jokes there…. #
  • Laundered money (with a distinctive tracking colour) http://twitpic.com/3k5tdo #WalletsDoNotNeedWashing #