Late to the performance party

Everyone else probably already knows this, but $project is/was doing two queries on the MySQL database every time the end user typed in something to search on

  1. to get the data between a set range (SELECT x,y….. LIMIT n, OFFSET m or whatever) and
  2. another to get the total count of records (SELECT count(field) ….).

This is all very good, until there is sufficiently different logic in each query that when I deliberately set the offset in query #1 to 0 and limit very high and find that the of rows returned by both doesn’t match (this leads to broken paging for example)

Then I thought – surely everyone else doesn’t do a count query and then repeat it for the range of data they want back – there must be a better way… mustn’t there?

At which point I found:
http://forge.mysql.com/wiki/Top10SQLPerformanceTips
and
http://dev.mysql.com/doc/refman/5.0/en/information-functions.html#function_found-rows

See also the comment at the bottom of http://php.net/manual/en/pdostatement.rowcount.php which gives a good enough example (Search for SQL_CALC_FOUND_ROWS)

A few modifications later, run unit tests… they all pass…. all good.

I also found some interesting code like :

$total = sizeof($blah);
if($total == 0) { … }
elseif ($total != 0) { …. }
elseif ($something) { // WTF? }
else { // WTF? }

(The WTF comment were added by me… and I did check that I wasn’t just stupidly tired and not understanding what was going on).

The joys of software maintenance.

August 2010

I would start with saying that not a huge amount happened in August… but then having thought about it, I’d be lying.

My right ankle is still in a state of disrepair – after hurting my achilles tendon … so no running, and I’m feeling fat / unfit as a result. I have got my mountain bike out of the shed and started to cycle again and found a few interesting routes around Dodford.

No running, means no Nottingham Marathon. They did however send me the running top – so I can at least pretend to people that I did it – “Look! I have the t-shirt to prove it!”. I am hoping to start running again within the next week …

In other news, Bromsgrove Hockey club started to do some stuff again, although my one big toe appears to have been broken in the first game back (hint: keep your feet out of the way). The second time out (last weekend) was on grass, which bought back some memories and was quite enjoyable (if only 30 minutes play in total).

Pale Purple moved office (yes, so don’t ask: “Did you do anything nice on the August bank holiday weekend?” …). The cost of the office is effectively the same, but now we have more room and it’s a far nicer (not dim and dingy).

I’ve also joined the local RoundTable group – although I’ve not coughed up any membership fees yet, so perhaps I’m premature in saying “joined”. The first event involved driving a motor bike around a rough field in the rain (good fun) and the second involved playing Discus Golf. All good stuff, and the guys seem a great bunch.

Rowan’s started to potty train; Anya smiles and makes cute noises. I’ve had a hair cut. Fun times.

Oh, and work’s been busy and somewhat stressful, but that’s all hopefully over with now (as $site_migration is complete).

Twitter Weekly Updates for 2010-08-29

  • Hammer and chisel = superior fridge de-icing 🙂 #
  • Well the new office's dsl connection seems to be 80% working. Just no dhcp response. :-/ #
  • Looks fun 🙂 What's your role @schwukette ? (@schwukette)http://yfrog.com/n1x5ejj #
  • Wish the Chav family here could resist smoking in the playground. Grr. #arrowvalley #
  • I have swingers shoulder. #
  • Waiting for @rowangoodwin to wake up. Then buy lunch, duck food and stuff before invading the park and stealing ice cream. #
  • If only "attaching a screenshot" did not involve a word document. #
  • The train may soon go on holiday. #
  • http://www.kingtonlions.org/EventDetail.asp?EventNo=3781&Section=Information – Aberystwyth to Kington bike ride; 5th sept. #
  • Patiently waiting for @bitesms to release a fix for the facebook vs bitesms issue #

Twitter Weekly Updates for 2010-08-22

  • Tonight I will be dreaming "single, double, king. £3". How can you keep shouting that continually for hours on end you horrible man? #
  • Come on. DVDs are £1… Roll up.. Roll up. #
  • A very fat woman just bought maternity trousers from. So glad I'm not her. #studley #carboot #
  • My poor toe. http://yfrog.com/7dsy0zj – moral: better trainers needed for hockey. #
  • Good hockey games. Body aching, toe nicely bruised. #
  • "hello? Happy birthday to you too. Bye!" #toddler #phoneconversation #
  • Why won't @rowangoodwin say "[flash] gordon's alive!"… #
  • -> RT @dick_turpin iphone users have more sex than Android users http://ping.fm/sRDjd <- oh really? Why do my employees want 'droids? #
  • Facebook places eh? 4sq might soon be made redundant. Wish the updated iphone app worked for me tho. #
  • We have a new weapon against automated sales calls. tt-weasels and tt-monkeys. Thanks #asterisk #
  • Why can't a uni compsci dept. put a form online? Must i really print, write on dead tree and scan it in to submit/email. Grr. #aber #fail #
  • Today I did a total of 101 pushups thanks to the Hundred Pushups iPhone app. (Week 3, Day 2, Level 3) #100Pushups #
  • #WhatWouldYouTellYour18YearOldSelf Here are the lottery numbers for the next month 🙂 #
  • Who'd have known… Dodford is quite good for mountain biking 🙂 Nutnells wood and Pepperwood -good evening 🙂 http://osm.org/go/euw@v@f4- #
  • To cycle or run; that is the question of this evening. #

Twitter Weekly Updates for 2010-08-15

  • Sorcerers apprentice 7/10. Better than inception. #
  • pah, sorcerers apprentice.. i know what i meant. #
  • Internet: last airbender or the magicians apprentice? #
  • Today I did a total of 85 pushups thanks to the Hundred Pushups iPhone app. (Week 3, Day 1, Level 3) #100Pushups #
  • http://www.bbc.co.uk/news/magazine-10965608 rise of the MAMILS (Middle Aged Men in Lycra) – middle aged men cycling…. #
  • Shopping for three items turned into three carrier bags of stuff. #fail #
  • Good morning sunny Aberystwyth. We are armed with bucket and spade. Beware locals. #
  • The wheels on the bus go round and round….. And I'm falling asleep. #
  • £3 for the largest starbucks hot chocolate which isn't very big. Tasted crap. Should stick to costa or buy my own galaxy. #grumpyoldman #
  • I'm so cool. I have a Thomas the tank engine ringtone. #hip #cool #dad #
  • Seaside today. Better wake up early for once…. #
  • Rowan says I'm Shrek. ROAAAAAR!! #
  • Chocolate: £6.50ish, envelope: £1.50, postage to Canada: £12. Lesson learnt: don't bet with foreigners. #
  • http://code.google.com/p/iphone-backup-decoder/ <- seems to work quite well. Win. #
  • Clearly now is a bad time to go shopping for take away lunch. #
  • Install wordpress from svn && firewall webserver to not make port 80 requests out from apache && modify class-snoopy.php -> perhaps secure? #
  • #phpwm meeting tomorrow; don't forget … Or BurgerWM as some are referring to it already… #food #faminearoundthecorner #
  • To think, I was pretty sure I'd turned off the twitter integration in those exercise apps. #fail on my behalf. #
  • Today I did a total of 126 situps thanks to the 200 Situps iPhone app. (Week 2, Day 3, Level 3) #200Situps #
  • Today I did a total of 86 pushups thanks to the Hundred Pushups iPhone app. (Week 2, Day 3, Level 3) #100Pushups #

Asterisk – setting the caller id through an ldap lookup

Having just upgraded to asterisk 1.6, I was faced with the problem of callerid lookup no longer working.
We’d previously used the external LDAPGet module to lookup against our internal LDAP directory – but the module doesn’t compile in 1.6, so here’s a quick and dirty workaround which requires you have ldap-utils installed on your server.

In my extensions.conf :
exten => _[a-zA-Z0-9].,9,Set(CALLERID(name)=${SHELL(ldapsearch -w thepassword -D “cn=admin,dc=mydomain,dc=com” -h ldapserver -b “ou=addressbook,dc=mydomain,dc=com” -s children “(&(objectClass=person)(telephoneNumber=${CALLERID(num)}))” cn | grep ^cn | cut -d: -f2 | xargs echo -n)})

Where I used to have :

exten => _[a-zA-Z0-9].,9,LDAPget(CALLERID(name)=cidname)

Twitter Weekly Updates for 2010-08-08

  • Woke to the sound of lions roaring at 4am… (well I think it's the lions; not like the MGM lion) #
  • Campfire's burning .. Campfire's burning…. Draw nearer. http://yfrog.com/eqc4fj #
  • We're going to be camping at the safari park. ROAR! ROAR! #
  • Today I did a total of 111 situps thanks to the 200 Situps iPhone app. (Week 2, Day 2, Level 3) #200Situps #
  • Today I did a total of 75 pushups thanks to the Hundred Pushups iPhone app. (Week 2, Day 2, Level 3) #100Pushups #
  • Coffee + hot chocolate seems a good drink. Why do shops not sell chocolate coffee or hot choffee? #
  • "I design monkeys for zoos" … #pickupline #dayjob http://www.b3ta.com/questions/stupidity/post667972 #
  • Achilles tendonopathy (sp?). Exercises should fix it. Thanks to Taft physiotherapy, 46 new road, bromsgrove #
  • And now to physio-ize my ankle. I'm fed up not being able to run. #
  • Yey. Fixed a long standing sporadic bug … Silly soap data types. #
  • Today I did a total of 68 pushups thanks to the Hundred Pushups iPhone app. (Week 2, Day 1, Level 3) #100Pushups #
  • Jailbreakme worked first time. Quick. Exceedingly easy. Idiot proof. Well done @comex etc #
  • Jailbreakme looks excellent. Apple will be pissed that it appears so easy to do. I've so missed sbsettings and bitesms #
  • Toy story 3 …. Someone's transfixed by the big big big telly a least. Shame there's a head in my way. #

Twitter Weekly Updates for 2010-08-01

  • Sea life centre – £10ish… Seeing your son run into a mirror twice in a hall of mirrors – priceless. #evildad #
  • Apple Grammar fail (their != they're) http://store.apple.com/uk/product/H0462ZM/A?fnode=MTY1NDA0Ng&mco=MTM3NTE4NzA #
  • Birmingham sealife centre for me today it seems. #
  • RT @madeupstats 16% of the residents of Newark are unaware that their hometown is an anagram of 'Wanker'. #
  • Ooh, look… fancy URL rewriting (e.g. http://…./Light_Therapy) … I wonder what'll happen when I put a ' in there… #sqlinjection #
  • Protip – Webdevs submitting a CV, check your reference URLs; today's example: 2 domains parked; 2+ have SQL injections in the URL #
  • I told Ofcom what's wrong with the #DEAct code It only takes 2 min http://bit.ly/reply-to-ofcom #
  • Thanks for the nits toddler…. #joysofparenthood #
  • Nom nom. 5 crunch corners for £1. 52p each. Crazy stuff @asda #
  • 'ukathletics ltd' sent me some spam – 'london disability athletics challenge' … containing only remote images #accessibility #fail #
  • Why don't cows lay round bottles any longer? Is this a case of genetic over engineering? #
  • I wonder what people put in sandwiches before cheese was invented? #
  • The BBC's Sherlock (well first epsiode) was pretty good – glad to finally see some decent UK telly. #
  • while [ true ]; do echo 'thanks for waiting, one of our advisors will be with you as soon as possible'; done #onhold #hmrc #tax is-taxing #
  • I hope it's bromsgrove's air that smells of crap and not me… #
  • Email:"only 7 weeks to marathon… Get training". Me: that might be a problem as both legs voice displeasure. Stupid body. #
  • In other news, I'm now mayor of the cinema after two visits on 4sq. Guess no one plays it around here… #backwardsbirmingham #
  • Saw inception today. OK, but predators was better. End & plot twist too predictable #caveman #unculturedmanwhoprobablydidntgetit #

Grr…

Why would someone write Python and mix tabs and spaces. Do you really want to have random arbitrary bugs … grr….

:%s/^I/    //g

Grr. grr. grr. stupid programmer. Grr.. That’s my final moan. honest.