Twitter Weekly Updates for 2010-02-07

  • Wondering where to run to tomorrow. #
  • Rowan now has a facebook account to go with his Twitter @rowangoodwin and blog … (, #
  • Haha msnbot it appears you are enjoying indexing a customers server a bit much. Have you heard of being 'nice' ? #
  • Today I did a total of 132 pushups thanks to the Hundred Pushups iPhone app. (Week 4, Day 3, Level 3) #100Pushups #
  • Rowan is still asleep. Lie in ftw. #
  • RT @loudmouthman What Woman will not tell you about pregancy is great breakfast reading #
  • RT @jzy: @codinghorror you should try "Outside" sometimes. – looks like a great game…. Cost? Hardware/platform? #
  • If only this developer knew database design. Adding (a) new table(s) for an additional dataset is just WRONG! #uneducated #heathen #
  • Today I did a total of 119 pushups thanks to the Hundred Pushups iPhone app. (Week 4, Day 2, Level 3) #100Pushups #
  • More sleep plz? Kthxzzzz #
  • Hmm aroundme's augmented reality looks cool; @moobert might kill me if i ask him to do it on the food hygiene iPhone app we're planning tho #
  • Don't you just hate it when you save data to a backend, only for it to silently vanish. (Apc + zend_cache. Perhaps apc needs more memory?) #
  • I am now 200 grams heavier. #chocolate #
  • Why do I need to FAX docs to apple to develop on the iPhone. FFS. What's wrong with email ? #fail #not1980sanylonger #
  • Today I did a total of 121 situps thanks to the 200 Situps iPhone app. (Week 2, Day 3, Level 3) #200Situps #
  • Interviewing possible php contractor this morning. I hope he's good – i've got work stacking up. #
  • *ding dong* "Time to wake up chuggers!" #
  • .RT @metofficeWMids ADVISORY of Heavy Snow for West Midlands valid from 0349 – 2359 Wed 03 Feb #
  • RT @scottmac: Announcing HipHop for PHP – #
  • RT @grifferz Impregnation via the proximal gastrointestinal tract in a patient with an aplastic distal vagina: (via @jwz) #
  • Today I did a total of 102 pushups thanks to the Hundred Pushups iPhone app. (Week 4, Day 1, Level 3) #100Pushups #
  • Sleep little toddler. Sleep. #
  • Cold cold. Brrr. Snow melted tho 🙁 #
  • Well that was a crash course into learning mailscanner with exim on rhel. What next dear customers ? #
  • Hmm 144 bus turns up an hour and a half late. Chances of getting to Birmingham before 1030 – near zero. #
  • Bromsgrove looks a bit clogged up this morning. Stupid snow. Wondering if I'm going to make
    the train now. #
  • Today I did a total of 108 situps thanks to the 200 Situps iPhone app. (Week 2, Day 2, Level 3) #200Situps #

Still looking for a PHP contractor….

At work I’m still looking for a short term PHP contractor. Perhaps I’m being unrealistic in my expectations/requirements (rate/location/duration/skills etc), but nevertheless…. As I’ve not found anyone via normal channels (twitter/phpwm user group etc) I thought I’d turn to a random recruitment agency (who I’d spoken to a week or so ago).

Yesterday I interviewed one guy – who’d been a programmer for a number of years (10+) – using Visual Foxpro (whatever that is) – presumably it’s a dead language, as he wants to move across into PHP. He has very basic PHP experience (yet claims 2 years on his CV), figured out how to do FizzBuzz and Recursion without too much help – but didn’t know anything about object orientation, separation of concerns (specifically MVC), security (obvious SQL injection) or unit testing and failed to make any comment on what is almost the worst code I could find to present to him. This isn’t necessarily a problem – I would normally be happy to train someone – however, not when I’m paying him £25/hour and I’d be lucky if he was productive within a week. (Hint: students are better than this when they’ve only been in University for two years).

Today, I therefore continued hunting, with mixed success. I had three more CVs – all asking for more money, and one looked quite good – but had a requirement he worked remotely after the first few days (well he does live in Telford). Another, who is local, I’m interviewing tomorrow. Wanting to do some homework on him, I had a look at a couple of websites mentioned in his doctored CV  – the first is clearly .Net from the error message it throws when you pass a > into it’s search box – so either they replaced his PHP site quickly or his CV is misleading. The second has a PHP error on it – and is only (effectively) a themed wordpress site which looks like it’s slowly rotting. From these I found out his address (hint: whois $flamingodomain) and an invalid email address/domain (which seems to not do much with). Typing in his name into Google / LinkedIn, Facebook etc produces no obvious matches. So I know hardly anything about him, and for all intent he may as well not exist. Great sales job there.

From talking to the recruiters it seems it’s difficult to find decent PHP programmers – and anyone who may be decent will almost certainly not be programming PHP as their primary language (i.e. they’ll be doing web development in Java/.Net, and know PHP quite well). This seems a shame, but really only confirms what I already knew from interacting with others in the community. I’ve known for ages that I’ve effectively taken a large pay cut by running my own company, and doing PHP. It sucks that this continues to be the case. Clearly I’m a martyr or something.

So, if you happen to be a contractor looking for work, please make an effort. I’m not overly impressed so far, and may just end up stalling customers for another week/month instead.

(Oddly I wrote this post, posted it, and it vanished. What are you up to wordpress? Why do you want me to retype things in twice?)

Can you write a web app, like Ebay, for me?

Today I had a phone call which went along the lines of ….

Prospect: Do you develop web applications?
Me: Yes… <cue sales pitch>
Prospect: I’ve got a great idea, it’s like eBay…. I need a programmer….
Me: <thinking: oh not another….>
Prospect: I think it’s about 2 developer months worth of work….
Me: Well, we’d need to see your requirements spec to determine that.
Prospect: Would you be willing to do the work for free in return for a stake in the resultant venture? How much would it cost?
Me: Well, I’ve not seen any sort of requirements specification; I’ve no idea what’s involved….. Just how long is a piece of string?

I don’t quite understand why people think they’ll be able to create an eBay/Amazon/Google/whatever killer/competitor with minimal funding, a couple of months development and also persuade me to do the work for free on the outside chance they’re successful (that’s like a <5% chance). I might consider reducing the price of the development in return for a share in any resultant company – but I’m sure as not going to commit to anything before I’ve seen any sort of business plan.

Perhaps tomorrow I’ll get someone wanting a site “like Facebook” costing a few thousand pounds… ‘cos we do PHP just like them… and it can’t be hard can it?

Rate limiting http traffic (mod_evasive and iptables)

A customer has a relatively busy web site, which contains lots of juicy information (business names, addresses, email address, phone numbers etc etc). Currently there is nothing in place to stop people spidering it – unless someone explicitly looks at the log files and does something.

Blocking annoying people who spider the site is easy enough –

iptables -I INPUT -s 80.x.x.x -j REJECT

However, I’d obviously rather automate this if possible – and ideally without having to change the PHP code (as each request would need perform some sort of DB lookup it’s part of a spidering attempt)

So, my first idea was to manipulate an existing rule I have to limit SSH connection attempts, giving something like :

iptables -I INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --set
iptables -I INPUT -p tcp --dport 80 -i eth0 -m state --state NEW -m recent --update --seconds 60 --hitcount 40 -j LOG --log-prefix "http spidering?" --log-ip-options --log-tcp-options --log-tcp-sequence --log-level 4

Annoyingly however, even though these are the first rules in the iptables output – and they should therefore work, they don’t – i.e. I’m not seeing anything being logged, when doing e.g. the following on a remote server :

while [ true ] ; do
wget -q -O -

So, I’m still trying to avoid making changes to the code base – although doing so would produce the best user experience (namely we could display a captcha or something and if someone really can browse that quickly they’d not encounter any problems).

And as I’ve just found mod_evasive which claims to provide DoS and DDoS protection. Thankfully Jason Litka has packaged it – so I have no problems from an installation point of view 🙂 (yum install mod_evasive)

Installation on Debian doesn’t result in a config file – but it’s not difficult to create (see /usr/share/doc/mod_evasive). However, it’s not a shiney, sunny ending – mod_evasive appears to be “tripped” by people requesting images – and in my case the client has about 10-20 images per page; so it’s difficult to differentiate between a normal user loading a page or someone running httrack on the website and only requesting the “php page”. If only mod_evasive took a regexp to ignore/match… and I can’t seem to find anyway of fixing this.

So application logic it is :-/ Perhaps caching in APC may be the way forward ….

Twitter Weekly Updates for 2010-01-31

  • RT @glynmoody Facebook rewrites PHP runtime – to be released as open source #facebook #php #
  • Poop. Spoke too soon. Snow tap being turned off. F'ing weather god. Curse you. #
  • Decent snow. 4/10 perhaps. B61 #uksnow … Just keep up for an hour or two and perhaps i can sledge/snow fight. #
  • Looks like we had a token amount of snow last night. Looks cold too :-/ 14 miles here I run. Still, cybrosis ppdcast episode to listen to:) #
  • Our car has depreciated £1000 for every year (7) we've owned it. Perhaps it'll soon put on value when as it turns into a mobile pool? #
  • Today I did a total of 105 pushups thanks to the Hundred Pushups iPhone app. (Week 3, Day 3, Level 3) #100Pushups #
  • RT @stuherbert PHP 5.3 adoption: some numbers and talking points (please RT) #
  • Dogs have a very inefficient protocol for communication. Guessing lots of packet loss as they've been retransmitting for ages now. Woof woof #
  • The last apple in the shop should be avoided; keys are always in the last pocket you check. #lessonoftheday #
  • Today I did a total of 97 situps thanks to the 200 Situps iPhone app. (Week 2, Day 1, Level 3) #200Situps #
  • Twitterific appears to have won. Goodbye tweetdeck. #
  • Nice Run – roads (a38 etc) were almost empty, shame I'd have to get up at 5am to experience it more often :-/ #
  • 100 pushup thing is now hard; couldn't do last rep without two stops :-/ #100pushups weak puny arms get bigger! #
  • Today I did a total of 100 pushups thanks to the Hundred Pushups iPhone app. (Week 3, Day 2, Level 3) #100Pushups #
  • Today I did a total of 92 situps thanks to the 200 Situps iPhone app. (Week 1, Day 3, Level 3) #200Situps #
  • is giving tweetdeck a whirl… as a change from twitterific #
  • Shocked to receive apple MacBook he ordered online yesterday afternoon this morning. Win! #
  • Today I did a total of 80 pushups thanks to the Hundred Pushups iPhone app. (Week 3, Day 1, Level 3) #100Pushups #
  • Worringly I seem to like coffee (with chocolate biscuits) I wish there was no junk food in this house. I'd best help 'dispose' of it….. #
  • Today I did a total of 88 situps thanks to the 200 Situps iPhone app. (Week 1, Day 2, Level 3) #200Situps #

Verified by Visa …. what rubbish

On Wednesday I was trying to buy train tickets for an upcoming trip to London.

So, I book the tickets, and get to point of being asked for my card details … tap tap tap … kapow … Up comes the Verified by Visa payment screen (in a stupid iframe [how do I know this isn’t a phishing site?]). Well, it displays my ‘username’ correctly – a terrificly hard to guess one of MRDAVIDGOODWIN… I enter my details and it keeps decling them. Hmm.. Fine… perhaps I’ve incorrectly stored the password – “oooh look – reset password…” *click* – “You want me to enter my date of birth… is that the ONLY security check you’re going to do? WTF??? ”

Grr.. Why do they bother….

See also

Random PHP project progress

Random php development musing

Initially when we founded Pale Purple all our new PHP development used a combination of Propel, Smarty and some inhouse glue. Over time we seem to have drifted towards the Zend Framework, but I’ve never been particularly happy with Zend_Db or Zend_View. Why the Zend Framework? Well, it has loads of useful components (Cache, Form, Routing, Mail etc) and it’s near enough an industry standard from what we see – and obviously I’d rather build on the shoulders of others than spend time developing an in-house framework no one else will ever use.

For one customer, we’re currently working on the next iteration of their code base – which incorporates a number of significant changes. When we inherited the code base from the previous developers we spent a long time patching various SQL Injection holes (casting to ints), moving over to use PDO’s prepared statements and trying to keep on top of the customer’s new functionality requests and support issues. There’s still a lot of horrible code to refactor, plenty of security holes (although none public facing) and we know we’re moving in the right direction – hopefully patching and duct tape will soon be a thing of the past as it will develop some form of architecture and look like someone has thought about design and long term maintenance.

I’ve started to properly do Test First Development – at least from a support perspective – as too often we’d find we would patch a bug, only for it to reappear again in a few weeks/months time. This has been especially useful with the SOAP interface the application exposes. The tests run every 5 minutes, and we all get emailed when stuff breaks – it took all of 30 minutes to setup and put in place – then it was just a case of actually writing the unit tests themselves (the tests take minutes to write; finding/fixing any bugs they pin point takes somewhat longer :-/ ). I’ve also abused Simpletest’s web testing ‘stuff’ to also act as an availability checker of the live site (i.e. hit a few remote URLs, and check that we don’t get error messages back and do see expected strings).

The original code base had no ‘model’ like layer (or MVC ‘compliance’) – files containing HTML, CSS, SQL, Javascript and PHP were the norm – we’ve added Propel to the project as the ‘model’ layer – which took a few hours; and then when reverse engineering the database we found a few oddities (tables without primary keys and so on) – anyway, moving the functionality from a handful of legacy objects across into the Propel ones seems to be well underway, and I for one will be glad to see the end of :

$x = new Foo(5);

Accompanied with code that does the equivalent of :

class Foo {
    public function __construct($id = false) {
        if($id != false) {
            // select * from foo where id = 5
            // populate $this; don't bother checking for the edge case where $id isn't valid
       else {
           // insert into foo values ('');
          // populate $this->id; leaving all other fields as empty strings...
     public function setBaz($nv) { // repeat for all table fields
         $this->baz = $nv;
         global $db;
         $db->query('update foo set baz = "' . $nv . '" where id = ' . $this->id);

Finally, we have a meaningful directory structure – where some things aren’t exposed in the document root. Hopefully soon a front controller and some decent routing. At the moment a huge amount of code is just sat in the ‘public’ directory due to it’s nature. We hope to fix this in time, and move to using Zend Controller – once we get Smarty integrated with it.

Propel has added some nice new features since we last used it (effectively v1.2); it was a toss up between it and Doctrine (as obviously the ZF is moving in that direction) – but we already had knowledge/experience with Propel and it seemed the easier option.

I’m hoping that with time we’ll be able to get up to at least 60% test coverage of the code base – at that point we should be able to refactor the code far easier and with less fear. At the moment I doubt the unit tests cover more than 5-10% – although maybe it’s time I pointed xdebug or whatever at it to generate some meaningful stats.

My final task is to get some decent performance measurements out of the code base – so we can track any performance regressions. I’m fairly confident that moving to Propel will result in an speedup as duplicate object hydrations will be eliminated thanks to it’s instance pool, however having hard figures and nice graphs to point at would be ideal. So far I’ve knocked up my own script around ‘ab’ which stores some figures to a .csv file and uses ezComponents to generate a graph file. This seems to be a crap solution, but I can’t think or find anything better. Any suggestions dear Internet? Perhaps I should integrate changeset/revision id’s in my benchmarking too. Suggestions here would be exceedingly appreciated.

There, I should have ticked all necessary boxes wrt development practices now. Now to work on finding a contract PHP developer….

The PHP Security Journey begins…

Here’s the slides from the PHPWM talk I gave last week PHPWM Presentation – The Security Journey Begins ; thanks to DeanC on #phpwm for reminding me to upload them 🙂

The presentation focusses on security issues in web applications – specifically, PHP – although obviously other web facing languages face the same problems. It’s a very condensed version of what I normally give as a two day PHP security training course – so there are bits missing, and many things aren’t explained fully… and obviously the demonstration after the slides is missing 🙂

(250kb, PDF file… I think)

Twitter Weekly Updates for 2010-01-24

  • Bubble blowing fail day. Do mixtures have a BBE date? Rowan seems happy with one bubble in 10 goes. #
  • 820 days uptime is sufficient; time for a long overdue reboot I think. #linux #
  • There's still snow outside tesco. Strange redditch. #
  • Today I did a total of 77 situps thanks to the 200 Situps iPhone app. (Week 1, Day 1, Level 3) #200Situps #
  • RT @Ade_B OMG I didnt realise they were making a new A Team Movie via @purityale w00t #
  • Wonder why everyone wishes they'd stayed in bed today?. Today was quite good for me…. #
  • Today I did a total of 81 pushups thanks to the Hundred Pushups iPhone app. (Week 2, Day 3, Level 3) #100Pushups #
  • Wake up little bunnies! #
  • RT @loudmouthman Well when you put it like that #
  • This side of heaven is right next door to hell. #
  • Enjoying Thunderbird 3 – faster, better UI; 3.0.1 is now out – #email #floss #thunderbird #
  • Met office once again fail. There's no snow here. #uksnow b61 #
  • The @scottsigler iphone app looks cool (chainsaw and kitten juggling eh?). It's free, gives easy access to great audiobooks +more #podcast #
  • Today I did a total of 74 pushups thanks to the Hundred Pushups iPhone app. (Week 2, Day 2, Level 3) #100Pushups #
  • Yawn. #
  • RT @rowangoodwin This time 2 years ago I was preparing to make my grand entrance! #
  • Wish my iPhone had a fingerprint/ facial/retinal recognition, instead of asking me for a password all the time. It has a camera afterall. #
  • – Google autocomplete rocks. See also #
  • trying to find a decent twitter username for $customer; it's like domain name squatting all over again. #
  • RT @evilneuro another reason not to use Internet Explorer, ever: – switch to chrome? #
  • iPhone voice recognition is getting worse. "phone Katherine Goodwin" != "phone kathryn reeve" *sigh* need aliasing or shortcuts #
  • Did aliens help plot the location of Woolworths? #
  • Today I did a total of 63 pushups thanks to the Hundred Pushups iPhone app. (Week 2, Day 1, Level 3) #100Pushups #
  • Hmm. Heavy snow for weds; heavy rain for thurs. Fun times ahead. #

Twitter Weekly Updates for 2010-01-17

  • Fantastic Mr Fox looks pretty good; Rowan seems to approve too 🙂 #
  • 13.26 miles, 1hour and 45 mins or thereabouts. Icy roads. #
  • Lets try #
  • Hotel chocolate is very nice; I lack self control and gorge myself on a packet at a time … And subsequently feel yucky. no common sense. #
  • Joined the weirdos in Bromsgrove by walking a childless push chair. Next up wearing womens clothes?shouting at people? X rd to avoid
    me! #
  • Installing Quickbooks is not fun. Pain. License keys. Pain. Updates etc. Payroll still to do :-/ #
  • Prezzo's Sticky toffee pudding is very nice; shame about the sugar rush afterwards. #
  • Today I did a total of 61 pushups thanks to the Hundred Pushups iPhone app. (Week 1, Day 3, Level 3) #100Pushups #
  • We're trainees and we're making tracks, wheels to the rails… Clackety clack! #chuggington #
  • Virtualbox OSE seems much better and quicker than vmware server. Bye bye vmware, your version 2 web ui will not be missed. #
  • Windows 7 looks pretty similar to vista to me. Wondering what the fuss is about? #
  • #uksnow b61 (bromsgrove) 1/10; 1" on ground. Roads appear ungritted. #timeforachange #
  • Bromsgrove roads appear ungritted. Traffic moving on a38; stourbridge rd worst. #
  • Today I did a total of 58 pushups thanks to the Hundred Pushups iPhone app. (Week 1, Day 2, Level 3) #100Pushups #
  • Grr. Snow. You've outlived your welcome. B61 #uksnow #
  • RT @guardiantech Microsoft Office disappears from virtual shelves as i4i's injunction bites #
  • #
  • Cold weather is awesome – week old bread hasn't gone mouldy 🙂 #
  • Finally stuck #varnish infront of some #plone / #zope sites. Performance++. Should have done this ages ago. 7req/sec -> 300req/sec etc. #
  • Today I did a total of 48 pushups thanks to the Hundred Pushups iPhone app. (Week 1, Day 1, Level 3) #100Pushups #
  • Interesting talk with asda cashier re muppets panic buying recently (15 loaves of bread anyone?). No
    Fresh milk the only affected thing 2day #