Linux console dimming / screen saver

Our work server seems to enjoy crashing (at the moment) about once a day – but when I go to look at the console I can’t see anything because the screen has dimmed to black – and as the computer has locked up, I can’t wake the screen up to see if there is a useful kernel panic message.

It turns out my answer lies with /etc/console-tools/config – as this is Debian Squeeze.

See http://serverfault.com/questions/137728/how-do-i-permanently-disable-linuxs-console-screen-saver-system-wide

Next up, I just needed to stop it from dimming the console font – adding this to /etc/rc.local seems to help :

setterm -half-bright off

Netbeans vs Vim … round 1 … fight

So, I think I’ve changed ‘editor’. Perhaps this is a bit like an engineer changing their calculator or something.

For the last 10 years, I’ve effectively only used ‘vim‘ for development of any PHP code I work on.

I felt I was best served using something like vim – where the interface was uncluttered, everything was a keypress away and I could literally fill my entire monitor with code. This was great if my day consisted of writing new code.

Unfortunately, this has rarely been the case for the last few years. I’ve increasingly found myself dipping in and out of projects – or needing to navigate through a complex set of dependencies to find methods/definitions/functions – thanks to  the likes of PSR0. Suffice to say, Vim doesn’t really help me do this.

Perhaps, I’ve finally learnt that ‘raw’ typing speed is not the only measure of productivity – navigation through the codebase, viewing inline documentation or having a debugger at my fingertips is also important.

So, last week, while working on one project, I eventually got fed up of juggling between terminals and fighting with tab completion that I re-installed netbeans – so, while I’m sure vim can probably do anything netbeans can – if you have the right plugin installed and super flexible fingers.

So, what have I gained/lost :

  • a 10 second slow down – that’s how long it takes to start Netbeans – even with an SSD and a desktop that should be quick enough (perhaps this is justification for a faster desktop?). Lesson learnt – do not close Netbeans.
  • indexing of the code – although this takes some time when importing an existing project – it does make it very quick to ‘find uses’ of function/class X
  • the ability to easily jump to a function/class definition (right click, find uses …)
  • a list of ‘todo’ items – from my code comments … this has led to me removing some old legacy cruft…
  • the ‘refactor’ functionality – i.e. rename function old_name() and change all instances of it to new_name()
  • code indentation seems to work better under netbeans
  • inline ‘warnings’ (i.e. unused variable, previously undefined variable[x], syntax errors etc) – so I don’t have to wait for them to appear in a browser or elsewhere.
  • inline documentation etc while typing (i.e. type in in_array( and it reminds me where the needle and haystack are)
  • with the right them, a nicer editor Window (mine is based on BrandonDark)

x – Fails with global variables on legacy projects though – in that netbeans doesn’t realise the variable has been sucked in through a earlier ‘require’ call.

I did briefly look at sublime a few weeks ago, but couldn’t see what the fuss was about – it didn’t seem to do very much – apart from have multiple tabs open for the various files I was editing.

irssi – crapbuster (cleaning up the irc window)

Tired of seeing join/part/quits messages in your IRC window, and want a quick way to selectively get rid of it – then look no further –

mkdir -p ~/.irssi/scripts
cd ~/.irssi/scripts
wget http://scripts.irssi.org/scripts/crapbuster.pl

Then within IRSSI –

/load crapbuster.pl
/set crapbuster_levels JOINS PARTS QUITS NICKS CLIENTCRAP CRAP MODE TOPICS KICKS

And then :

/crapbuster

Will remove all of the above stuff from your current IRC window.

 

If you put the crapbuster.pl script within ~/.irssi/scripts/autorun then it’ll load up by magic (saving you from doing the one line); after running the /set command, do a ‘/save’ and your irssi config will be updated – so in the future you’ll just need to type in ‘/crapbuster’

 

MySQL table defragmentation (python script)

I’ve written the below Python script to defragment MySQL database tables. This should work with both InnoDB and MyISAM table formats, and is just calling “OPTIMIZE TABLE blah.foo”. It rummages through the information_schema for tables which may be defragmented.

You should be able to run it on any Debian based system without making any alterations.

 

#!/usr/bin/env python
import ConfigParser
import os
import time

# On Debian, /etc/mysql/debian.cnf contains 'root' a like login and password.
config = ConfigParser.ConfigParser()
config.read("/etc/mysql/debian.cnf")
username = config.get('client', 'user')
password = config.get('client', 'password')
hostname = config.get('client', 'host')

defragmented_tables_cmd="mysql -u %s -p%s -h %s --skip-column-names --batch  -e 'SELECT TABLE_SCHEMA, TABLE_NAME, Data_free FROM information_schema.TABLES WHERE Data_free > 1000'" % (username, password, hostname)

# This will explode if any databases or tables have spaces in their names.
for dbtable in os.popen(defragmented_tables_cmd).readlines():
    (database,table,space_free) = dbtable.split(None)
    print " Defragmenting database : %s, table: %s, space free: %s " % (database, table, space_free)
    os.popen("mysql --batch -u %s -p%s -h %s -e 'OPTIMIZE TABLE %s' " % (username, password, hostname, database + '.' + table))

Having written this, I don’t think it does any more than “mysqloptimize –all-databases”, althoug you could make the above run selectively (i.e. only on certain databases / table formats / disk free amounts ).

Chocolate headed pirates

My son woke up at 6am crying quite loudly this morning – he’d been having a nightmare involving a Chocolate Headed Pirate being mean and nasty to him.

Personally I think a chocolate headed pirate would be quite tasty – although perhaps it’d be a bit more than I could eat in one sitting (or one week),  however, just like I don’t really understand his fear of Mr Choco Pirate, I’m sure there are many things I’m afraid of which he’d find funny.

Oh, the joys of childhood!

Spring trip to Barcelona

A monster in the sand!

Well, sort of random. As I don’t normally bother to upload any pictures and so on, I thought I might as well for once – and I said I’d send my father a postcard, but then failed to …. perhaps this will make up for it. I didn’t notice any postcard selling shops either – perhaps they’ve gone out of fashion?

Anyway, as I’m the boss, I left work at midday on Friday, drove up to Liverpool and flew to Barcelona with EasyJet. Thankfully I’m quite happy flying – but a minor bump triggered a load of men near me to exchange flying horror stories (“Once, over America, we hit some sort of air pocket and dropped 1000 feet!”). Anyway, I arrived in Barcelona at about 20:00 local time (I think flight time was about an hour and a half – apparently we had a 70mph tailwind, so were a bit early) and then fought through the cattle^h^h^hpassengers to get through security etc etc… train, tube, walk -> arrive at Anna’s flat. Fall asleep.

On Saturday we went shopping – so it was a boring day in that respect. But in the evening we went to my favourite Japanese restaurant to eat sushi stuff (nom-nom-nom) – but as I failed to take my camera/phone with me, there’s no photo(s).

On Sunday – we’d went to Sitges and walked into some sort of old-car rally – which was nice to see. Unfortunately this made the town quite busy, so we had to wait for ages for lunch (well, 3pm lunch).

Then we went back home, I did some DIY and the weekend was pretty much over.

Monday involved flying back (uneventful) and then a long drive home. Liverpool seemed very dull and dreary compared to Spain.

WTFs per minute

I’m currently refactoring some legacy third party PHP code, and as the old saying goes, the real metric is WTFs per minute.

So, just to entertain any readers, how about :

  1. Writing pagination links for a search form, but if there are more than 20 pages of results, add 20 onto whatever the maximum number of pages there are – so you get 20 invalid links at the end of the pagination list (clicking on them will show no results). I guess it looks like there are lots of results at least.
  2. if(isset($_GET[‘foo’]) == 0) … (wouldn’t if(!isset($_GET[‘foo’]) be easier to read?).
  3. Presumably not knowing what a while(…) { … } loop is, and always using something like: $row = mysql_fetch_assoc($x); do { … } while ($row = mysql_fetch_assoc($x)) ….
  4. Always including mysql_free_result($foo) after every query…. why bother?
  5. Always having an //END IF comment, even if the if(..) { } statement is only 3 lines long.
  6. The write_out_the_header() function which consists of a switch statement nearly 2900 lines long, which is just responsible for setting things like the <title> and some meta tags for every page in the site.
  7. When doing results pagination, for even numbered page links, write out the ‘jump’ URL differently (starting with a &, instead of a ?). Some numbers are more even/equal than others…. I guess.
  8. Executing a separate query each time within a loop rather than doing a simple join to start with….
And don’t get me started on the lack of error checking…..

A week of fire fighting (aka why you should <3 unit tests).

I feel like I’ve spent most of this week debugging some PHP code, and writing unit tests for it. Thankfully I think this firefighting exercise is nearly over.

Perhaps the alarm bells should have been going off a bit more in my head when it was implied that the code was being written in a quick-and-dirty manner (“as long as it works….”) and the customer kept adding in additional requirements – to the extent it is no longer clear where the end of the project is.

“It’s really simple – you just need to get a price back from two APIs”

then they added in :

“Some customers need to see the cheapest, some should only see some specific providers …”

and then :

“We want a global markup to be applied to all quotes”

and then :

“We also want a per customer markup”

and so on….

And the customer didn’t provide us with any verified test data (i.e. for a quote consisting of X it should cost A+B+C+D=Y).

The end result is an application which talks to two remote APIs to retrieve quotes. Users are asked at least 6 questions per quote (so there are a significant number of variations).

Experience made me slightly paranoid about editing the code base – I was worried I’d fix a bug in one pathway only to break another. On top of which, I initially didn’t really have any idea of whether it was broken or not – because I didn’t know what the correct (£) answers were.

Anyway, to start with, it was a bit like :

  • Deploy through some weird copy+pasting manner due to Windows file permissions
  • No unit test coverage
  • No logging
  • Apparently finished
  • Apparently working (but the customer kept asking for more changes)

Now:

  • Deployment through git; Stupid Windows file permissions fixed.
  • Merged in changes by third party graphics designer – should be much easier to track any changes he makes in the future
  • ~80% test code coverage. I’ve had to modify the ‘real’ scripts to accept a new parameter, which would make them read a local XML file (rather than the remote API/service) – in order for the tests to be reproducible (and quick to run)
  • Logging in place, so there’s some traceability
  • Better error handing
  • Calculations manually checked and confirmed.

Interesting things I didn’t like about the code :

  • Premature attempt at dependency injection – the few functions there are/were have a $db variable being passed in – but often don’t use it.
  • There is significant duplication in the code base still.
  • The code didn’t (and still doesn’t really) support testing particularly well – I’m having to retrieve output through a Zend_Http_Client call (i.e. mimicking a browser) – which means I can’t get code test coverage stats easily.
  • In some places a variable was being set to 0 (e.g. $speed) which would be used to determine a special case (if $speed == 0). Having multiple meanings in/on the same variable makes it difficult to follow what’s going on – and is a pain when the customer requests it behaves a bit differently. Really a separate parameter should have been used.