PHP Generators – an example

I’ve ignored generators for sometime in PHP, but recently realised why they can be quite handy 🙂

As an example, imagine you are querying a web service, which returns  data in chunks of up to 100 results….

To get results 100-200, it’s necessary to pass an ‘offset’ value to the query ($after).

The JSON response from the service looks a little like :

{
    "paging": { "after": "foo" },
    "data": [ 
         "stuff.we.care.about",
         "stuff.we.care.about",
         "stuff.we.care.about",
    ]
}

Naive (without error handling etc) PHP code for this could look a bit like the below :

function get_things_in_chunks($after = null) {
    $x = new \GuzzleHttp\Client(/* config */);
    $response = $x->get(
         'https://graph.facebook.com/vX.Y/something/blah', 
         [
             'query' => array_filter(
                 ['limit' => 100, 'q' => 'foo', 'after' => $after]
         ]);

    return json_decode($response->getBody()->getContents());
}

So : 

• My consumer (whatever is calling ‘get_things_in_chunks’) needs to know how paging is specified in the result returned back.
• My consumer needs to be able to figure out if it’s possible for there to be more results from the web service, in order to know when to make a new request …

So, if I want to keep iterating through the data returned until I meet some condition, my calling code could look a bit like the below :

<?php

$dt = new DateTime("1 year ago");

$after = null;

while (true) {
    $data = get_things_in_chunks($after);

    // find the paging key/data for a next request.
    $after = $data['paging']['after'] ?? null;
    
    // was there data returned? if not, break out of while(..)
    if(empty($data['data'])) {
        break;
    }

    
    foreach($data['data'] as $post) {
        if (!check_created_at_before_dt($post->created_at, $dt)) {
            break(2);
        }
        // do something
    }
}

We can simplify things for the consumer by using generators – like the example below.

The nice thing about this is that the caller no longer needs to know/care about the underlying response format from the web service  – so no longer needs to read $data[‘paging’][‘after’]  or whether there is data in $data[‘data’].

<?php
function get_things_in_chunks()
{
    $after = null;

    while (true) {
        $x = new \GuzzleHttp\Client(/* config */);
        $response = $x->get(
            'https://graph.facebook.com/vX.Y/something/blah', [
                'query' => array_filter(
                   ['limit' => 50, 'q' => 'foo', 'after' => $after]]));

        $data = json_decode($response->getBody()->getContents());

        if (!isset($data['data']) || empty($data['data'])) {
             return;
        }

        $after = $data['paging']['after'] ?? null;

       foreach ($data['data'] as $post) {
           yield $post; // magic here!
       }
    }
}

Now the caller can look more like :

<?php
$dt = new DateTime("1 year ago");

foreach (get_things_in_chunks() as $post) {
    if (!check_created_at_before_dt($post->created_at, $dt)) {
        break;
    }
    // do something.
}

So – now the caller only has to care about things it should be caring about ($post) and does not need to know or care about the paging mechanism that’s in place. 

If multiple consumers are using the same ‘get_things_in_chunks()’ function, then there’s less repetition / usage becomes far simpler (no $after, no need to check if there are more posts etc).