Addshore

It's a blog

Guzzle 6 retry middleware

Recently I switched from using Guzzle 5 to Guzzle 6 in my mediawiki-api-base PHP library. Everything went very smoothly except for there being no compatible version of the retry-subscriber that I had previously used. The subscriber has been replaced by retry middleware of which I was provided an extracted example. In this post I cover my implementation for the mediawiki-api-base library.

Guzzle 5

With Guzzle 5 you would create a retry subscriber with a filter and then attach it to the event emitter for the guzzle client. A full example can be seen below where RetrySubscriber::createStatusFilter can be seen here.

// Retry 500 and 503 responses
$retry = new RetrySubscriber([
    'filter' => RetrySubscriber::createStatusFilter(),
    'delay'  => function ($number, $event) { return 1; },
    'max' => 3,
]);

$client = new GuzzleHttp\Client();
$client->getEmitter()->attach($retry);

Guzzle 6

Guzzle 6 got rid of its event system and switched to the afore mentioned middleware system.

Middleware is added to a handler stack upon Client construction as seen below:

$handlerStack = HandlerStack::create( new CurlHandler() );
$handlerStack->push( Middleware::retry( retryDecider(), retryDelay() ) );
$client = new Client( array( 'handler' => $handlerStack ) );

The retry middleware takes two parameters, the first is the callable function that decides if a request / response should be retried and the last deciding how long to wait before retrying (similar to in Guzzle 6). Each can be seen below.

function retryDecider() {
   return function (
      $retries,
      Request $request,
      Response $response = null,
      RequestException $exception = null
   ) {
      // Limit the number of retries to 5
      if ( $retries >= 5 ) {
         return false;
      }

      // Retry connection exceptions
      if( $exception instanceof ConnectException ) {
         return true;
      }

      if( $response ) {
         // Retry on server errors
         if( $response->getStatusCode() >= 500 ) {
            return true;
         }
      }

      return false;
   };
}
function retryDelay() {
    return function( $numberOfRetries ) {
        return 1000 * $numberOfRetries;
    };
}

When implementing this for the mediawiki-api-base library I actually ended up creating a MiddlewareFactory which can be seen at on github here which is fully tested here.

This implementation includes a more complex retry decider including logging.

Enjoy!

9 Comments

  1. is there a provision of setting timeout with guzzle retry middleware? I want the retry attempt to be made upon expiration of pre-defined timeout interval

  2. Thanks for the hint. The retry middleware is really a must when having to deal with an API that isn’t 100% reliable.

  3. I’m curious, is this where OAuth token refreshes would be performed, or is that something which would have to be down further back up the chain?

    • addshore

      November 5, 2017 at 1:16 pm

      I have not really worked with OAuth & Guzzle yet so can’t give you the ‘right’ answer. It definitely sounds possible though.

  4. I’ve been following your example & making multiple asynchronous requests with Promise/any() functions.
    When retry middleware is not used & some of the api end-points are down, this is working fine. But when adding retry middleware while some end-points are down, the request gets slow. It seems that, though other end-points are up, but the failing points are blocking and retrying.

    trait RetriesRequest {
    
    public function getRetryHandler()
    {
        $handlerStack = HandlerStack::create(new CurlHandler());
    
        $handlerStack->push(Middleware::retry($this->retryDecider(), $this->retryDelay()));
    
        return $handlerStack;
    
    }
    
    public function retryDecider()
    {
    
        return function (
            $retries,
            Request $request,
            Response $response = null,
            RequestException $exception = null
        ) 
    
        {
    
    
            // Limit the number of retries to 5
            if ($retries >= 5)
            {
    
                return false;
    
            }
    
    
            // Retry connection exceptions
            if ($exception instanceof ConnectException)
            {
    
                return true;
    
            }
    
    
            if($exception instanceof AggregateException)
            {
    
                return true;
    
            }
    
    
            if ($response)
            {
                // Retry on server errors
                if ($response->getStatusCode() >= 500 )
                {
    
                    return true;
    
                }
    
            }
    
            return false;
        };
    
    }
    
    
    
    /**
     * delay 1s 2s 3s 4s 5s
     *
     * @return Closure
     */
    
    public function retryDelay()
    {
    
        return function ($numberOfRetries) 
        {
            return 2000;
            // return 1000 * $numberOfRetries;
        };
    
    }
    

    }

    Request Call Code:
    trait SearchesFlights
    {

    use ParsesJason, RetriesRequest;
    
    private $endPoints = [
                    'v1'=> '127.0.0.1:5354', //post request,  body.date
                    'v2' => '127.0.0.1:5939', //post request,  body.date
                    'v3' => '127.0.0.1:4949', //post request,  body.date
                    'v4' => '127.0.0.1:5051', //get request, no body
                    'v5' => '127.0.0.1:5052', //get request, no body
                ];
    
    
    
    private $postDate = '02/04/2018';
    
    
    public function searchByDate()
    {
    
        $client = new Client(array('handler' => $this->getRetryHandler()));
    

    /* $client = new Client();*/

        $promises = $this->getPromiseArray($client);
    
        $promise = Promise\any($promises)->then(
    
            function (PsrResponse $response){
    
    
                return $this->getAll(json_decode($response->getBody()));
    
    
            },
    
            function ($reason) {
    
                return $reason;
    
            }
    
        );
    
        $result = $promise->wait();
    
        return $result;
    
    }
    
    
    
    
    /*Returns Promise Array with post requests*/
    
    private function getPromiseArray($client)
    {
    
        $options = ['form_params' => ['date' => $this->postDate]];
    
        return $promises = [
            'server1' => $client->requestAsync('POST', $this->endPoints['v1']),
            'server2' => $client->requestAsync('POST', $this->endPoints['v2']),
            'server3' => $client->requestAsync('POST', $this->endPoints['v3'], $options),
    
    
        ];
    
    
    }
    
    
    }
  5. Should Line 20 be \GuzzleHttp\Exception\ConnectException?
    I am getting following error.
    Argument 4 passed to guzzelRetryClient::{closure}() must be an instance of GuzzleHttp\Psr7\RequestException, instance of GuzzleHttp\Exception\ConnectException given

  6. Edvinas Gurevičius

    February 21, 2020 at 1:20 pm

    Great article 😉 thank you sooo much!

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

© 2020 Addshore

Theme by Anders NorénUp ↑

%d bloggers like this:
Secured By miniOrange