An auto-retry recipe for Twisted.

You can find chinese version of this article here : Deferred應用的實例 : retry

–

For grabbing pages from websites with twisted, you can write this:

getPage("http://www.google.com").addCallback(printPage)

But what about the HTTP server is too busy to response? Your getPage fails. For reasons, you have to modify your program to retry once the getPage failed. So you might write a method like this:

getPageRetry(5, "http://www.google.com")

Well, it seems works fine, but how about you got some other operations to retry? connectToClient, getTime and so on. You have to write

connectToClientRetry(5, "example.com")
getTimeRetry(5, "timeserver.com")

You have to write retry-version version of every function that may fail. That’s not a good idea. Twisted provides a very beautiful ansyc callback object named Deferred. It looks like Chain-of-responsibility, but it can also handle errors well. With its beautiful design, I wrote a retry function that can warp all function return Deferred that might fail. That’s no need to modify any exists code of getPage to make it to retry automatically. Here is the recipe:

import logging  
 
from twisted.internet import defer  
 
log = logging.getLogger('retry')  
 
def retry(times, func, *args, **kwargs):
    """retry a defer function 
 
    @param times: how many times to retry
    @param func: defer function
    """
    errorList = []
    deferred = defer.Deferred()
    def run():
        log.info('Try %s(*%s, **%s)', func.__name__, args, kwargs)
        d = func(*args, **kwargs)
        d.addCallbacks(deferred.callback, error)
    def error(error):
        errorList.append(error)
        # Retry
        if len(errorList) &lt; times:
            log.warn('Failed to try %s(*%s, **%s) %d times, retry...', func.__name__, args, kwargs, len(errorList))
            run()
        # Fail
        else:
            log.error('Failed to try %s(*%s, **%s) over %d times, stop', func.__name__, args, kwargs, len(errorList))
            deferred.errback(errorList)
    run()
    return deferred

Here you are! Now you can use it to make your operation to retry automatically!

from twisted.internet import reactor
from twisted.web.client import getPage  
 
def output(data):
    print 'output', data  
 
def error(error):
    print 'finall error', error  
 
d = retry(3, getPage, 'http://www.google2.com')
d.addCallbacks(output, error)  
 
d = retry(3, getPage, 'http://www.google.com')
d.addCallbacks(output, error)  
 
reactor.run()

Be Sociable, Share!

An auto-retry recipe for Twisted.

One Response to An auto-retry recipe for Twisted.

發表迴響取消回覆

關於作者

贊助商連結

彙整

分類

近期迴響

友站連結

An auto-retry recipe for Twisted.

One Response to An auto-retry recipe for Twisted.

發表迴響 取消回覆

關於作者

贊助商連結

標籤

彙整

分類

近期迴響

友站連結

發表迴響取消回覆