Publishing results

We saw in the Quickstart how to create and run an experiment. Now let’s see how we can take the data gathered in that experiment and publish it to make it useful to us.

Laboratory makes no assumptions about how to do this — it’s entirely for you to implement to suit your needs. For example, timing data can be sent to graphite, and mismatches could be written to disk for debugging at a later date.

Publishing

To publish, you must implement the publish() method on an Experiment.

The publish method is passed a Result instance, with control and candidate observations available under result.control and result.candidates respectively.

Experiment.publish(result)

Publish the results of an experiment. This is called after each experiment run. Exceptions that occur during publishing will be caught, but logged.

By default this is a no-op. See Publishing results.

Parameters:result (Result) – The result of an experiment run

StatsD implementation

Here’s an example implementation for statsd:

class StatsdExperiment(laboratory.Experiment):
    def publish(self, result):
        if result.match:
            statsd.incr('experiment.match')
        else:
            statsd.incr('experiment.mismatch')

        statsd.timing('experiment.control', result.control.duration)
        for obs in result.candidates:
            statsd.timing('experiment.%s' % obs.name, obs.duration)