API Reference

Experiment

class laboratory.experiment.Experiment(name='Experiment', context=None, raise_on_mismatch=False)

Experiment base class. Handles running your control and candidate functions. Should be subclassed to add publishing functionality.

Variables:

• name (string) – Experiment name
• raise_on_mismatch (bool) – Raise MismatchException when experiment results do not match

classmethod decorator(candidate, *exp_args, **exp_kwargs)

Decorate a control function in order to conduct an experiment when called.

Parameters:

• candidate (callable) – your candidate function
• exp_args (iterable) – positional arguments passed to Experiment
• exp_kwargs (dict) – keyword arguments passed to Experiment

Usage:

candidate_func = lambda: True

@Experiment.decorator(candidate_func)
def control_func():
    return True

control(control_func, args=None, kwargs=None, name='Control', context=None)

Set the experiment’s control function. Must be set before conduct() is called.

Parameters:

• control_func (callable) – your control function
• args (iterable) – positional arguments to pass to your function
• kwargs (dict) – keyword arguments to pass to your function
• name (string) – a name for your observation
• context (dict) – observation-specific context

Raises:

LaboratoryException – If attempting to set a second control case

candidate(cand_func, args=None, kwargs=None, name='Candidate', context=None)

Adds a candidate function to an experiment. Can be used multiple times for multiple candidates.

Parameters:

• cand_func (callable) – your control function
• args (iterable) – positional arguments to pass to your function
• kwargs (dict) – keyword arguments to pass to your function
• name (string) – a name for your observation
• context (dict) – observation-specific context

conduct(randomize=True)

Run control & candidate functions and return the control’s return value. control() must be called first.

Parameters:randomize (bool) – controls whether we shuffle the order of execution between control and candidate Raises:LaboratoryException – when no control case has been set Returns:Control function’s return value

enabled()

Enable the experiment? If false candidates will not be executed.

Return type:bool

compare(control, candidate)

Compares two Observation instances.

Parameters:

• control (Observation) – The control block’s Observation
• candidate (Observation) – A candidate block’s Observation

Raises:

MismatchException – If Experiment.raise_on_mismatch is True

Return bool:

match?

publish(result)

Publish the results of an experiment. This is called after each experiment run. Exceptions that occur during publishing will be caught, but logged.

By default this is a no-op. See Publishing results.

Parameters:result (Result) – The result of an experiment run

get_context()

Return dict:Experiment-wide context

Observation

class laboratory.observation.Observation(name, context=None)

Result of running a single code block.

Variables:

• name (string) – observation name
• failure (bool) – did the function raise an exception
• exception (Exception) – exception raised, if any
• exc_info – result of sys.exc_info(), if exception raised
• value – function return value

duration

How long the function took to execute

Return type:timedelta

get_context()

Return observation-specific context

Result

class laboratory.result.Result(experiment, control, candidates)

Variables:

• experiment (Experiment) – The experiment instance that recorded this Result
• control (Observation) – The control observation
• candidates ([Observation]) – A list of candidate observations
• match (bool) – Whether all candidates match the control case

Exceptions

exception laboratory.exceptions.LaboratoryException(message, *a, **kw)

Base class for all laboratory exceptions

exception laboratory.exceptions.MismatchException(message, *a, **kw)