Effect of Blocking on Incentive Compatibility of Experiments

Abstract

This thesis considers the problem of incentive-compatible experimental design, the challenge of designing experiments when the treatments are applied by self-interested agents. Each agent strategically selects the version of its treatment that is most likely to make it appear superior and win the experiment. The experiment designer's goal is to incentivize agents to play their natural actions, the actions they would take in the absence of competitors, so that the experiment serves as an accurate preview of how the winning agent will behave once selected. The contribution of this thesis is analyzing how blocking, a technique from classical experimental design theory, affects the incentive compatibility of experiments. We consider mean- and variance-based strategies that agents can employ when selecting their treatment version. Additionally, we compare two different ways the designer could pick a winner based on the experimental data: a statistical significance and a score-based approach. We derive the incentives induced by blocking under these different combinations of strategy spaces and winner determination approaches. If the designer appropriately matches the winner determination approach to the class of strategies available to agents, a blocked design is more effective than an unblocked design at (1) encouraging agents to play their natural actions and (2) identifying the strongest agent.