Passive verification of the strategyproofness of mechanisms in open environments

Consider an open infrastructure in which anyone can deploy mechanisms to support automated decision making and coordination amongst self-interested computational agents. Strategyproofness is a central property in the design of such mechanisms, allowing participants to maximize their individual benefit by reporting truthful private information about preferences and capabilities and without modeling or reasoning about the behavior of other agents. But, why should participants trust that a mechanism is strategyproof? We address this problem, proposing and describing a passive verifier, able to monitor the inputs and outputs of mechanisms and verify the strategyproofness, or not, of a mechanism. Useful guarantees are available to participants before the behavior of the mechanism is completely known, and metrics are introduced to provide a measure of partial verification. Experimental results demonstrate the effectiveness of our method.


INTRODUCTION
Recent work in computational mechanism design has uncovered many interesting methods for decision-making in domains with self-interested agents, each with private information about its utility for different outcomes [7; 1; 2; 15, e.g.].Common to much of this work is the drive for strategyproof mechanisms; i.e., direct-revelation mechanisms in which truthful reporting of private information is a dominant strategy equilibrium.This is useful because it side-steps the semantic and computational difficulties of more intricate equilibrium concepts, such as Bayes-Nash equilibrium.Many have argued that strategyproofness is important to encourage the adoption of agent-mediated decision making: with users able to trust autonomous agents to "do the right thing" because agents have a simple and provably optimal strategy.
Our work is motivated by a future in which multiple entities (e.g.firms, people, organizations, network services) are able to deploy decision mechanisms in an open computational infrastructure.These mechanisms can be used, for instance, to coordinate purchasing decisions, allocate resources, schedule bandwidth, or form coordinated plans of action.
A new issue arises for mechanism design in open settings.Why should participants trust that a mechanism has the strategyproof property that it claims?We address this problem, proposing and describing a passive verifier, able to monitor the inputs and outputs of mechanisms and verify the strategyproofness, or not, of a mechanism.Thus, our focus is not on the design of new mechanisms but on the verification of strategyproofness.Indeed, without verification careful design is worthless: it is only when a mechanism is both strategyproof and known to be strategyproof that participants gain the benefits of simple strategies and mechanisms behave as designed.Thus, verification is in the interest of both designers and participants.
The passive verifier that we design exploits a price-based characterization of truthful mechanisms, and is inspired by the work of Gui et al. [11] on graph-theoretic characterizations of truthful mechanisms.We formulate the verification problem as one of checking for feasible solutions to a constraint satisfaction problem defined on a graph representing the rules of a mechanism.Verifiers are situated in the computational infrastructure and intermediate between participants and a mechanism.A verifier has the power to veto the outcome of a mechanism when the mechanism is proved to violate strategyproofness.
We identify techniques to both accelerate the process of verification as well as reduce the space complexity of verification.The idea is to place (weak) restrictions on the space of allowable mechanisms in order to simplify the task of verification.We provide three illustrations of this idea.First, we introduce the notion of summarization, which requires that a mechanism place restrictions on the complexity of its pricing rule, in identifying some subset w < n of n participants, that define the price faced by each agent.Summarization provides an exponential reduction in the space complexity of verification, from O(md n−1 ) to O(md w ), in domains with m alternatives and with d possible agent valuations.Second, when a mechanism is required to satisfy a standard property on payments, that we term natural payments, we can leverage constraint propagation to further accelerate verification.Third, when a mechanism is required to satisfy a property of envy-freeness, reasonable for simple domains, additional acceleration is achieved.
The biggest challenge is to provide useful feedback to participants before the complete input space for a mechanism has been observed.Here, we identify a property that allows the verifier to guarantee that an agent's best strategy is truthful reporting (conditioned on the mechanism passing the verifier) even though only a subset of possible inputs has been observed.We identify a weaker condition that ensures that truthful reporting maximizes the worst-case utility of a participant against an adversarial (but ultimately non-strategyproof) mechanism in early stages of verification.We also provide two metrics, namely probability-ofstrategyproofness and price-flexibility, to measure the degree of strategyproofness that is ensured for a mechanism that is still not completely verified.These metrics facilitate an empirical study in which we demonstrate the speed of verification (or failure of verification when using a weaker, baseline method) on various mechanisms, both strategyproof and non-strategyproof.

Related Work
Plenty of prior work has leveraged the methods of cryptography and secure function evaluation [18; 5, e.g.] to achieve absolute trust (i.e. the rules of a mechanism are published and correctness is verifiable to any party).When designing an auction that will be used for a multi-million dollar allocation of government bonds the overhead of cryptographicallysecured proofs is well justified.Our motivation comes from a vision of dynamic, open environments that can support a multitude of frequent, perhaps even mundane, decisions.For instance, we wish to enable infrastructure that can support the minute-by-minute allocation of computational services, the buying and selling of limited-play songs, and services to make dinner reservations, solicit information about flight prices, and find answers to trivia questions.
Similarly, there is a large literature on the use of logic formalisms to verify the correctness of mechanisms and other institutions [20,21,8].Here, a logic specification makes explicit the rules of a mechanism and properties such as strategyproofness can (in principle) be established through methods such as model checking.Of particular relevance is the work of Guerin and Pitt [10], who share the same motivation of allowing for the deployment of trusted mechanisms in open environments.Moreover, these authors discuss the use of "sentinel agents" which can be used to verify the properties of mechanisms at run time.Here, we are only interested in the verification of strategyproofness and not in the verification of other properties.Second, we deliberately eschew logic-based formalisms because model checking is intractable for appropriate logics [22], and because logic formalisms are a bad fit for quantitative properties such as strategyproofness and for the methods of combinatorial optimization that are important for internal decision making by mechanisms.

PRELIMINARIES
Formally, a mechanism consists of a social choice function f : V1 × . . .× Vn → A, that chooses an alternative f (v) = a ∈ A from a space of alternatives A, and a payment function p : V1 × . . .× Vn → R n that defines a payment pi(v) by each agent.Here, vi ∈ Vi is the valuation of agent i, where vi(a) ∈ R is the value of agent i for alternative a.Also, denote v = (v1, . . ., vn) ∈ V and v−i = (v1, . . ., vi−1, vi+1, . . ., vn).The valuation (or type) of an agent is private information, although we assume the type space Vi of each agent is common knowledge.Define Ei(a) = {d ∈ A|vi(a) = vi(d), ∀vi ∈ Vi} 1 , and let R f (v−i) denote the range of possible alternatives given social choice function f and reports v−i from all but one agent.Now we define a strategyproof mechanism.
Definition 1.A mechanism M is strategyproof if for all agents i with type vi and for every Well-known examples of strategyproof mechanisms include the family of Vickrey-Clarke-Groves(VCG) mechanisms [13], of which the second-price Vickrey auction is a special case.Other examples include: the greedy mechanism for single minded agents due to Lehmann et al. [15], mechanisms for one-parameter agents [1, e.g.], and truthful and competitive auctions for digital goods [7].
The key observation that we use for verifying the truthfulness (or strategyproofness) of mechanisms is that they must be price-based (see for instance [3,23]).
Theorem 1.A mechanism M =< f, p > is strategyproof if and only if for every agent i and every v−i: (A1) the mechanism charges an agent-independent price pi(a, v−i) whenever alternative a is selected (A2) for any report vi, any v−i, the mechanism chooses an alternative Here is some intuition for the sufficiency of (A1) and (A2) for strategyproofness: an agent cannot change the price that it faces (A1), and the mechanism maximizes its utility with respect to its reported valuation given these prices (A2).A simple argument can also be constructed for the necessary direction.

Problem Definition
Our verifiers act as lightweight trusted intermediaries (or "wrappers") that are situated as a default interface between published mechanisms and participants.A verifier receives bids from agents, passes them on the a mechanism and then receives the outcome and payments from the mechanism.At this point the verifier checks that the mechanism has not violated strategyproofness, and if this is OK it will pass the outcome on to the agents.Otherwise the verifier can exercise veto power on the outcome.
A mechanism is treated as a black box, i.e., the verifier can only observe a sequence of inputs, v 1 , v2 , . . .v k , . .., where v k denotes the reported valuations of the agents in the kth run of the mechanism, and a sequence of outputs, < a 1 , p 1 >, < a 2 , p 2 >, . .., < a k , p k >, . . .where < a k , p k > denotes the alternative and vector of payments produced by the mechanism in the kth run.
Definition 2 (passive verification problem).An online decision problem in which a verifier observes a sequence of inputs and outputs to a mechanism (assumed fixed and deterministic) and determines whether the mechanism is strategyproof or not strategyproof.
A useful verifier will also reject a non-strategyproof mechanism quickly, and provide guidance on whether or not a mechanism is likely to be strategyproof even before a final "reject" or "accept" is generated.We also demonstrate that a verifier can prove that a mechanism is strategyproof given the private valuation of a bidder, even before the mechanism has been fully verified.
Here are our main assumptions: 1.The space of alternatives and the type space are finite.
2. The verifier is trusted, is able to observe all inputs and outputs to a mechanism, and has veto power on any decision made by the mechanism (e.g. as soon as some decision, perhaps this decision, provides proof that the mechanism is not strategyproof ).

3.
Once the mechanism has decided on an alternative and payments for an instance, it can no longer change that decision in the future. 2  4. The mechanism is anonymous, The first assumption is needed to ensure that a sound and complete verifier can operate with bounded memory requirements.The second assumption is essential to our approach to passive verification.The third assumption ensures that the entire history of instances is always relevant for validating or invalidating strategyproofness.The final assumption allows the valuation profile, v−i, of all agents except i to be treated as an unordered set of n − 1 elements (allowing for repeated elements).This is helpful computationally, but should be relaxed in settings where anonymity is not provided (e.g.optimal auctions [17] where prior information about bidder valuations is used to bias the outcome of a mechanism.)These anonymous semantics for v−i will be assumed for the rest of the paper.

PASSIVE VERIFICATION
In this section we define a set of rules that can be implemented by a passive verifier, and prove that verification with these rules is sound and complete.A constraint network formulation is given for the rules, which leads to a concrete algorithmic instantiation for a verifier.

SimpleChecker: Rules for Verification
Fix v−i.By condition (A2) from Theorem 1, a strategyproof mechanism must choose an alternative SimpleChecker.Initialize history H to an empty history.
1.For a new instance < v k , a k , p k >, and for each agent i, consider history f H (v k −i ) and if non-empty: : 2. If any check fails, then reject the mechanism, else update history for v k −i as necessary (i.e.whenever a new vi and/or new alternative was observed).
3. Once all vi ∈ Vi for all v−i ∈ V−i have been observed, then pass the mechanism.
for all vi ∈ Ga.Hence, we get the following inequality: Similarly, considering cases where f (vi, v−i) = b, we get: Combining the two, for each pair < a, b >, we have: This is well known, see for instance Gui et al. [11] and Lavi et al. [14], and suggests the following simple verification procedure.
Let H denote the history of instances currently available to the verifier.Define the following: v ∈ v H denotes bids v ∈ V N received so far; vi ∈ v H i (v−i) denotes the bids from agent i that have been observed for v−i from the other agents; vi ∈ v H i (a, v−i) denote the bids received from agent i with the additional restriction that alternative a (or an equivalent) was selected; a = f H (v) denotes the alternative selected given input v (it is sufficient to choose any one of a set of equivalent alternatives); f H (v−i) ⊆ A denotes the set of alternatives selected for v−i, and p H i (a, v−i) denotes the payment made by agent i given alternative a and v−i.For every new instance we first check whether the exact same value profile has been seen before.If this is the case then the same alternative must be selected.By case (b), if a k (or equivalent) has been seen before for v k −i then by (A1) the payment by i must be the same.Case (c) ensures that previous agents could not have done better given this new information: if a k is a new alternative for v k −i then the price on this alternative must be high enough for (A2), and thus for inequality (3) to hold for bids that have already been observed.Case (d) ensures that the current agent could not have done better given the current history: if v k i is a new bid for v k −i then the price on this alternative must be low enough for (A2), and thus for inequality (4) to hold for other alternatives that have been observed.
Example 1.Consider an auction mechanism for an allocation problem with multiple identical items.Suppose that the sequence of instances in Table 1 are observed (all with two bidders and two items).Instance 1 indicates that bidder 1 has value 4 for 1 unit and 8 for 2 units.Bidder 2 has value 4 for 1 unit and 9 for 2 units.The allocation gives both units to bidder 2, and bidder 2 makes payment 8.The auction implements the VCG mechanism for instances 1-3 but deviates in instance 4 (The VCG mechanism would choose the same allocation but with payments (0, 9)).
Consider the build-up of history for v−i = (4, 9), and consider allocations in which bidder i receives 0, 1 or 2 items.Let pi(n) denote the price that agent i faces when it wins n items.After instance 1, we get pi(0) = 0.After the second instance, the verifier learns that pi(1) = 5, and checks that the constraints (c) pi(1 are satisfied.After instance 3, agent i wins one item and pays the price of 5, so the verifier checks that (b) p 3 i (1) = pi(1) = 5 holds.After instance 4, the verifier learns pi(2) = 10, and checks that (c) However, the verifier also checks for the constraint (d) , which is violated (the price is too high).Hence, this mechanism is rejected by Sim-pleChecker after the fourth instance.

Establishing Soundness and Correctness
The first task is to establish soundness and correctness of the verifier.We hold these properties to be necessary for an (exact) verifier, although insufficient to show the utility of passive verification.Soundness simply proves that SimpleChecker will detect the failure of strategyproofness eventually, once all inputs are observed.The main challenge is to provide intermediate feedback, discussion of which is delayed until Section 5.
Theorem 2 (soundness).The rules as defined by Sim-pleChecker will detect a non-strategyproof mechanism even if all agents are not truthful as long as all inputs are eventually observed.
Proof.A mechanism is not strategyproof if either: Suppose toward contradiction that a non-strategyproof mechanism passes the SimpleChecker after all possible reports v ∈ V are observed.First, suppose ¬ (A1).This is not possible because check (b) of SimpleChecker would catch the price deviation.Second, suppose ¬ (A2), such that there exists some i, v−i and vi, . By (7), and step (d) of SimpleChecker would fail.A contradiction.If l > k, then when instance l is observed, and step (c) would fail.A contradiction.
Theorem 3 (correctness).The rules as defined by SimpleChecker will never halt a strategyproof mechanism even for agents that are untruthful.
Proof.Suppose toward contradiction that a strategyproof mechanism M is rejected by SimpleChecker after k instances.Since M satisfies (A1), it will not fail step (b).So it either fails (c) or fails (d).Suppose it fails (c).Then there exists A2) and a contradiction.Now, suppose the mechanism fails (d).Then there exists A2) and a contradiction.

Passive Verification via Constraint Networks
An algorithm for SimpleChecker can be identified by reformulating the task as that of checking for a feasible solution to a constraint network.
We gain two main advantages from formulating the problem in terms of constraint networks.First, we can leverage standard data structures (e.g.linked-lists) and standard algorithms (e.g.all-pairs shortest path algorithms) to construct a passive verifier.Second, we can gain additional accelerations by introducing constraints between networks to capture additional problem structure (see Section 4.2).
For each v−i, we construct a constraint network in which each node in the network is associated with an alternative and arcs in the network impose constraints on the difference in prices between pairs of alternatives.Each network contains a single node for each set of equivalent alternatives; e.g. one node for all allocations in which agent i receives bundle of goods S, irrespective of the allocation to other agents.In addition to binary constraints, each node is also annotated with an exact price, once known. 3 Recall that every strategyproof mechanism must satisfy inequalities (3) and ( 4).In order to capture this requirement for some pair of alternatives < a, b > given current history H, a directed arc b → a is labeled with a weight w(b, a) Similarly, a directed arc a → b is created and labeled with weight w(a, b).In addition, each node that represents an observed alternative can be annotated with the price.
As described, our constraint network is of the same form as a Simple Temporal Problem (STPs)(Dechter et al. [6]).Prices take the role of time and the weights on arcs are now bounds on the difference between prices.The existence of prices that form a feasible solution to this network is equivalent to the nonexistence of negative-length cycles in the constraint network.

Theorem 4. [6] A given STP T is consistent if and only
if it has no negative-length cycles.
Thus, we can check for the existence of prices satisfying all constraints by checking for the existence of negative-length cycles using an all-pairs-shortest-path algorithm.We also refer to this process as tightening the network.
Let N denote the current set of networks (one for each v−i that has been observed).New networks are introduced dynamically (for new v−i sets) and a new node is introduced on a network when an additional alternative is observed.
Algorithm NetworkChecker implements the sound and correct rules defined in SimpleChecker.If a set of feasible prices exist, by step (c) and step (e) of NetworkChecker, ) for all existing nodes a. Rewriting, we get , which is precisely the condition that is checked in step (c) of the SimpleChecker.Now consider step (d), in combination with step (e), of NetworkChecker: if an arc from node a to node a k already exists, and the arc has weight w < v k i (a k ) − v k i (a), then we know that p a k i − p a i ≤ w even before observing instance k, and no constraints are 3 Compared with the directed graph formalism introduced in Gui et al. [11] for reasoning about strategyproof mechanisms, the constraint network of a passive verifier is typically incomplete (only containing a subset of alternatives) and associates fixed prices with alternatives that have been observed.These prices imply constraints on future prices.In comparison, Gui et al. use the constraint network to reason (in analysis) about whether any assignment of prices is possible.
1.For a new instance < v k , a k , p k >, and for each agent i, if there is a constraint graph G ∈ N for v k −i work with this graph.Otherwise, create an empty graph G and add to set N .Then: . (e) tighten the network and check for feasibility.
2. If any check fails, then reject the mechanism, else update the history as necessary.
3. Once all vi ∈ Vi for all v−i ∈ V−i have been observed then pass the mechanism.
updated.Otherwise, we have a new (or tighter) constraint on the set of feasible prices, namely, ) for all a such that w(a, a k ) is updated.Combining the two cases, we get , which is the condition checked in step (d) of the SimpleChecker.

ACCELERATED VERIFICATION VIA STRUCTURAL REQUIREMENTS
The biggest potential shortcoming of this approach to verification is that the space complexity quickly becomes untenable: there are d n−1 subnetworks, for a type space of size d and n agents, and thus the space complexity is exponential in the number of agents.
In order to address this problem we propose that the passive verifier impose structural requirements.We consider three kinds of structural requirements: (a) summarization, (b) natural payments, (c) envy-freeness.These are illustrative of a more general approach which is to restrict the space of implementable mechanisms in order to allow for efficient verification.More than just reducing the space requirements for verification, these structural requirements also accelerate verification by allowing for stronger inference and more rapid proofs of non-strategyproofness.
A strategyproof mechanism may wish to volunteer structural requirements in order to facilitate faster verification.But, given that the concern in verification is to identify the "bad apples" then it is in general more appropriate for the verifier to require certain additional properties, especially properties that appear to hold for many plausible mechanisms.We will comment on the restrictiveness of each property as it is introduced.

Summarization
Summarization provides an exponential reduction in memory and computational requirements on the verifier and also significantly accelerates the process of verification.

Definition 3 (valid summarization function).
Given reports v−i, a summarization function s selects a subset s(v−i) of reports, and is valid when pi(a, v−i) = p i (a, s(v−i)) for all a, all v−i, for some price function p i .
Example 3. In a single-item Vickrey auction the price that an agent faces is determined by the highest bid among bids from other agents and s(v−i) = {max j =i {vj }} is a valid summarization function.In this case, the induced price rule p i (i wins item, s(v−i)) = x given s(v−i) = x and p i (i does not win item, s(v−i)) = 0.
Example 4. Consider a combinatorial auction with singleminded bidders.A bidder is single-minded if there exists a set g of goods and value b such that v(g ) = b if g ⊆ g , and v(g ) = 0 otherwise.The LOS mechanism [15] consists of the greedy allocation rule and the greedy payment rule.The greedy allocation rule works as follows: bids are sorted in decreasing order according to the average amount 5 of a bid vj =< gj , bj >, defined as b j |g j | , where |gj | is the number of goods in the desired set gj of agent j, and bj is her value for gj.Then given this sorted list L, each bid is examined in order and is granted if and only if it does not conflict with any of the bids previously granted.For each vj ∈ L, define the set Dj = {i|i > j, gi ∩ gj = ∅, ∀l < i, l = j, l th bid granted ⇒ g l ∩ gi = ∅}.Note that i > j . Dj is the set of indices of the bids that 5 The average amount can be replaced with any norm that satisfied bid-monotonicity, i.e., is nondecreasing in bi and norm(< g, b >) ≥ norm(< g , b >) ∀g ⊆ g .are denied, but would have been granted if it were not for the presence of vj .Under the greedy payment rule, if agent j's bid is granted and Dj = ∅, j pays |gj| b i |g i | where i = min Dj .In other words, j pays the average amount of the first bidder whose bid was denied due to j, per good won.Otherwise, j pays 0. In the LOS mechanism, the summarization function s(v−i) = {vj | ∀k < j, k = i, g k ∩ gj = ∅}.If VCG payments are used instead, the set s(v−i) is given by first sorting the set v−i according to the norm used in the LOS allocation rule and then removing each bid whose set of desired items is a superset of the set of desired items of any bid that appears before it in the ordered list.
Summarization information can be defined by a mechanism incrementally, by identifying a subset of values v−i that define the prices to agent i for each instance.In this case, this is done by extending the interface between the verifier and the mechanism.Summarization functions can also be defined statically by instantiating an explicit function.As long as the summarization function is constant, then passive verification remains sound and correct.Theorem 5. A passive verifier that implements rules Sim-pleChecker will continue to detect a non-strategyproof mechanism and continue to pass any strategyproof mechanism when used in combination with a fixed summarization function.
Proof.See appendix.
The following example shows that summarization can facilitate faster identification of a non-strategyproof mechanism.
Without summarization none of the v−i subnetworks have more than one node and the mechanism would not have been rejected until more instances were seen.
Summarization also provides an exponential improvement in space complexity.Let d = maxi|Vi|, m = |A| and let n denote the maximal number of agents.Without summarization, we need d n−1 subnetworks, each with at most m nodes and thus O(md n−1 ) nodes.Theorem 6.With summarization there are O(md w ) nodes across all subnetworks, where w is the maximal number of agents required in a summarization, i.e. w = maxi,v i |s(v−i)|.
Thus, the space complexity is exponential in the worstcase number of agents required for summarization instead of the worst-case number of agents.Example 6.Consider the single item Vickrey auction with 4 agents, where |Vi| = 10.In this case, only the highest bid among bids of other agents matters.Without summarization, we would need to keep 1000 subnetworks, while with summarization, we only need to keep 10 subnetworks.

Natural Payment Functions
So far we have focused on the constraints within a single v−i subnetwork.The additional structure provided by natural payments, allows internetwork constraints which improve the speed with which a mechanism can be validated as strategyproof, or proved not to be strategyproof.
Consider again inequalities ( 3) and ( 4).Following Lavi et al. [14], define Definition 4 (natural payment functions).A mechanism has a natural payment function if the agent-independent price function satisfies for all pairs < a, b >.
The natural payment function is often satisfied by known mechanisms, for instance, it is typically satisfied by the VCG mechanism (this depends on the value domain) and the LOS mechanism.Natural payments permit the introduction of internetwork constraints, as demonstrated by the following lemma.
Lemma 1.Consider a strategyproof mechanism with natural payment functions.Suppose for an alternative a, v−i and v −i are such that ∀j = i, either Proof.See appendix.Thus, given a constraint in the subnetwork for v−i and some alternate reports v −i by all agents except one that satisfies the condition of Lemma 1, then we can add a corresponding constraint to the subnetwork for v −i , or vice versa.The additional constraint provides the following kind of inference: given pi(a, v−i) ∈ [c, d] for some constants c and d, then pi(a, v −i ) ≤ d.To illustrate the condition of Lemma 1, consider the case of single-minded bidders.Here, the condition is satisfied if each agent j = i bids for the same bundle in v −i as in v−i, and all agents j = i who win in v−i submit the same or larger value in v while losing agents do not increase their bid in v .
Modified NetworkChecker.Add a new step (d1) between steps (d) and (e): and v−i, and repeat previous step.
Example 7. Consider a strategyproof mechanism with 2 agents and two items s and t, and suppose a = {agent 1 wins s, agent 2 wins t }.If v 2 is such that agent 2 values alternative a higher relative to all other alternatives, then if the alternative is a for both v and v , agent 1 must not pay a higher price under v .
We can implement the internetwork constraints as follows.First, define the indicator function: Then, for each alternative a and valuations vi, v−i such that I(a, v−i, v −i ) = 1, we add the following constraints: pi(a, where Fi(v−i) denotes the space of feasible prices defined by the constraint network.Note that max

Envy-freeness
Our third example to illustrate the power of structural requirements is the property of envy-freeness [9].
Let fi(v) denote the allocation of goods to agent i defined by social choice function f .
This property is easily verified by checking that there is no agent that prefers an outcome-price pair that was received by another agent.Thus, this can be checked by adding an additional set of internetwork constraints.
Many strategyproof mechanisms are also envy-free.For example, VCG mechanisms in domains with superadditive valuations satisfy the property of envy-freeness [19].As another example, it is easy to show that envy-freeness holds for any strategyproof mechanism in the domain of singleminded combinatorial auctions that is fair, which requires that for any set w and any two agents i and j such that the bids are bi = (w, vi) and bj = (w, vj ) where vi > vj , then agent j should not win.The LOS family of mechanisms satisfy fairness and therefore all LOS mechanisms are envy-free.

PROVIDING INTERMEDIATE FEED-BACK TO PARTICIPANTS
We describe in this section some intermediate feedback that can be provided to participants to guide their behavior even before a mechanism's strategyproofness (or lack thereof) is completely established.This is the main technical achievement of our work.

Partial Strategyproofness
First, recall that the verifier is able to check the outcome of a mechanism before the outcome is implemented and before payments are collected from agents and veto the result if the input-output instance provides proof that the mechanism is not strategyproof.This veto power is important in establishing the following useful result.
Theorem 7. Given history H, and consider an instance in which the reports of agents except i are v−i and instance (vi, v−i) has been observed by the mechanism for agent i's true valuation vi.In this case, and conditioned on the mechanism passing the verifier in this instance, then agent i's best-response is to report her true valuation.
Proof.Fix v−i.Condition on the case that all checks in the verifier pass.First, if the agent reports vi then the mechanism must make the same decision as previously, by checks (a) and (b) in SimpleChecker.Now suppose the agent reports some v i = vi.We argue that this cannot improve the agent's utility.Case 1.The mechanism selects an alternative a already observed.Now we argue that the agent must actually prefer the outcome, a, that would have been selected given truthful report vi.Either a was observed after a in which case check (c) would have ensured that a was preferred to a by an agent with type vi, or a was observed after a and check (d) would have ensured that a was preferred to a by an agent with type v .Case 2. The mechanism selects a new alternative, not previously observed for v−i.In this case, because vi was already observed (with outcome a) then to pass check (c) it must be the case that outcome a would preferred by an agent with true type vi.This concludes the proof.Note that the above theorem continues to hold even if the mechanism is ultimately not strategyproof: it is still rational for an agent to bid her true type if vi and v−i (reports by the other agents) have already been observed.Moreover, the observation leads to the following two corollaries.Here, when truthful reporting is an ex post Nash equilibrium given knowledge Ṽ ⊂ V about joint types, then every agent must maximize its utility by reporting its true type in equilibrium as long as every other agent is truthful and whatever the actual joint valuation v ∈ Ṽ .
Corollary 1.Given history H, let Ṽ ⊂ V denote some subspace of joint type space V for which all joint inputs v ∈ V have been observed.Given knowledge that v ∈ Ṽ , and conditioned on the mechanism passing the verifier in this instance, then truthful reporting is an ex post Nash equilibrium.
Corollary 2. Given history H, and considering agent i with true type vi, then if (vi, v−i) has been observed by the verifier for all v−i, then conditioned on the mechanism passing the verifier in this instance truthful bidding is a dominant strategy for bidder i.
We see that there are reasonable conditions under which a partially informed agent (e.g. with information about the space of possible types of other agents) should report its type truthfully even before the strategyproofness of a mechanism is fully verified.Moreover, these observations suggest that it will be useful for a verifier to publish the set of types for which the mechanism is provably strategyproof.Note also that these results can all be rephrased in terms of summarizations of reports from other agents when summarization is used.
A somewhat weaker guarantee is also available to a bidder in the case that report v−i has been observed but not in combination with the agent's type vi.This guarantee is provided in the context of an adversarial mechanism, which seeks to minimize the utility to agent i while passing the verifier in this instance.The earlier guarantees are provided for any mechanism.
Theorem 8. Given history H, then agent i maximizes her worst-case utility against an adversarial mechanism by reporting truthfully whenever report v−i has been observed, contingent on the mechanism passing the verifier in this instance.
Proof.Worst-case utility refers to the minimum utility achieved over all possible choices that can be made by the mechanism in responding to the input, while still passing the verifier.Fix vi and v−i ∈ H. First suppose the agent is truthful.In this case the best the adversary can do, in terms of minimizing the utility to the agent, and still pass the checks (e.g. in SimpleChecker) is to select the preferred alternative for the agent in a ∈ f H (v−i). Choosing some less-preferred alternative in f H (v−i), or some lesspreferred alternative outside of f H (v−i) would violate check (d) in SimpleChecker.Now suppose the agent is untruthful and reports v i = vi.In this case, the mechanism can always choose one of the current alternatives and pass since check (c) would not be triggered and check (d) is satisfied by choosing a ∈ f H (v−i), the most-preferred alternative with respect to v i .Note that the alternative selected when reporting vi is at least this good.Also, if the mechanism chooses to select some new alternative then this must necessarily be even worse for the agent (since the mechanism is adversarial).This completes the proof.

Metrics for Partial Verification
We introduce two quantitative metrics for the degree to which a mechanism's strategyproofness has been partially verified.First, Corollary 1 can be used to define a lowerbound on the probability that truthful reporting is an ex post Nash equilibrium, given history H and given a distribution on types of bidders.
Corollary 3. Given history H and probability distribution g on agent types, then truthful bidding is an ex post Nash equilibrium conditioned on the mechanism passing the verifier in this instance, with probability: where g(v) is the probability that agents have types v and I H (v) is an indicator function, and equal to 1 if and only if v ∈ v H .
We refer to this metric as the probability-ofstrategyproofness.In combination with summarization, this becomes: where I H (s(v−1), . . ., s(v−N )) is an indicator function, and equal to 1 if and only if summarizations s(v−i) for all i have been observed by the mechanism given history H.A probability-of-strategyproofness metric can also be defined for a single agent i, where the average is computed with respect to the marginal distribution on the reports of agents = i, given agent i's type vi.
A complementary measure is provided by the priceflexibility metric.This metric takes into account the range of prices still available to a mechanism in setting prices on alternatives not yet observed.Let Feas H (v) ⊆ A denote the set of alternatives that can be selected without failing the verifier given input v and history H.If v has been observed this is the singleton containing the alternative f H (v) that was previously selected.But, even when v has not been observed we have restrictions on alternatives that can be selected.There can be alternatives for which, if selected, there is no price that can be assigned to satisfy checks (c) and (d) in the verifier for all agents.
For a ∈ Feas H (v), we define the price flexibility on a, denoted flex H (v, a) as: This represents the range of prices available to the mechanism without violating the checks in the verifier, and is non-empty since a is in the feasibility set.Now, for valuation profile v = (v1, . . ., vN ), define the price-flexibility, as: where Notice that if all inputs v−i have been observed for vi then PF H (vi) = 0.This is as we would expect: a fully-verified mechanism has no flexibility and must continually make the same decisions and assign the same prices as in the past.

EXPERIMENTS
We present experimental results to demonstrate that the verifier can impose useful restrictions on the mechanism design space (via summarization, natural payments, and envyfreeness) and increase the confidence in truthful mechanisms and accelerate the detection of failure in manipulable mechanisms.
For illustrative purposes, the experimental results are presented for a mixture of known and invented mechanisms.We consider both Vickrey and first-price auctions for the allocation of multiple identical items.In addition, we consider single-minded bidders in a combinatorial auction (with multiple non-identical items) and the LOS mechanism [15] as well as the LOS greedy allocation rule in combination with VCG payments (this mechanism is called greedyVCG, and is manipulable).We also find it useful to construct an artificial, manipulable mechanism for single-minded bidders using local search techniques combined with VCG payments.We call this mechanism localVCG.
To provide a baseline we compare with an algorithm that simply checks for violations of (A1).This algorithm is sound but not complete.Actually, it completely fails to reject the greedyVCG mechanism, which only makes errors of type (A2).On the other hand, it is sound against first-price auctions, which only violate (A1).The localVCG mechanism is useful because it violates both (A1) and (A2), permitting a comparison between the baseline and our verification methods.

Multi-item Auctions
We consider a first-price multi-item auction with g identical goods and construct agent valuations by defining the marginal value of the kth item for agent i as independently drawn from a uniform distribution over {0, 1, . . ., z − 1}.Here, z denotes the max number of values in the value domain for any one of these items.With g items the size of the type space for each agent is m = z g .We consider the verifier both with and without the internetwork constraints that arise from the requirement of natural payment functions (which hold for the VCG mechanism in this auction environment), but do not consider the use of summarization.
Figure 2 illustrates the number of instances observed before failure.Fixing z = 5, we first vary the number of items g from 1 to 10 for n = 3 agents (i.e. with m = z g = 5 g types per agent, m 3 different instances, and at most m 2 networks).Second, we vary the number of agents n from 1 to 10 for g = 3 items (i.e. with m = 5 3 = 125 types per agent, 125 n instances and at most 125 n−1 networks).For each experiment each point represents an average over 10 runs.The NetworkChecker is tested with and without internetwork constraints due to natural payments, and labeled base and prop accordingly.
Observe that internetwork constraints considerably speed  up the verification process.In addition, since at most n new nodes are created for each instance, the number of nodes in the graph is roughly proportional to a multiple of n times the number of instances observed before failure and the running time and space complexity of the verifier is roughly linear in the number of instances observed.Thus, imposing natural payments and enabling internetwork constraints improves space and time complexity.

Single-Minded Bidders
For single-minded bidders we generated a distribution of instances by using the L4 Legacy distribution of the Combinatorial Auctions Test Suite (CATS) [16].We discretized the type space by rounding the values to the nearest 250 (the values fell into a range of [0, 3000]).There are g distinct items, and each agent can choose any subset of the g items, and have a value ∈ {0, 250, . . ., 2750, 3000} for the desired bundle.The results are averaged over 10 trials.
In this experiment we impose both summarization and envy-freeness, which is a reasonable structural requirement in this environment (as noted in Section 4).As a summarization function we adopt the summarization function defined in Example 4 for the VCG mechanism, which is also valid for the LOS mechanism and therefore reasonable to impose.We also compare with the benchmark algorithm, which fails to catch the non-strategyproofness of greedyVCG but is effective for localVCG.
Figures 3 (a) and (b) illustrate the results for Network Checker ("base"), and also with summarization ("sum") and with the additional structure provided by envy-freeness ("envy-free"), which is used here without summarization 6In Figure 3 (a) we vary the number of agents n while fixing the number of items at g = 3 (m = 137 types, since there are 2 3 − 1 bundles of 3 items).In Figure 3 (b) we vary the number of items g while fixing the number of agents at n = 4 (m = 13 2 g −1 ).The use of summarization provides a significant speed-up, and the imposition of envy-freeness makes the verification of a non-strategyproof mechanism almost immediate.As expected, summarization becomes more important as the number of agents increases, and its benefit increases, because without summarization it gets less and less likely that the history contains observations relevant to a new instance.Figures 3 (c) and (d) illustrate the results for Network Checker with and without summarization and the benchmark algorithm with and without summarization, this time with the localVCG mechanism.The same summarization function is adopted as above. 7We again vary the number of agents, and then the number of items.In comparison with the benchmark algorithm, the verifier detects non-strategyproofness more quickly both with and without summarization, with the benefit over the benchmark with summarization most noticeable as the number of items increases.

Computing Metrics
We have also experimented with the probability of strategyproofness (PrSP) and price flexibility (PF) metrics defined in Section 5.These metrics allow the verifier to provide feedback to agents before finally verifying that a mechanism is strategyproof (or not).We present results for single-minded combinatorial auctions and the (strategyproof) LOS mechanism.Monte Carlo analysis is used to estimate PrSP and PF over all possible valuations v ∈ V , where the valuations are drawn according to the CATS Legacy distribution.
Figure 4 (a) displays the estimated PrSP for LOS in a domain with 4 agents and 3 items.The results are averaged over 100 trials.We consider NetworkChecker without summarization and then with the two summarization functions described in Example 4 (the first is the more general VCG rule, the second is the rule designed for LOS.) Summarization greatly improves the probability that truthful reporting is a dominant strategy at any given point in the verification process, especially the second summarization function which is designed for LOS.Thus, in deploying the LOS mechanism it would be useful to instruct the verifier to adopt this strict summarization function in order to accelerate verification.
Figure 4 (b) displays the estimated PF for LOS in a domain with 3 agents and 3 items.The results are averaged over 300 trials.We compare NetworkChecker with and without the imposition of natural payments and thus with and without the availability of internetwork constraints.We find that internetwork constraints are somewhat effective in reducing the price flexibility, although the effect is not as pronounced as that of summarization on the probability of strategyproofness.

CONCLUSIONS AND FUTURE WORK
We have introduced the problem of verifying whether a mechanism is strategyproof and provided a constraintnetwork based algorithm for verification.The verifier is able to reject mechanisms that are not strategyproof based on the violation of constraints imposed by strategyproofness on the price space.Experimental results demonstrate the potential for accelerated checking when there is additional structure to exploit and also suggests metrics that can be informative in guiding bidders before a mechanism is fully verified.Useful intermediate guarantees can also be provided to participants.
In practice, we believe that the most benefit from passive verification (as defined here) will be realized in combination ria.Local search is used first to approximate a solution with all agents.During this phase we also track the best solution found for each marginal economy.For payments, we then run local search with a smaller number of search steps on each of the marginal economies and adopt for the economy without i the best allocation discovered in the main search and in the explicit search in the marginal economy.In addition to failing (A2), localVCG can also fail (A1) because the price to a buyer can now depend on the search path adopted in the main economy, which can on the buyer's own bid.
with approximations that allow the verifier to forget some of its history and further reduce the space complexity of verifiers.In future work we will explore limited memory checking, where we let the verifier forget part of the history, and allow false positives; e.g. using data structures such as Bloom filters [4] to allow for fast checking.Another important avenue for future work is to introduce methodologies that can allow for a continuous type space.

10 (b) After instance 4 Figure 1 :
Figure 1: Constraint network for Multiple Identical Items with Two Bidders and Two items.
, b) denotes the weight on the arc from a to b in subnetwork v −i .

Figure 2 :
Figure 2: Number of instances before failure in a first-price multi-item auction.
Number of instances before failure in greedyVCG: Agents.Number of instances before failure in greedyVCG: Items.Number of instances before failure in localVCG: Agents.Number of instances before failure in localVCG: Items.
no agent can do better by misreporting her type, no matter what the other agents report.

Table 1 :
Now this constraint is violated (the price too low), and the mechanism is rejected after the fourth instance.Sequence of Instances: 2 Agents and 2 Identical Items ))} = 9.
or equivalent4) exists in graph G then check p k i equals price on node.(c)Otherwise,add node a k and assign price p k i to the node, and for every other node a in the graph: add a new arc from the new node a k to a with weight w= inf v i ∈v H i (a,v −i ) (vi(a) − vi(a k )) (d)add a new arc from every other node a to the new node a k with weight v k i (a k ) − v k i (a) if no arc exists.If one already exists, and