Repeated Games Where the Payoffs and Monitoring Structure are Unknown

This paper studies repeated games with imperfect public monitoring where the players are uncertain both about the payoff functions and about the relationship between the distribution of signals and the actions played. We introduce the concept of perfect public ex post equilibrium (PPXE), and show that it can be characterized with an extension of the techniques used to study perfect public equilibria. We develop identifiability conditions that are sufficient for a folk theorem; these conditions imply that there are PPXE in which the payoffs are approximately the same as if the monitoring structure and payoff functions were known. Finally, we define perfect type-contingently public ex post equilibria (PTXE), which allows players to condition their actions on their initial private information, and we provide its linear programming characterization.


INTRODUCTION
THE ROLE OF REPEATED PLAY in facilitating cooperation is one of the main themes of game theory.Past work has shown that reciprocation can lead to more cooperative equilibrium outcomes even if there is imperfect public monitoring, so that players do not directly observe their opponents' actions, but instead observe noisy public signals whose distribution depends on the actions played.This work has covered a range of applications, from oligopoly pricing (e.g., Green and Porter (1984) and Athey and Bagwell (2001)) to repeated partnerships (Radner, Myerson, and Maskin (1986)) and relational contracts (Levin (2003)).These applications are accompanied by a theoretical literature on the structure of the set of equilibrium payoffs and its characterization as the discount factor approaches 1, most notably Abreu, Pearce, andStacchetti (1986, 1990 (hereafter APS)), Fudenberg and Levine (1994 (hereafter FL)), Fudenberg, Levine, and Maskin (1994 (hereafter FLM)), and Fudenberg, Levine, and Takahashi (2007).All of these papers assume that the players know the distribution of public signals as a function of the actions played.In some cases this assumption seems too strong: For example, the players in a partnership may know that high effort makes good outcomes more likely, but not know the exact probability of a bad outcome when all agents work hard.This paper allows for such uncertainty, and also allows for uncertainty about the underlying payoff functions.
Specifically, we study repeated games in which the state of the world, chosen by Nature at the beginning of the play, influences the distribution of public signals and/or the payoff functions of the stage game.The effect of the state on the payoff functions can be direct, and can also be an indirect consequence of the effect of the state on the distribution of signals.For example, in a repeated partnership, the players will tend to have higher expected payoffs at a given action profile at states where high output is most likely, so even if the payoff to high output is known, uncertainty about the probability of high output leads to uncertainty about the expected payoffs of the stage game.
Because actions are imperfectly observed, the players' posterior beliefs need not coincide in later periods, even when they share a common prior on the distribution of states.This complicates the verification of whether a given strategy profile is an equilibrium, and thus makes it difficult to provide a characterization of the entire equilibrium set.Instead, we consider a subset of Nash equilibria, called perfect public ex post equilibria or PPXE.A strategy profile is a PPXE if it is public, that is, it depends only on publicly available information, and if its continuation strategy constitutes a Nash equilibrium given any state and given any history.In a PPXE, a player's best reply does not depend on her belief, so that the equilibrium set has a recursive structure and the analysis is greatly simplified.2Moreover, like other forms of ex post equilibrium, PPXE are robust to variations in priors beliefs: A PPXE for a given prior distribution is a PPXE for an arbitrary prior. 3efore developing our general characterization of PPXE, we give a few examples.The first two examples consider special structures that make it easy to give explicit constructions of PPXE.One important fact these examples illustrate is that even though players start out not knowing the state, conditioning play on outcomes can indirectly allow the state to determine play and equilibrium payoffs.For example, if the outcome perfectly reveals the state, there can be PPXE where player 1's preferred perfect public equilibrium (PPE) is played from period 2 on in state ω 1 and player 2's preferred PPE is played from period 2 on in state ω 2 .The third and fourth examples are partnership games where the uncertainty concerns the productivity of effort.Here the nonconstructive characterization that we develop in the rest of the paper lets us show that the folk theorem applies in Example 3 while payoffs can be bounded away from efficiency in Example 4. In both of these examples, the distribution of outcomes can reveal the state and the folk theorem would hold in each state if the state were known; the key is that in Example 4 the states are "entangled," while in Example 3 they are not.
To characterize the limit of the set of PPXE payoffs as the discount factor goes to 1, we extend the linear programming characterization of the limit payoffs of PPE.That is, we show in Section 4 that the limit of the set of payoff vectors to PPXE as the discount factor goes to 1 is the intersection of the "maximal half-spaces" in various directions, where each component λ ω i of the direction vector λ corresponds to the weight attached to player i's payoff in state ω.The main new feature is that in a PPXE, the equilibrium payoffs are allowed to vary with the state and can do so even if the state does not influence the expected payoffs to each action profile-for example, there can be PPXE where player 1 does better in state ω 1 and player 2 does better in state ω 2 .Thus PPXE can involve a form of "utility transfer" across states.For this reason, the maximal half-space in these "cross-state directions" can be the whole space, while in FL the maximal half-space in each direction is bounded by the feasible set.
In Section 5, we use this characterization to give sufficient conditions for an "ex post" folk theorem: For any map from states to payoff vectors that are feasible and individually rational in that state, there is a PPXE whose payoffs in each state approximate the target map as the discount factor tends to 1.As in FLM, this theorem uses individual and pairwise full-rank conditions, and assumes that the set of feasible and individually rational payoffs satisfies a fulldimension condition.In addition, the theorem adds the assumption that for every pair (i ω) and (j ω) of individuals and states, there is a profile α that has "statewise full rank," which means roughly that the observed signals reveal the state regardless of whether i or j (but not both!) unilaterally deviates from α.
As in FLM, a weaker, "static-threats," version of the folk theorem holds under milder informational conditions.Section 6 shows that pairwise full rank can be replaced by the condition of "pairwise identifiability," which can be satisfied with a smaller number of signals, and that statewise full rank can be relaxed to "statewise distinguishability."Very roughly speaking, this condition says that for every pair of players i, j and pair of states ω, ω, there is a strategy profile whose signal distribution distinguishes between the two states regardless of the deviations of player j, and such that continuation payoffs can give a large reward to player i in state ω without increasing player i's incentive to deviate and without affecting player j's payoff in state ω.We use this condition to explain the difference between Example 3 and Example 4.
Finally, we explain how to extend our analysis to games where the players have initial private information.In such games, the PPXE still satisfy all of the incentive constraints and are still PPXE; they now correspond to pooling equilibria where all types of a given player use the same strategy.We then introduce the concept of "perfect type-contingently public ex post equilibria" or PTXE; this concept allows players to condition on their initial private information as well as the subsequent public history.The set of PTXE has a recursive structure, and the set of limit payoffs can be characterized by an extension of the linear programming algorithm that allows the action profile used to generate a given score to depend on the vector of types.However, despite the similar formal structure, the set of limit payoffs can be very different than before, as the players now have additional sources of information about the state.Thus we postpone a detailed exploration of PTXE to Fudenberg and Yamamoto (2010a), where we provide weaker sufficient conditions for a folk theorem and then provide a detailed analysis of several special cases, including that of games with a known monitoring structure.
While the study of uncertain monitoring structures is new, there is a substantial literature on repeated games with unknown payoff functions and perfectly observed actions, notably Forges (1984), Sorin (1984, 1985), Hart (1985), Aumann and Maschler (1995), Cripps and Thomas (2003), Gossner and Vieille (2003), Pȩski (2008), Wiseman (2005Wiseman ( , 2008)), Hörner andLovo (2009), andHörner, Lovo, andTomala (2010). 4Our work makes two extensions to this literature-first to the case of unknown payoff functions and imperfectly observed actions but a known monitoring technology, and from there to the case where the monitoring structure is itself unknown.Our work is closest to that of Hörner and Lovo (2009) and Hörner, Lovo, and Tomala (2010), as PTXE reduces to the belief-free equilibria they considered when actions are perfectly observed; we say more about these papers in Fudenberg and Yamamoto (2010a).PPXE is also related to belief-free equilibria in repeated games with private monitoring, as in Piccione (2002), Ely and Välimäki (2002), Ely, Hörner, and Olszewski (2005), Yamamoto (2007), Kandori (2010), andYamamoto (2009).5However, unlike the belief-free equilibria in those papers, the ex post equilibria we consider do not require that players be indifferent, and so it is not subject to the robustness critiques of Bhaskar, Mailath, and Morris (2008); this is what motivates our choice of a different name for the concept.

UNKNOWN SIGNAL STRUCTURE AND PERFECT
I} represent the set of players.At the beginning of the game, Nature chooses the state of the world ω from a finite set Ω = {ω 1 ω O }.
4 Gossner and Vieille (2003) and Wiseman (2005) studied symmetric-information settings.In Aumann and Hart (1992), Aumann and Maschler (1995), Cripps and Thomas (2003), Pȩski (2008), Hörner and Lovo (2009), Wiseman (2008), andHörner, Lovo, andTomala (2010), players receive private signals about the payoff functions and so can have different beliefs.(In Wiseman (2008), the players privately observe their own realized payoff each period; in the other papers the players do not observe their own realized payoffs, and the private signals are the players' initial information or "type.")Assume that players cannot observe the true state ω, and let μ ∈ Δ(Ω) denote the players' common prior over ω. 6 For now we assume that the game begins with symmetric information: Each player's beliefs about ω correspond to the prior.We relax this assumption in Section 7.
Each period, players move simultaneously, with player i ∈ I choosing an action a i from a finite set A i .Given an action profile a = (a i ) i∈I ∈ A ≡ × i∈I A i , players observe a public signal y from a finite set Y according to the probability function π ω (a) ∈ Δ(Y ); we call the function π ω the monitoring technology.Player i's realized payoff is u ω i (a i y), so that her expected payoff conditional on ω ∈ Ω and on a ∈ A is g ω i (a) = y∈Y π ω y (a)u ω i (a i y); g ω (a) denotes the vector of expected payoffs associated with action profile a.
In the infinitely repeated game, players have a common discount factor δ ∈ (0 1).Let (a τ i y τ ) be the realized pure action and observed signal in period τ, and denote player i's private history at the end of period t ≥ 1 by h t i = (a τ i y τ ) t τ=1 . 7Let h 0 i = ∅ and for each t ≥ 1, let H t i be the set of all h t i .Likewise, a public history up to period t ≥ 1 is denoted by h t = (y τ ) t τ=1 , and H t denotes the set of all h t .A strategy for player i is defined to be a mapping s i : Let S i be the set of all strategies for player i and let S = × i∈I S i .Note that the case of a known public monitoring structure corresponds to a single possible state, Ω = {ω}.
We define the set of feasible payoffs in a given state ω to be where Δ(A) is the set of all probability distributions over A: As in the standard case of a game with a known monitoring structure, the feasible set is both the set of feasible average discounted payoffs in the infinite-horizon game when players are sufficiently patient and the set of expected payoffs of the stage game that can be obtained when players use a public randomizing device to implement distribution η over the action profiles.Next we define the set of feasible payoffs of the overall game to be 6 Because our arguments deal only with ex post incentives, they extend to games without a common prior.However, as Dekel, Fudenberg, and Levine (2004) argued, the combination of equilibrium analysis and a noncommon prior is hard to justify.
7 As written, this formulation assumes that players do not observe their realized payoffs u ω i (a i y), unless the realized payoff does not depend on ω.Since we restrict attention to ex post equilibria, where players' beliefs about the state do not matter, we do not need to impose this restriction, with the exception of Lemma 9, where the restriction is explicitly stated.If players observe the realized payoff, then player i's private history after period t also includes (u ω i (a τ i y τ )) t τ=1 .
Note that a given v ∈ V may be generated using different action distributions η(ω) in each state ω.If players observe ω at the start of the game and are very patient, then any payoff in V can be obtained by a state-contingent strategy of the infinitely repeated game.Looking ahead, there will be equilibria that approximate payoffs in V if the state is identified by the signals, so that players learn it over time.Note also that even if players have access to a public randomizing device, the set of feasible payoffs of the stage game is the smaller set because play in the stage game must be a constant independent of ω.

Perfect Public ex post Equilibria
This paper studies a special class of Nash equilibria called perfect public ex post equilibria (PPXE); this is an extension of the concept of perfect public equilibrium that was introduced by FLM.Given a public strategy profile s ∈ S and a public history h t ∈ H t , let s| h t denote its continuation strategy profile after h t .DEFINITION 1: A strategy s i ∈ S i is public if it depends only on public information, that is, for all t ≥ 1, h t i = (a τ i y τ ) t τ=1 ∈ H t i , and DEFINITION 2: A strategy profile s ∈ S is a perfect public ex post equilibrium if for every ω ∈ Ω, the profile is a perfect public equilibrium of the game with known monitoring structure π ω .8 Given a discount factor δ ∈ (0 1), let E(δ) denote the set of PPXE payoffs, that is, E(δ) is the set of all vectors v = (v ω i ) (i ω)∈I×Ω ∈ R I×|Ω| such that there is a PPXE s ∈ S satisfying is the expectation with respect to the probability measure on histories induced by the strategy profile s at state ω.
Note that v ∈ E(δ) specifies the equilibrium payoff for all players and for all possible states.Note also that the set of PPXE can be empty, in contrast to the case of perfect public equilibria of games with a known state.9However, the conditions of our ex post folk theorem imply that PPXE exist for sufficiently large discount factors.
Because PPXE is an ex post equilibrium concept, each player's payoff in each state must be at least the minmax value in that state.Let v ω i = min α −i max a i g ω i (a i α −i ) be the minmax payoff for player i in state ω, and let V * (ω) = {v ω ∈ V (ω)|v ω i ≥ v ω i } be the set of feasible and individually rational payoffs at state ω.Let be the subset of the feasible payoff state where each player receives at least her minmax payoff in each state; this is the set of feasible and individually rational payoffs of the overall game.Since each player's payoff in each state must be at least the minmax value of that state, E(δ) ⊆ V * .Our folk theorem assumes that V * has nonempty interior (as a subset of R I×|Ω| ); this in turn will be the case if V * (ω) has nonempty interior (as a subset of R I ) for each ω.
By definition, any continuation strategy of a PPXE is also a PPXE.Thus any PPXE specifies PPXE continuation play after each signal y, where the continuation payoffs w(y) = (w ω i (y)) (i ω)∈I×Ω corresponding to this signal specify payoffs for every player and every state.We will write π ω (α) • w ω i for y π ω y (α)w ω i (y), which is player i's expected continuation payoff at state ω under action profile α.This recursive structure of the equilibrium payoff set motivates the following definition.
For δ ∈ (0 1) and W ⊆ R I×|Ω| , a pair (α v) ∈ (× i∈I Δ(A i )) × R I×|Ω| of an action profile and a payoff vector is ex post enforceable with respect to δ and W if there is a function w = (w ω ) ω∈Ω : Y → W such that for all i ∈ I and ω ∈ Ω, and For each δ ∈ (0 1), W ⊆ R I×|Ω| , and α ∈ × i∈I Δ(A i ), let B(δ W α) denote the set of all payoff vectors v ∈ R I×|Ω| such that (α v) is ex post enforceable with respect to δ and W .Let B(δ W ) be a union of B(δ W α) over all α ∈ × i∈I Δ(A i ).
To prove our main results, we will use the fact that various useful properties of PPE extend to PPXE.
THEOREM 1: If a subset W of R I×|Ω| is bounded and ex post self-generating with respect to δ, then W ⊆ E(δ).
For the proof see the Supplemental Material (Fudenberg and Yamamoto (2010b)).The proof is very similar to APS.The key is that when W is ex post self-generating, the continuation payoffs w(y) used to enforce v ∈ W ⊂ R I×|Ω| have the property that for each y ∈ Y , the vector w(y) ∈ R I×|Ω| can in turn be ex post generated using a single next-period action α (independent of ω) so that the strategy profile constructed by "unpacking" the ex post generation conditions does not directly depend on ω.
See the Supplemental Material for the proof, which is a straightforward generalization of FLM.

EXAMPLES
Before proceeding with the general analysis, we present several examples to illustrate properties of PPXE.The first two examples make special assumptions that permit the explicit construction of PPXE strategies.The third and fourth examples are variants of a repeated partnership game.Here we use our nonconstructive techniques to show that whether incentive problems lead to inefficiency depends on certain details of the information structure.EXAMPLE 1: There are two players, I = {1 2}, and two possible states, Ω = {ω 1 ω 2 }.In every stage game, player 1 chooses an action from A 1 = {U D}, while player 2 chooses an action from A 2 = {L R}.Their expected payoffs Here, the left table shows expected payoffs for state ω 1 and the right table shows payoffs for state ω 2 .Note that the feasible payoff set at state ω is full dimensional for each ω, and so is the feasible payoff set V of the entire game.Suppose that the set of possible public signals is Y = A × Ω, and that the monitoring technology is such that π ω y (a) = ε > 0 for y = (a ω) and π ω y (a) = 1 − 7ε for y = (a ω).
After every one-period public history h 1 ∈ H 1 , the continuation strategy profile is a PPXE.Also, given any state ω ∈ Ω, nobody wants to deviate in period one, since (U L) is a static Nash equilibrium and players cannot affect the distribution of the continuation play.Therefore, this strategy profile is a PPXE; its payoff vector converges to In particular, this equilibrium approximates the efficient payoff vector ((2 2) (2 2)) as the noise parameter ε goes to zero.
The idea of this construction is that continuation play depends on what players have learned about the state.When players observe y ∈ Y (ω 1 ) and learn that ω 1 is more likely, they choose "always (U L)," which yields an efficient payoff (2 2) in state ω 1 , but gives an inefficient outcome (1 1) in ω 2 .Likewise, when players observe y ∈ Y (ω 2 ) and learn that ω 2 is more likely, they choose "always (D R)" to achieve an efficient payoff (2 2) in state ω 2 but an inefficient payoff in ω 1 .In this sense, PPXE allows "utility transfers" across states.
Example 1 is misleadingly simple, because there is an ex post equilibrium of the static game, and for this reason there is a PPXE for all discount factors.It is also very easy to construct equilibria that approximate efficient payoffs in this example: Simply specify that (U L) is played for T periods and then either (U L) or (D R) is played forever afterward, depending on which state is more likely.In the next example there is no static ex post equilibrium, and hence no PPXE for a range of small discount factors.
EXAMPLE 2: Now we consider the game where players can learn the true state from observed signals.Suppose that there are two players and two states, so that I = {1 2} and Ω = {ω 1 ω 2 }.The payoffs for state ω 1 are shown in the left panel and those for state ω 2 in the right: Note that the stage game is prisoner's dilemma for each state, but the roles of actions are reversed; specifically, (U L) is efficient for state ω 1 , while (D R) is efficient for state ω 2 .Assume that the set of possible public signals is Y = A × Ω, and that the monitoring technology is perfect: π ω y (a) = 1 if y = (a ω) and π ω y (a) = 0 otherwise.As we will see, this example satisfies all of the full-rank conditions of our general ex post folk theorem, so in particular a PPXE exists, but our proof of the general folk theorem is not constructive.
Because this example has perfect monitoring, it is easy to give an explicit construction of a PPXE whose payoffs converge to the efficient frontier in each state.The basic idea is to wait one period, learn the state, and play a subgame-perfect equilibrium for the corresponding known-state game.However, the strategies used in the construction need to be a bit more complicated, as the recursive nature of PPXE requires that the strategies specify incentive compatible play following every sequence of signals, including those that have probability 0 unless "Nature deviates." Consider a strategy with the following four phases: • Phase "Regular ω 1 ."Players play (U L), which gives the efficient payoffs for state ω 1 .If y = ((U L) ω 1 ), stay.If y = ((D L) ω 1 ), y = ((U R) ω 1 ), or y = ((D R) ω 1 ), go to "Punish ω 1 ."If y = ((U L) ω 2 ), go to "Regular ω 2 "; otherwise, go to "Punish ω 2 ." • Phase "Punish ω 1 ."Players play (D R), which gives the minimax payoffs for state ω , go to "Punish ω 2 "; otherwise, stay.
• Phase "Regular ω 2 ."Players play (D R), which gives the efficient payoffs for state It is straightforward to check that this strategy profile with initial phase "Regular ω 1 " is a PTXE and approximates ((2 2) (2 2)).
Claim 7 in the Appendix shows how to extend the idea of this construction to any case where actions and states are perfectly observed, and uses it to prove a folk theorem for this observation structure.
The next two examples are partnership games with two actions {C i D i } per player, corresponding to high and low effort on the group project.There are three possible outcomes H, M, and L, two states, and stage game payoffs that make D i a dominant strategy in each state.The probability distribution generated by (D 1 D 2 ) is known and so independent of the state; what is unknown is the productivity of high effort.Moreover, the monitoring structure (or production function) in these games is additive: the change in probabilities induced by player i's changing from C i to D i is the same regardless of the action of the other player.EXAMPLE 3: In this example the uncertainty is symmetric in the state: In state ω 1 , if player 1 chooses C 1 instead of D 1 , then the probabilities of H and M increase by p H and p M , while player 2's choice of C 2 increases the probabilities by q H and q M ; in state ω 2 , the roles are reversed.The realized payoff functions are independent of ω and are given by for each i ∈ I, ω ∈ Ω, and y ∈ Y .We assume that for each i ∈ I, Here the left-hand side of the second inequality is the cost of player 1's choice of C 1 for state ω 1 (or the cost of player 2's choice of C 2 for state ω 2 ), and the right-hand side is an increase in player 1's benefit from the project when he chooses C 1 instead of D 1 for state ω 1 (or an increase in player 2's benefit when he chooses C 2 for state ω 2 ).Since the left-hand side is greater than the right-hand side, we conclude that D 1 strictly dominates C 1 for state ω 1 and that D 2 strictly dominates C 2 for state ω 2 .Likewise, the third inequality asserts that D 1 strictly dominates C 1 for state ω 2 and that D 2 strictly dominates C 2 for state ω 1 .Thus, D i strictly dominates C i for each state.Moreover, we assume that for each i ∈ I, so that choosing C i instead of D i always increases the total surplus.Summing up, the payoff matrix of the stage game corresponds to a prisoner's dilemma for each sate; hence, V * has a nonempty interior and (D 1 D 2 ) is a static ex post equilibrium.
EXAMPLE 4: In this example, the state only influences the productivity of player 2's effort: If player 1 chooses C 1 instead of D 1 , then the probabilities of H and M increase by p H and p M , independently of the state.In contrast, if player 2 chooses C 2 instead of D 2 , then the probabilities of H and M increase by q H and q M in state ω 1 , but they increase only by βq H and βq M in state ω 2 , where 0 < β < 1.
As in Example 3, the payoffs have the form for each i ∈ I and y ∈ Y .We once again impose restrictions on the realized payoffs so that the stage-game payoffs in each state correspond to a prisoner's dilemma: D i is a dominant strategy, so is efficient, and V * has a nonempty interior. 10n both of these examples, the conditions of FLM's Theorem 6.1 apply in each state considered in isolation, so if the state were known, the folk theorem would apply.Moreover, in each example there are action profiles that reveal the state, in the sense that the outcome distribution at that profile is different at state ω 1 than at state ω 2 .However, our ex post-threats folk theorem applies only to Example 3, while in Example 4 the folk theorem fails and, moreover, PPXE payoffs can be bounded away from efficiency.
As we will show, the key difference is that in Example 4, the two states are "entangled" in the sense that for any α 1 , the distribution π ω 2 (α 1 C 2 ) is a convex combination of π ω 1 (α 1 C 2 ) and π ω 2 (α 1 D 2 ) while this is not the case in Example 3 provided that α 1 assigns positive probability to D 1 .Hence in Example 4, lowering the expected value of the continuation payoffs under π ω 2 (α 1 D 2 ) (which can be necessary to provide incentives) also lowers the continuation payoffs under π ω 2 (α 1 C 2 ), and this bounds the set of PPXE payoffs away from the efficient frontier.

Using Linear Programming to Bound E(δ)
In this subsection, we provide a bound on the set of PPXE payoffs that holds for any discount factor; the next subsection shows that this bound is tight as the discount factor converges to 1.
Consider the following linear programming problem.Let α ∈ × i∈I Δ(A i ), λ ∈ R I×|Ω| , and δ ∈ (0 1).Then If there is no (v w) satisfying the constraints, let k * (α λ δ) = −∞.If for every K > 0 there is (v w) satisfying all the constraints and λ Here condition (i) is the "adding-up" condition, condition (ii) is ex post incentive compatibility, and condition (iii) requires that the continuation payoffs lie in the half-space corresponding to direction vector λ and payoff vector v.Note that when λ ω i = 0 and λ ω j = 0 for some ω = ω, condition (iii) allows "utility transfer" across states.This utility transfer is the most significant way that LP Average differs from the linear program in FL, so we will discuss it in more detail below.
As we show in Lemma 1(a), the value k * (α λ δ) is independent of δ, so we denote it by k * (α λ).Now let be the highest score that can be approximated in direction λ by any choice of α.
For each be the maximal half-space in direction λ and set PROOF: As in past work, part (a) follows from the fact that the constraint set in (iii) is a half-space: If (v w) satisfies constraints (i)-(iii) in LP Average for (α λ δ), then (v w) satisfies the constraints for (α λ δ), where w Since we already know that E(δ) ⊆ V * , part (d) of Lemma 1 shows that Q is "not too big": it does not contain any payoff vector we can rule out on a priori grounds.The next lemma shows that Q is "big enough" to contain all the payoffs of PPXE.
LEMMA 2: For every δ ∈ (0 1), The proof is the same as in Fudenberg, Levine, and Takahashi (2007); we restate it in the Supplemental Material to make it easy to see that the proof applies to the present setting.
To help explain the role of cross-state utility transfers, we will show that the conclusion of Lemma 2 does not hold if constraint (iii) is replaced by the uniform-over-states version The resulting "uniform" LP problem corresponds to a form of ex post enforceability on half-spaces.This condition is too restrictive to capture all of the payoffs of PPXE, as shown by the combination of the following claim.
CLAIM 1: In the LP Uniform problem formed by replacing (iii) in LP Average with (iii ), the solution k U (α λ δ) satisfies k U (α λ δ) ≤ λ • g(α) for each α and λ.Therefore, k U (λ δ) ≡ sup α k U (α λ δ) ≤ sup α λ • g(α) and the corresponding set of payoffs Q U is a subset of the payoffs V C that can be attained with actions that are independent of the state.
PROOF: Inspection of the constraints in the LP Uniform problem shows that it is equivalent to solving a separate LP problem for each state ω ∈ Ω in isolation.As FL showed, a solution to the LP problem for given (α ω) cannot exceed i∈I λ ω i g ω i (α).Therefore, k U (α λ δ), the maximal score in LP Uniform for a given α, is at most Q.E.D.
In both Examples 1 and 2, we constructed PPXE with payoffs outside of V U .

Computing the Limit of E(δ) as Players Become Patient
Now we show that the set E(δ) of PPXE payoffs expands to equal all of Q as the players become sufficiently patient, provided that a full-dimensionality condition is satisfied.For each set B, let int B denote the interior of B and let bd B denote the boundary of B.
DEFINITION 5: A subset W of R I×|Ω| is smooth if it is closed and convex, it has a nonempty interior, and there is a unique unit normal for each point on bd W .11 LEMMA 3: For any smooth set W in the interior of Q, there is δ ∈ (0 1) such that W ⊆ E(δ) for δ ∈ (δ 1).
From Lemma 1(d), Q is bounded, and hence W is also bounded.With this fact in hand, the rest of the proof is standard; we include it in the Supplemental Material for completeness.
Because any full-dimensional convex subset of R I×|Ω| can be approximated arbitrarily closely by a smooth subset,12 this lemma together with Lemma 2 proves the following theorem.
It is possible that dim Q < I × |Ω|, so that this theorem does not apply, but that lim δ→1 E(δ) = ∅.A trivial example of this occurs when the state ω has no effect on either the monitoring structure or the payoffs, so that it cannot possibly be observed, but is simply a nuisance parameter.In this case, E(δ) is a subset of the space V U of payoff that can be generated with actions that are independent of the state, so Q ⊆ E(δ) has dimension at most I.The solution is obviously to ignore the state and characterize the perfect public equilibria of the game where (any) ω is known; these equilibria correspond to the full set of PPXE of the game with the noise parameter added.More generally, the full-dimension conditions could fail due to the imperfect observability of ω or because the feasible payoff set itself does not have full dimension.In this case, one might be able to characterize lim δ→1 E(δ) using an extension of the iterative algorithm in Fudenberg, Levine, and Takahashi (2007).

A PERFECT EX POST FOLK THEOREM
In this section we give simple and easy-to-verify sufficient conditions for a folk theorem to hold in PPXE.This theorem shows that any map from states of the world to feasible and individually rational payoffs in that state can be approximated by equilibrium payoffs as the discount factor goes to 1, and in particular by payoffs of a PPXE.More formally, our folk theorem gives conditions under which lim δ→1 E(δ) = V * . 13When this is true, so that efficient payoffs can be approximated by PPXE, the payoffs do not provide much incentive for players to play other sorts of equilibria or to try to change the monitoring structure.On the other hand, when the set of PPXE is empty or when all PPXE are far from efficient but there are efficient sequential equilibria, the payoffs do provide an incentive for change and the PPXE restriction might be less compelling.
Since we have already shown that Q ⊆ V * and that lim δ→1 E(δ) = Q under the full-dimension condition, it remains to show that V * ⊆ Q, which is equivalent to showing that k * (λ) ≥ max v∈V * λ • v for each direction λ.Our sufficient conditions are actually stronger than that: they will imply that k * (λ) = ∞ for directions λ with nonzero components in two or more states.Conversely, the folk theorem fails if there is a λ such that k * (λ) < max v∈V * λ • v; we use this fact in Example 4 below.
For each i ∈ I, α ∈ × i∈I Δ(A i ), and ω ∈ Ω, let Π (i ω) (α) be a matrix with rows Individual full rank implies that at each state, every possible deviation of any one player leads to a statistically different distribution on outcomes; under this condition there are continuation payoffs that make every player indifferent between all actions.However, as we discuss in Section 6, many of our results hold under weaker but harder-to-verify conditions.
DEFINITION 7: For each (i ω) and (j ω) satisfying i = j, profile α has pairwise full rank for (i ω) and (j ω) if Π (i ω)(j ω) (α) has rank equal to Pairwise full rank implies that deviations by player i can be distinguished from deviations by j.It is satisfied for generic distributions on outcomes provided that the number of outcomes is at least |A i | + |A j | − 1.In the partnership games of Examples 3 and 4, each player has two actions and there are three possible outcomes.This is why the folk theorem can apply there when the state is known With only two signals, as in Radner, Myerson, and Maskin (1986), equilibrium payoffs are bounded away from efficiency uniformly in the discount factor.DEFINITION 8: For each (i ω) and (j ω) satisfying ω = ω, profile α has statewise full rank for (i ω) and Note that both pairwise full rank and statewise full rank imply individual full rank.Note also that the pairwise full-rank conditions require as many outcomes as required by pairwise full rank in FLM, and the statewise full-rank conditions require at most twice as many outcomes. 14Once again, these conditions are satisfied by generic distributions on outcomes provided that the number of outcomes is as large as the number of rows that need to be linearly independent.
The statewise full-rank condition guarantees that the observed signals will reveal the state, regardless of the play of player i in state ω and the play of player j (possibly equal to i) in state ω, assuming that everyone else plays according to α.This condition is more restrictive than necessary for the existence of a strategy that allows the players to learn the state: For that it would suffice that there be a single profile α where the distributions on signals are all distinct, which requires only two signals. 15On the other hand, the condition is less restrictive than the requirement that the state is revealed to an outside observer even if a pair of players deviates.For example, statewise full-rank is consistent with a signal structure where a joint deviation by players 1 and 2 could conceal the state from the outside observer, as in a two-player game with A 1 = A 2 = {L R} and π ω y (L R) = π ω y (R L).Intuitively, since equilibrium conditions only test for unilateral deviations, the statewise full-rank condition is sufficient for the existence of an equilibrium where the players eventually learn the state.In Section 6, we introduce the more complicated but substantially weaker condition of statewise distinguishability, and show that it is sufficient for a static-threat version of the folk theorem.
The following is an ex post folk theorem.Note that the set of assumptions of this theorem is generically satisfied if |Y | ≥ 2|A i | for all i ∈ I.
CONDITION IFR: Every pure action profile has individual full rank.
CONDITION PFR: For each (i ω) and (j ω) satisfying i = j, there is an action profile α that has pairwise full rank for (i ω) and (j ω).
CONDITION SFR: For each (i ω) and (j ω) satisfying ω = ω, there is an action profile α that has statewise full rank.
The following lemmas are useful in this proof.
LEMMA 4: Suppose that PFR holds.Then there is an open and dense set of profiles each of which has pairwise full rank for all (i ω) and (j ω) satisfying i = j.
The proof is analogous to that of Lemma 6.2 of FLM.
LEMMA 5: Suppose that IFR holds.Then for any i ∈ I, ω ∈ Ω, and ε > 0, there is a profile α ω such that and α ω has individual full rank for all (j ω) = (i ω).
The proof is analogous to that of Lemma 6.3 of FLM.
LEMMA 6: Suppose that a profile α has statewise full rank for (i ω) and (j ω) satisfying ω = ω, and that α has individual full rank for all players and states.Then k * (α λ) = ∞ for direction λ such that λ ω i = 0 and λ ω j = 0.
REMARK 1: Because k * (α λ) ≤ λ • g(α) in the known-monitoring-structure case of FL, this lemma shows a key difference between that setting and the uncertain monitoring-structure case we consider here.The idea is that under statewise full rank, the continuation payoffs in such half-spaces can give player i a very large payoff in state ω by giving player j a very low payoff in that state, while reversing this transfer in state ω.
The Supplemental Material contains a direct proof of Lemma 6 that some readers have found opaque; the proof we present below is less concise but more revealing.
PROOF OF LEMMA 6: Assume to begin with that each player has only two actions, A i = {a i a i } and A j = {a j a j }, and consider the special case of a direction λ such that λ ω i = λ ω j = 1 and all other components of λ are zero.Constraints (i) and (ii) for (l ω) ∈ I × Ω \ {(i ω) (j ω)} can be satisfied by some choice of (w ω l (y)) y∈Y because of individual full rank, and constraint (iii) is vacuous for these coordinates.So the LP problem reduces to finding (w ω i (y)) y∈Y and (w ω j (y)) y∈Y to solve has statewise full rank.It suffices to show that for any sufficiently large v ω i and v ω j , there exist (w ω i (y) w ω j (y)) y∈Y that satisfy the first four constraints with equalities and Eliminate this last equation by solving for w ω j (y).Then the coefficient matrix for the set of the remaining four equations is The statewise full-rank condition guarantees that this matrix has rank 4, so the system has a solution for any (v ω i v ω j ), and thus k * (α λ) = ∞.Intuitively, this construction makes w ω i (y) large for signals y that are more likely in state ω than in state ω and makes w ω i (y) negative for signals that are more likely under ω, while keeping player i indifferent between all actions in state ω, and player j indifferent in state ω.This would not be possible if the signal distribution were the same at the two states or, more generally, if the above matrix were singular.
This example only explains why the k * can be made arbitrarily large when exactly two components of λ are nonzero, but we can use this construction to obtain arbitrarily high scores in any direction λ that gives nonzero weight to two or more states.For example, suppose that λ ω i = λ ω j = λ ω l = 1 and other components are zero.First, choose (v ω i v ω j w ω i w ω j ) as in the above example, so that constraints (i) and (ii) for (i ω) and (j ω) are satisfied, v ω i and v ω j are large, and w ω i (y) + w ω j (y) = 0 for all y ∈ Y .What remains is to find w ω l that satisfy constraints (i) and (ii) for (l ω) and the feasibility constraint The individual full-rank condition implies there is w ω l (y) that satisfies constraints (i) and (ii), and since w ω i (y) + w ω j (y) = 0 and v ω i + v ω j can be arbitrarily large, the feasibility constraint can be satisfied for any value of w ω l (y).Finally, although the argument above assumes each player has two actions, it can easily be extended: In general, after eliminating w ω j (y), there will be |A i | + |A j | equations to be satisfied and the statewise full-rank condition assures that the coefficient matrix of the system of these equations has full rank.Therefore, the system has a solution for any (v ω i v ω j ) and hence k * (α λ) = ∞ as before. Q.E.D.
LEMMA 7: Suppose that profile α has pairwise full rank for all (i ω) and (j ω) satisfying i = j.Fix a direction λ such that for some ω, λ ω i is nonzero for at least two i, and λ ω i = 0 for all i ∈ I and ω = ω.Then k * (α λ) = λ • g(α).
As in the proof of Lemma 6, for each i ∈ I and ω = ω, there exist (w ω i (y)) y∈Y such that for all a i ∈ A i .Moreover, it follows from Lemmas 4.3, 5.3, and 5.4 of FLM that there exist (w ω i (y)) (i y) such that for all i ∈ I and a i ∈ A i , and Obviously, the specified continuation payoffs are in H(λ λ • g(α)) and enforce (α g(α)), as desired. Q.E.D.
LEMMA 8: Suppose that α has individual full rank for all (j ω) = (i ω) and has the best-response property for player i and for state ω.
The proof is a straightforward generalization of Lemmas 5.1 and 5.2 of FLM.
PROOF OF THEOREM 4: From Lemma 3, it suffices to show that Q = V * .To do so, we will compute the maximum score k * (λ) for each direction λ.
Case 1.Consider λ such that λ ω i = 0 and λ ω j = 0 for some ω = ω and i possibly equal to j.In this case, players can transfer utilities across different states ω and ω while maintaining the feasibility constraint and this construction allows k * (α λ δ) > λ • g(α), as Example 1 shows.In particular, from Condition SFR and Lemma 6 we obtain k * (λ) = ∞ for this direction λ.
Case 2. Consider λ such that (λ ω i ) i∈I has at least two nonzero components for some ω while λ ω i = 0 for all i ∈ I and ω = ω.Lemma 4 shows that every profile α can be approximated arbitrarily closely by a profile that has pairwise full rank for all players, and it follows from Lemma 7 that k * (λ) = sup α k * (λ α) = max v∈V λ • v.
Case 3. Consider λ such that λ ω i = 0 for some (i ω) and λ ω j = 0 for all (j ω) = (i ω).Suppose first that λ ω i > 0. Since every pure action profile has individual full rank, a * ∈ arg max a∈A g ω i (a) also has individual full rank.Therefore, from Lemma 8, Next, suppose that λ ω i < 0. It follows from Lemmas 5 and 8 that for every ε > 0, there is a profile Q.E.D.

A STATIC-THREATS FOLK THEOREM
In this section we present an alternative theorem that uses weaker informational conditions to prove a "static-threats" folk theorem, meaning that the theorem only ensures the attainability of payoffs that Pareto-dominate the payoffs of a static ex post equilibrium.Consequently, this theorem assumes that a static ex post equilibrium exists.This is always true when the state only matters for the monitoring structure but has no impact on the expected payoffs (that is g ω (a) = g(a)), and it is also satisfied for generic payoff functions g when the state has a sufficiently small impact on the payoff function.Several of our other assumptions in this section seem more likely to be satisfied if the uncertainty is "small," though that is not necessary, as shown by Example 3. DEFINITION 9: For each (i ω) and (j ω) satisfying i = j, profile α is pairwise identifiable for (i ω) and This is the same as the FLM definition of pairwise identifiability.Note that it does not require individual full rank, so that a given player may have several actions that generate the same signal distributions, and not all actions need be enforceable.
We say that α is ex post enforceable if it is ex post enforceable with respect to R I×|Ω| and δ for some δ ∈ (0 1).This is equivalent to α being enforceable with respect to R I and δ for each information structure π ω in isolation.CONDITION X-EFF: If pure action profile a gives a Pareto-efficient payoff vector for some ω ∈ Ω, it is ex post enforceable.
FLM show that any Pareto-efficient action profile is enforceable.Condition X-Eff extends this to ex post enforceability, so it is automatically satisfied when there is a single state.
CONDITION U-EFF: If pure action profile a gives a Pareto-efficient payoff vector for some ω ∈ Ω, then it gives a Pareto-efficient payoff vector for every ω Condition U-Eff says roughly that efficient actions are uniformly efficient.It is typically satisfied if the stage-game payoffs g ω i (a) are not too sensitive to the state, which in turn will be the case if the realized payoffs u ω i (y a i ) are insensitive to ω and the various distributions π ω are sufficiently similar.(It can also be satisfied when the distributions π ω differ substantially, depending on the details of the functions u ω i .)The condition is satisfied in the partnership games in Examples 3 and 4, where (C C) is the efficient profile in both states.
CONDITION PID: For each (i ω) and (j ω), every pure action profile is pairwise identifiable for (i ω) and (j ω).
Condition PID is stronger than needed; it is sufficient that it applies to the pure action profiles that yield Pareto-efficient payoffs.
LEMMA 9: If u ω i (y a i ) is independent of ω and U-Eff holds, then Condition X-Eff holds.
PROOF: Because each player's payoff depends only on his or her own action and the realized signal, Lemma 6.1 of FLM applied to each state ω in isolation implies that profile a is enforceable for each ω. Q.E.D.
As argued, statewise full rank can require that there be twice as many signals as required by the FLM folk theorem.The following, more complex, condition can be satisfied with far fewer signals.In part, this condition is related to the fact that linear independence of the outcome distributions is not needed for an action profile to be enforceable, as linear independence tests all linear combinations of the distributions, while it is sufficient to rule out convex combinations. 16  DEFINITION 10: Profile α statewise distinguishes (i ω) from (j ω) if there is a vector ξ = (ξ(y) We illustrate these conditions in Figure 1.Clause (i) implies that the signals generated by α statistically distinguish ω from ω. Clearly, there must be some such profile for there to be equilibria where the play varies with the state.Clause (ii) says that changing player i's continuation payoff function in state ω from w ω i (y) to w ω i (y) + ξ(y) preserves incentive compatibility for player i. Clause (iii) says that the change in player i's continuation payoff (of w ω i (y) ≡ ξ(y)) can be offset to preserve the feasibility constraint (λ ω i w ω i (y) + λ ω j w ω j (y) = 0) without changing player j's expected continuation payoff to any action.Note that this transfer scheme increases player i's expected continuation payoff by E[ w ω i |α] ≡ π ω (α) • ξ, so the maximal score for λ with λ ω i > 0 can be made infinitely large by utility transfer between states ω from ω. 17 16 See Kandori and Matsushima (1998).In the study of mechanism design with transferable utility, Kosenok and Severinov (2008) and Rahman and Obara (2010) gave a weaker sufficient condition for budget-balanced implementation; the balanced-budget constraint roughly corresponds to directions λ, where every component is strictly positive. 17If λ ω i < 0, then player i's continuation payoff must be decreased to achieve a high score.This requires a different sort of transfer and in turn requires a different condition on the information structure, but this condition is not needed for a static-threats folk theorem.CONDITION SD: For each (i ω) and (j ω) satisfying ω = ω, there is an ex post enforceable action profile α that statewise distinguishes (i ω) from (j ω).
Condition SD is sufficient for the static-threat folk theorem, as it implies that profile α can generate an infinite score in all of the required "cross-state" directions.
THEOREM 5: Suppose that PFR holds or Conditions X-Eff and PID hold.Suppose also that Condition SD holds.Assume that there is a static ex post equilibrium α 0 and let Then, for any smooth set W in the interior of V 0 , there is δ ∈ (0 1) such that W ⊆ E(δ) for all δ ∈ (δ 1).So if V 0 has nonempty interior, for each v ∈ V 0 there is δ ∈ (0 1) such that v ∈ E(δ) for all δ ∈ (δ 1).This theorem is established by the following lemmas that determine the maximal score k * in various directions.The next lemma says that the score of a static ex post equilibrium can be enforced in any direction; this score will be used to generate the score in directions that minimize a player's payoff.
The next lemma determines the maximal score for direction λ that considers a single state ω and has a positive component or at least two nonzero components, when Condition X-Eff holds.
LEMMA 11: (a) Suppose that Conditions PFR or X-Eff and PID hold, and that profile a gives Pareto-efficient payoffs for some ω ∈ Ω.Then k * (a λ) = λ • g(a) for direction λ such that (λ ω i ) i∈I has at least two nonzero components while λ ω j = 0 for all j ∈ I and ω = ω.
PROOF: (a) Lemma 1(b) shows that the maximum score in direction λ is at most λ • g(a).Because a is a pure action profile, and it is enforceable for all ω and pairwise identifiable from Conditions X-Eff and PID, it is enforceable on hyperplanes corresponding to λ from Theorem 5.1 of FLM, so the score λ • g(a) can be attained.If Condition PFR holds, this follows from Lemmas 4 and 7.
(b) Let a be a Pareto-efficient profile that maximizes player i's payoff in state ω.If Condition X-Eff holds, a is ex post enforceable, and since the profile has the best-response property in state ω, Lemma 5.2 of FLM implies it is enforceable on λ.If Condition PFR holds, this follows from Lemma 8. Q.E.D.
The following lemma shows that Condition SD is sufficient for the maximal score to be infinite in every cross-state direction λ that has at least one positive component.See the Appendix for the proof.
We now apply these concepts to demonstrate the differences between the two partnership games that we introduced in Section 3. EXAMPLE 3-Continued: Recall Example 3 from Section 3, where the effect of the uncertainty is symmetric across states and players: In state ω 1 , if player 1 chooses C 1 instead of D 1 then the probabilities of H and M increase by p H and p M , while player 2's choice of C 2 increases the probabilities by q H and q M ; in state ω 2 , the roles are reversed.Note that individual full rank is satisfied, and that pairwise full rank is satisfied at every profile and every state if the matrix p H p M q H q M has full rank.For example, the matrix and this matrix has rank 3 if the above two-by-two matrix has full rank.Therefore, the profile (D 1 C 2 ) has pairwise full rank for (1 ω 1 ) and (2 ω 1 ).On the other hand, statewise full rank is not satisfied at any profile, as there are only three signals, while four signals would be needed to satisfy this stronger condition.Nevertheless, we will show that the static-threat folk theorem holds in this example, because statewise distinguishability is satisfied.
In this case, let ξ = (ξ(y)) y∈Y be a solution to the system for some K > 0. This system has a solution, since the matrix p H p M q H q M has full rank, and the solution satisfies For ((i ω) (j ω)) = ((2 ω 1 ) (2 ω 2 )), we can use the same ξ.For ((i ω) This claim shows that Condition SD holds in Example 3, so that the staticthreat folk theorem applies to Example 3. In contrast, payoffs are bounded away from efficiency in Example 4, which is a related partnership game.This is because the states in Example 4 are "entangled" in the following sense: DEFINITION 11: Profile α entangles states ω and ω for player j if there is π ∈ co{π ω(a j α −j )|a j ∈ A j } such that π ω(α) = κπ ω (α) + (1 − κ)π for some κ ∈ (0 1]. LEMMA 13: If profile α statewise distinguishes (i ω) from (j ω), then α does not entangle ω and ω for player j.
EXAMPLE 4-Continued: Recall Example 4, where the state only influences the productivity of player 2's effort: If player 1 chooses C 1 instead of D 1 , then the probabilities of H and M increase by p H and p M , independent of the state.In contrast, if player 2 chooses C 2 instead of D 2 , then the probabilities of H and M increase by q H and q M in state ω 1 , and by βq H and βq M in state ω 2 .Individual full rank and pairwise full rank are satisfied at every profile and every state if the matrix p H p M q H q M has full rank.However, every profile entangles ω 2 and ω 1 for player 2, essentially because player 2 working with probability x in state ω 2 generates the same signal distribution as player 2 working with probability βx in state ω 1 , so the sufficient conditions for the static-threats folk theorem are not satisfied.Moreover, we will show that the folk theorem fails in this example and, more specifically, that the maximal score k * (λ) in direction λ = ((1 0) (0 1)) is strictly less than λ • g(C 1 C 2 ), which in turn is less than max v∈V * λ • v.
To show that the folk theorem fails, we use the fact that the monitoring technology has an additive form, so that it suffices to consider only the pure action profiles, as in Lemma 4.1 of FL. 18CLAIM 3: PROOF: See the Appendix.The inefficiency here comes from the fact that the two states are entangled for player 2 and hence the profile (C 1 C 2 ) does not statewise distinguish (1 ω 1 ) from (2 ω 2 ). Q.E.D.
CLAIM 4: The proof is the same as in the previous claim.
CLAIM 5: the set of the constraints in the LP Average problem for λ is isomorphic with that for λ = ((0 0) (1 1)).Then the maximal score for λ equals that for λ and the statement follows from Lemma 1(b). Q.E.D.
The proof is the same as in Claim 5. Now we combine these claims to show that k Here, the second equality comes from the state independence of player 1's marginal contribution, which implies that g Combined with the previous claims, it follows that k * (λ ) < λ • g(C 1 C 2 ) so that the folk theorem fails.Moreover, because the player's equilibrium payoffs cannot be below their minmax level in any state, this bound implies that for some parameter values, player 2's PPXE payoff in state ω 2 is strictly less than g ω 2 2 (C 1 C 2 ). 19

INCOMPLETE INFORMATION AND PTXE
So far we have assumed that the players have symmetric information about the state at the beginning of the game.Now suppose that each player i observes a private signal θ i ∈ Θ i at the beginning of the game, where Θ i is a partition of Ω.Let θ i (ω) ω be the partition element that i observes when the state is ω, which we will call player i's type, and let θ(ω) = (θ i (ω)) i∈I .Any public strategy s i of the game where player i has the trivial partition Θ i = {Ω} induces a public strategy for any nontrivial partition Θ i : player i simply ignores his type and sets s i (h θ i ) = s i (h) for all h and all θ i .Since, by definition, play in a PPXE is optimal regardless of the state, any PPXE for the symmetricinformation game (where all players have the trivial partition) induces a PPXE for any incomplete-information game (any partitions Θ i ) with the same payoff functions and prior.Thus the PPXEs of the incomplete-information games are isomorphic to the PPXEs of the associated symmetric-information game, so the limit PPXE payoffs can be computed using LP Average and, in particular, our sufficient conditions for the folk theorem are still sufficient.
However, we expect there to be other equilibria where different types of a given player use different strategies.To analyze these equilibria, we extend the notion of PPXE to perfect type-contingently public ex post equilibria (PTXE). 19For example, suppose that p H = 0 5, p M = 0, q H = 0, q M = 0 5, β = 0 8, r i (H) = 100, r i (M) = 99, r i (L) = 0, e 1 = 99, and e 2 = 79.Then the minmax payoffs are 0 for all players and all states, g ω In what follows, let h t i denote player i's private history from period one to period t, that is, h t i = (a τ i y τ ) t τ=1 .Let H t i denote the set of all h t i .Likewise, let h t = (y τ ) t τ=1 be a public history from period one to period t and let H t be the set of all h t .Player i's overall strategy is a map s i : Θ i × ∞ t=0 H t i → Δ(A i ).Player i's strategy s i ∈ S i is type-contingently public if it depends only on θ i ∈ Θ i and h t ∈ H t , that is, if s i (θ i h t i ) = s i (θ i ht i ) whenever h t i and ht i correspond to the same public history.A strategy profile s ∈ S is type-contingently public if s i is type-contingently public for each i ∈ I. Given a type-contingently public strategy profile s ∈ S, let s i | (θ i h t ) denote player i's continuation strategy when his type is θ i and the past public history is h t , and s| (θ h t ) = (s i | (θ i h t ) ) i∈I .20 DEFINITION 12: A strategy profile s ∈ S is a perfect type-contingently public ex post equilibrium if it is type-contingently public and if s| (θ(ω) h t ) is a Nash equilibrium for any ω ∈ Ω and h t ∈ H t .
Note that PTXE coincides with PPXE if there is no asymmetric information, that is, Θ i = {Ω} for all i ∈ I.In addition, it corresponds to the belief-free equilibrium of Hörner and Lovo (2009) and Hörner, Lovo, and Tomala (2010) in games with observed actions and incomplete information: These papers define a belief-free equilibrium to be a strategy profile s such that for each state ω, profile s is a subgame-perfect equilibrium of the game where all players know the state is ω.
By definition, any continuation strategy s| h t = (s| θ(ω) h t ) ω∈Ω of a PTXE is also a PTXE.Thus any PTXE specifies PTXE continuation play after each signal y, where the continuation payoffs w(y) = (w ω i (y)) (i ω)∈I×Ω corresponding to this signal specify the payoffs for every player and every state.This recursive structure allows us to extend our linear programming characterization to PTXE.
First we need to define some notation.We will write π ω (α) • w ω i for the expected continuation payoff at state ω under action profile α, where w ω i is the vector (w ω i (y)) y∈Y .Let α i = (α for each θ i ∈ Θ i , and let α = ( α i ) i∈I .In words, α is a type-contingent action profile; it specifies a mixed action for each private signal θ i of each player i.For example, if the true state is ω, then players have type profile θ(ω), so that α says to play α θ(ω) = (α θ i (ω) i ) i∈I .The definitions of ex post enforceability extends to PTXE in the obvious way: for all i ∈ I and ω ∈ Ω, and for all i ∈ I, ω ∈ Ω, and a i ∈ A i .
Note that this definition takes into account the possibility that the action profile α varies with θ.Note also that the second condition is imposed at every state ω, so it does not depend on player i's type θ i .
Using the idea of type-contingent enforcement, we can extend the definition of ex post self-generation and local ex post generation to PTXE, and it is easy to verify that Theorems 1 and 2 hold as stated with this extension when E(δ) is interpreted as the payoffs of PTXE.
In a similar way, we can extend the linear programming characterization of the limit equilibrium payoffs.The key difference is that the players' actions can now depend on their type, so the action profiles used to generate the maximal scores are allowed to depend on the type profile.Thus the linear programs we consider are If there is no (v w) satisfying the constraints, let k * ( α λ δ) = −∞.If for every K > 0 there is (v w) satisfying all the constraints and λ If we use this program to define the set Q, then Theorem 3 holds as before: However, the nature of this set Q can be very different than before, as the players now have three possible sources of information about the state: (i) inference based on the public signals at state-independent action profile, as in the bulk of this paper; (ii) the information contained in their own types; and (iii) inferences based on the correlation between the opponents' actions and the opponents' types.In the companion to this paper (Fudenberg and Yamamoto (2010a)), we provide weaker sufficient conditions for a folk theorem that takes advantage of all three of these information channels.We then focus on cases with additional structure that simplifies the characterization of limit payoffs, such as games with a product structure, where there is a separate and independent signal associated with each player's action and each player knows the effect of his action on the signal distribution while the others do not.We also examine games with a known monitoring structure, where we show that the set of limit equilibrium payoffs with imperfectly observed actions is the same as in the observed-action case studied by Hörner and Lovo (2009) and Hörner, Lovo, and Tomala (2010) provided that the monitoring structure satisfies a full-rank condition.In addition, our techniques provide a simpler sufficient condition for the existence of belief-free equilibrium. 21

CONCLUDING REMARKS
This paper has shown that the sets of PPXE and PTXE payoffs have a recursive structure and that their limit payoffs can be analyzed with extensions of the techniques used to analyze PPE in games where the monitoring structure is known.When the statewise full-rank conditions hold, along with the standard individual and pairwise full-rank conditions, the set of PPXE satisfies an ex post folk theorem, even if the set of static ex post equilibria is empty.When a static ex post equilibrium does exist, there is an ex post PPXE folk theorem under even milder informational conditions.
Of course for a given discount factor, the full set of sequential equilibria of these games is larger than the set of ex post equilibria and can permit a larger set of payoffs.In particular, because the game has finitely many actions and signals per period and is continuous at infinity, sequential equilibria exist for any discount factor, even if the set of PPXE or PTXE is empty.This follows from the facts that sequential equilibria exist in the finite-horizon truncations (Kreps and Wilson (1982)) and that the set of equilibrium strategies is compact in the product topology (Fudenberg and Levine (1983)).So neither concept is well adapted to the study of games with uncertain monitoring structures and very impatient players.Conversely, when players are patient and mostly concerned with their long-run payoff, our informational conditions imply that there are PPXE where players eventually learn what the state is and obtain the same payoffs as if the state was publicly observed. 22Moreover, if players have initial 21 Hörner, Lovo, and Tomala (2010) gave tight conditions that ensure that the set Q is nonempty; this set equals the set of limit payoffs of PTXE when it has a nonempty interior.
22 When the ex post folk theorem holds and a feasible payoff vector v is not a limit payoff of PPXE, then for some player/state pair the payoff v ω i is not ex post individually rational, so the payoff vector v cannot pointwise dominate any point v ∈ V * .However, as Olivier Gossner pointed out, there may be priors such that the expected payoffs to a sequential (but not ex post) equilibrium Pareto-dominates all of the ex post equilibrium payoffs, essentially because revealing information destroys opportunities for insurance, as in Hirshleiffer (1971).private information, there can be folk theorems in the set of PTXE even when the set of PPXE is small or empty; Fudenberg and Yamamoto (2010a) develops the relevant information conditions and studies the set of PTXE payoffs in some cases of interest.APPENDIX A.1.An ex post Folk Theorem With Perfect Monitoring CLAIM 7: Suppose that monitoring is perfect, that is, Y = A × Ω and π ω y (a) = 1 if y = (a ω).Fix a payoff vector v ∈ int V * .Then there is δ such that for all δ ∈ (δ 1), there is a PPXE where players play a pure action profile α in period one and then along the equilibrium path play s ω (δ) from period two, where s ω (δ) is a subgame-perfect equilibrium for state ω and discount factor δ with payoff v ω .PROOF: Let v = (v ω ) ω∈Ω ∈ int V * and let ε > 0 be such that for each ω, any payoff vector within ε of v ω is in the set V * (ω).Then let δ ∈ (0 1) be such that (i) ε > 1−δ δ i∈I (max a∈A g i (a) − min a∈A g i (a)), (ii) for each ω, there is a subgame-perfect equilibrium s ω v ω for state ω and discount factor δ with payoff v ω , and (iii) for each ω ∈ Ω and for any payoff vector ṽω within ε of v ω , there is a subgame-perfect equilibrium s ω ṽω for state ω and discount factor δ. Note that these three conditions hold if δ is close to 1; the last condition (iii) comes from Theorem 6.2 of FLM.
Consider the following strategy profile: Phase 1. Play a pure action profile a in period one.If there is no unilateral deviator from a and ω is observed, then go to Phase (ω v ω ).If player i unilaterally deviates from a and ω is observed, then go to Phase (ω (v i − 1−δ δ (max ã∈A g i ( ã) − min ã∈A g i ( ã)) (v ω j ) j =i )).Phase (ω ṽω ).(Here, ω ∈ Ω and ṽω is within ε of v ω .)Play a subgameperfect equilibrium s ω ṽω in the remaining periods, as long as ω is observed in every period of this phase.(Recall that s ω ṽω is a subgame-perfect equilibrium for state ω with payoffs ṽω .)If in any period t, ω t = ω t−1 , then go to phase (ω t w ω t (a t )) in the next period, where w ω t (a t ) = (w ω t i (a t )) i∈I is chosen so that for all i ∈ I.This strategy profile is well defined, as w ω t (a t ) is within ε of v ω by construction, and it is easy to see that this strategy profile is a PPXE. Q.E.D.

A.3. Proof of Claim 3
CLAIM 3-Restatement: PROOF: Consider the associated LP Average problem and choose (v w) to satisfy constraints (i)-(iii) of this problem.From player 2's incentive compatibility (IC) constraint for state ω 2 , we have Arranging yields So we have This proves the desired result. Q.E.D.

DEFINITION 6 :
Profile α has individual full rank for (i ω) if Π (i ω) (α) has rank equal to |A i |.Profile α has individual full rank if it has individual full rank for all players and all states.