Deriving epistemic conclusions from agent architecture

One of our most resilient intuitions is that causality is a precondition for information flow: where there are no causal connections, we expect there to be no flow of information. In this paper, we study this idea as it arises in the computer science notion of systems architectures, which are high level designs that describe the coarse structure of a system in terms of its high-level components and their permitted causal interactions.


Introduction
One of our most resilient intuitions is that causality is a precondition for information flow: where there are no causal connections, we expect there to be no flow of information.In this paper, we study this idea as it arises in the computer science notion of systems architectures, which are high level designs that describe the coarse structure of a system in terms of its high-level components and their permitted causal interactions.
The MILS (Multiple Independent Levels of Security and Safety) initiative [Alves-Foss et al., 2006;Vanfleet et al., 2005;Boettcher et al., 2008] of the US Air Force proposes to use architecture as a key part of the certification case for high-assurance systems.It is envisaged that complex systems will be constructed from a mix of trusted and untrusted components composed according to an architecture.Trusted components might be formally verified to satisfy their specifications.Untrusted components might include commercial, off the shelf (COTS) software that is too complex to verify and may be unreliable or even malicious.It is desired that global security and safety properties can be guaranteed as a result of the behavior of trusted components and the architectural structure of the system, regardless of the behavior of untrusted components.
Architectures have generally been represented by diagrams of boxes and arrows whose meaning lacks a rigorous foundation.In a previous paper [Chong and van der Meyden, 2009] we gave formal semantics to a type of architectural specification.In addition to describing the system's causal structure, our specification format can express restrictions on information permitted to flow between components.We showed that these architectural specifications support interesting examples of rigorous reasoning about security properties expressed in an epis-temic logic.However, that framework does not deal well with situations in which agents synchronously observe information.In some applications, e.g., auctions, such synchronous observations are critical to the desired security and "fairness" properties.Fairness of an auction requires lower bounds on information flow, such as "the results of the previous round are known to all bidders".However, our previous approach expresses only upper bounds on information flow.
In this paper, we extend our previous approach by developing a richer architectural specification format that can express that certain components in the system synchronously observe the state of other components.This provides a way to represent lower bounds on information flow.Our extended specifications remain highly abstract: neither the states nor the agents' actions need to be explicitly specified.We show by means of a number of examples that the extended architectural specification format can enforce properties expressed using epistemic logic that cannot be enforced by our earlier framework.
There have to date been few examples to justify that an architectural approach to the construction of secure systems can be given a formal foundation.This work contributes by making formal sense of architectural specifications and showing that the type of reasoning envisaged by MILS has the potential to establish security properties at early stages of the design process.While our immediate motivation stems from computer security, we believe our results will also be of interest in software engineering, multi-agent Artificial Intelligence and economic mechanism design.
The structure of the paper is as follows.We begin with some preliminaries: Section 2 describes the formal machine model in which we interpret architectures, and Section 3 introduces the epistemic logic we use to specify security properties.In Section 4 we recall the architectural specification format of Chong and van der Meyden [2009], and motivate the need for the extension intro-duced in the present paper.We define this extension and its semantics in Section 5.This is followed by a number of sections that give applications of the extended architectural specification format: Section 6 deals with a bulletin board architecture, Section 7 concerns an architecture for auction systems, and Section 8 presents a general condition under which a group of agents is able to audit all flows of information from one part of the system to another.Some concluding remarks are made in Section 9.

Machine model
Intuitively, the architectural specifications we develop in this paper are interpreted in systems consisting of agents that have state-changing actions and an ability to make observations of the system state.Following terminology from the security literature, we refer to agents as domains; the motivation for this usage is that we may be interested in information flows between classes of agents (e.g., all agents cleared to read classified documents) in addition to single agents.Actions are assumed to be deterministic.We assume that domains operate asynchronously.
We formalize systems using a state-observed machine model [Rushby, 1992], which defines deterministic statebased machines.A machine has a set of actions A, and each action is associated with a security domain.Intuitively, if action a is associated with domain u, then a represents a decision, choice, or action taken by the system component represented by u.Actions deterministically alter the machine state, and we assume that the observations of each security domain are determined by the current machine state.
Formally, a machine is a tuple M = S, s 0 , A, step, obs, dom where S is a set of states, We assume that it is possible to execute any action in any state: function step is total.Given sequence of actions α ∈ A * , we write s•α for the state reached by performing the each action in turn, starting in state s.We define s • α inductively using the transition function step, by ( denotes the empty sequence) For notational convenience, we write obs u for the function obs(u, •), and obs u (α) for obs u (s 0 • α), where α ∈ A * .If G ⊆ D is a group of agents, we write obs G (α) for the tuple (obs u (α)) u∈G .We also write A u = {a ∈ A | dom(a) = u} for the set of actions belonging to u ∈ D.
Given a sequence α ∈ A * , the view of α of a group of domains G is the sequence of G's observations and actions that belong to G. The function view G with domain A * is defined inductively by: The definition uses the absorptive concatenation operator •: for set X, sequence α ∈ X * , and element x ∈ X, α • x = α if x is equal to the last element of α, and α • x = αx otherwise.Intuitively, G's view is the information that the group G would have if it were acting as a single agent operating with asynchronous perfect recall, remembering all its distinct observations and actions, but not being aware of the duration of invariant observations.When G = {u} is a singleton, we write view u for view G .When G is not a singleton, view G (α) can be understood as the maximal information that the group could have if we were to give it the ability to instantaneously communicate within the group any new information that any agent in the group acquires.
Finally, for any sequence of actions α ∈ A * , we write α G for the subsequence of α of actions whose domain is in the set G.

Specification Language
We use an epistemic logic to express information flow properties of architectures, with grammar: In addition to the standard abbreviations, we write K u φ as shorthand for K {u} φ; this is the same as E {u} φ, D {u} φ and C {u} φ.
A proposition is a set X ⊆ A * .We say proposition X is non-trivial if X = ∅ and X = A * .An interpretation function π is a function from propositional constants to propositions.
We define the semantics of the logic using satisfaction relation M, π, α φ, which intuitively means that formula φ is true given interpretation function π, and machine M that has executed sequence α ∈ A * .Figure 1 defines satisfaction relation M, π, α φ.We write M, π φ if for all α ∈ A * we have M, π, α φ.
To reason about information that is locally known to a group of domains G, we introduce G-local propositions [Engelhardt et al., 1998], and G-action-local propositions.A proposition is G-local if the view of domains G suffices to decide the proposition.Formally, a set of states but not vice-versa.As usual, we abbreviate {u} to u in these notions.

Architectures
In this section we recall the notion of architecture and its semantics as introduced in Chong and van der Meyden [2009] and give an example that motivates the need for the extension that we develop in the present paper.
Architectures are specifications of the causal structure of a system that place constraints on the permitted flows of information between security domains.Figure 2 gives an example of an architecture for an electronic election system, comprised of a number of voters v 1 . . .v n , and an election authority ElecAuth. ...

Figure 2: An architecture for elections
Intuitively, this diagram represents that the voters submit information (their votes) to the election authority, who returns the result of the election to the voters.In addition to these permitted information flows, the diagram also implies that some flows of information are prohibited.In particular, since there is no edge from a voter to any other voter, direct communication between voters through the election system is not permitted.Further, the election authority is not permitted to return more information to any voter than the results.In particular, the details of any individual voter's vote should not be revealed, except to the extent that information about it might be deducible from the result of the election.
Formally, an architecture is a pair A = (D, ), where D is a set of security domains, and ⊆ D × D × (L ∪ { }), where L is a set of function names.We write u f v when (u, v, f ) ∈ , and write u v as shorthand for ∃f.(u, v, f ) ∈ , and u v as shorthand for ¬∃f.(u, v, f ) ∈ .
The intuition for the relation u f v is that information flow from u to v is permitted, but may be subject to constraints.In case f = , there are no constraints on information flow from u to v: any information that may be possessed by u is permitted to be passed to v when u acts.If f ∈ L then information is allowed to flow from domain u to domain v, but it needs to be filtered through the function denoted by f : only information output by this function may be transmitted from u to v. If u v then no direct flow of information from u to v is permitted.
Architectures are required to have the following properties: Arch2.The relation is reflexive in that for all u ∈ D we have (u, u, ) ∈ .
The first condition requires that all permitted flows of information from u to v are represented using a single labeled edge.Intuitively, any policy with multiple such edges can always be transformed into one satisfying this condition, by combining the pieces of information flowing across these edges into a tuple that flows across a single edge.The second condition is motivated from the fact that information flow from a domain to itself cannot be prevented.When drawing architectures, we annotate arrows between domains with the filter function names and elide reflexive arrows.For arrows drawn without a label, and elided reflexive arrows, the implied label is .
Architectures do not define the interpretations of the function names L. If A = (D, ) is an architecture, an interpretation for A is a tuple I = (A, dom, I), where A is a set of actions, dom : A → D assigns these actions to domains of A, and I is a function mapping each f ∈ L to a function with domain A * (and arbitrary codomain).We call the pair (A, I) an interpreted architecture.
Intuitively, if u f v and α ∈ A * and a ∈ A is an action with dom(a) = u, then I(f )(αa) is the information that is permitted to flow from u to v when the action a is performed after occurrence of the sequence of actions α.Based on this intuition, we may define a function fta u with domain A * that captures that maximal information that domain u is permitted to have after a sequence of actions has been executed.
Given extended architecture (D, ) and an architectural interpretation I = (A, dom, I), the function fta u is defined inductively by fta u ( ) = , and, for α ∈ A * and a ∈ A, The intuition for this definition is as follows.First, the value fta u (α) is to be understood as a concrete representation of the maximal information that domain u is permitted to have after the sequence of actions α has been performed.The recursive definition describes how this maximal information grows when the action a is performed after α.This depends on the nature of the edge, if any, from dom(a) to u.The first clause says that if there is no edge from dom(a) to u, then u is not permitted to learn anything when a is performed.That is, we have fta u (αa) = fta u (α), so u's maximal information before and after the action is the same.The second clause says that if dom(a) u, then the additional information that u may have after a is the maximal information that the domain dom(a) performing the action had before a (i.e.fta dom(a) (α)), together with the fact that the action a has just been performed.Finally, in the case of edges of the form dom(a) f u, the interpretation of the function name f defines the additional information that u may acquire when a is performed.
A machine complies with an interpreted extended architecture if it has appropriate domains and actions, and for each domain u, what u observes in state s 0 • α is determined by fta u (α).We call such a machine FTAcompliant.("FTA" is derived from filtered transmission of information about actions.) Definition 1 (FTA-compliant) A machine M = S, s 0 , A, D, step, obs, dom is FTA-compliant with an interpreted architecture (A, I), with A = (D , ) and It is shown in Chong and van der Meyden [2009] that this definition of architectures and their semantics, together with some restrictions on the permitted interpretations of the function names, provides a specification format that, although highly abstract, already suffices to ensure that certain interesting epistemic consequences hold in any system that complies with the specification.For example, it is shown that, under some mild assumptions, the election architecture enforces the property that voters' actions in the system are anonymous.More precisely, any proposition considered possible by any voter v about other voters is also considered possible by v if we permute the names of the other voters.The assumptions required are that any action available to a voter is also available to any other voter and that the results function is invariant under permutation of the voters, so that the results depend only on what the votes were, not on which voter did what.
Note that this result holds very abstractly: the architecture leaves most aspects of the system's design unspecified, and places only some weak constraints on its structure and the behavior of the election authority.It says nothing about the local states maintained by the voters, and does not specify the nature of the votes (so these can be implemented as, e.g., a single vote for the preferred candidate, with voting in a sequence of rounds, or as a preference order on all candidates).There is also considerable flexibility in the details of the results function (this could be, e.g., the winner of the election, or the number of votes cast for each candidate.)Moreover, the architecture isolates the responsibility for the desired security property (anonymity) in a single "trusted" component, the election authority.Assuming that security mechanisms such as access control and physical communication links have been applied to constrain information flow to the edges in the architecture, to obtain this property it suffices to check that the election authority correctly computes the result, and ensures that only this information can flow to the voters.This illustrates the way that the MILS approach to secure systems construction proposes to build secure systems from a mix of trusted and untrusted components composed in the context of an architectural framework that enforces constraints on information flow.
One might expect, from the diagram for the election architecture, that certain other properties should hold.One property that is desirable in the application, and which might seem to be implied by the architecture, is that the results of the election will be common knowledge to the voters.To formalize this let M be a system that is compliant with the election architecture, and let π(p) be a proposition about the result of the election, e.g., "party P is the winner".Then we might expect that M, π |= K v p ⇒ C V p, where v is any voter and V is the set of all voters.Unfortunately, this does not follow.Suppose we construct the system M so that the election authority has an action a that posts the election result only to a location that is observable to v, but to no other voter.Then after a, if p holds then voter v will know that p, but the other voters will not, so it will not be common knowledge.Nevertheless, such a system is not inconsistent with the architecture.The definition of FTA-compliance states only an upper bound on information flow, whereas what the expected property requires is a lower bound.Although it may superficially appear that the architectures of Chong and van der Meyden [2009] are able to represent broadcast systems, they are not able to enforce that broadcast information will be available to intended recipients.One of the contributions of the present paper is to develop an extended notion of architecture that is able to capture such requirements.

Architectures with Observations
In order to express requirements such as the broadcast property expected in the election example discussed in the previous section, we now develop an extended notion of architecture that captures both upper and lower bounds on information flow.One of the well-known mechanisms by which agents obtain common knowledge is by simultaneous direct observation of their environment.Our extended architectural format is obtained by adding a new type of edge that represents observability of information.
An architecture with filter functions and direct observations (henceforth, simply architecture) is a tuple A = (D, , ), where , where L is a set of function names not including the special symbol , and 3.
We define architectural interpretations I and interpreted architectures (A, I) exactly as in the previous section (except that now A also contains a relation .) This definition extends the definition of Chong and van der Meyden [2009] discussed in the previous section by adding edges of the form u v to express that domain v can observe domain u.More specifically, domain v is required to observe the observations of domain u, in the sense that for all states s and t, if u v and obs v (s) = obs v (t) then obs u (s) = obs u (t).This imposes a lower bound on information flow in the system.
As in the previous section, we assume the properties Arch1 and Arch2 on the relation .Further, we assume that the relation is transitive and acyclic (hence irreflexive).Note that the semantic explanation of given above is transitive, so the assumption of transitivity of is without loss of generality. 1 We write there is no f such that u f v then we write u v, and similarly write u v if not u v.
We define the semantics of an architecture with filter functions and direct observation by defining a function ftao u , and requiring that ftao u is the maximal information that u is allowed to learn given a particular execution of the system.We also impose the lower bound on information flow mentioned above.Function ftao u is similar to fta u , but (in order to be compatible with the lower bound) also incorporates the observations of domains v such that v u.To define ftao u , we first define the directly observable domains DO(u) = {v | v u} to be the set of domains that u directly observes according to relation .Note that u / ∈ DO(u).Also, we define an operator + such that x + y = if x = y = , and x + y = (x, y) otherwise.For any sequence sequence σ we treat σ as identical to σ.

Given architecture (D, ,
), interpretation I = (A, dom, I), and machine M = S, s 0 , A, step, obs, dom , the function ftao u is defined inductively by ftao u ( ) = , and, for α ∈ A * and a ∈ A, ftao u (αa) = ftao u (α)(x + y) where Intuitively, ftao u (α) represents the maximal information that domain u is permitted to have acquired after the sequence of actions α.The value x + y represents the new information that u is permitted to acquire when the action a is performed after sequence α.The component x expresses that this includes the action a plus any information that dom(a) may have, if dom(a) u, or that information filtered by the function I(f ) if dom(a) f u.Moreover, the component y expresses that domain u is allowed to learn the observation of any domain in DO(u), provided this differs from the previous observation.(As in the definition of view u , we reduce stuttering observations to a single copy to model asynchrony.) The following definition gives semantics to interpreted architectures by stating that the information possessed by u has an upper bound of ftao u and a lower bound of the directly observable information.("FTAO" is derived from filtered transmission of information about actions and observations.) ) and I = (A, dom, I) if it has the same components D, A and dom, and 1. for all agents u ∈ D and all α, α ∈ A * , if ftao u (α) = ftao u (α ) then obs u (α) = obs u (α ), and 2. for all agents u, v ∈ D and all s, t ∈ S if u v and obs v (s) = obs v (t) then obs u (s) = obs u (t).
In the following sections, we give some examples of architectural specifications based on architectures with filter functions and direct observations, and show that these specifications suffice to establish interesting epistemic specifications.

Example: Bulletin board
We begin with an example that shows that architectures with observations support the intuitions concerning broadcast discussed in Section 4.  Given two additional local assumptions about the behavior of the bulletin board, we can show that for distinct agents c and d, any c-local proposition that is known to d must also be known to everyone.Thus, the bulletin board architecture ensures that no agent has an unfair advantage over another agent: anything d can learn about c can also be learned by any other agent.
The first local assumption, BB1, guarantees that broadcast is not anonymous: any update to the bulletin board reveals the domain of the action that updated it.The second local assumption, BB2, ensures that the observable state of the bulletin board is a function of the previous observations of the bulletin board, and the current view of the poster.To state BB2, we first define seq(f, α), which, for any function f with domain A * , and any α ∈ A * , is the sequence of output of f on successively longer prefixes of α, with stuttering removed: BB2.For all α, β ∈ A * and a ∈ A, if seq(obs BB , α) = seq(obs BB , β) and view dom(a) (α) = view dom(a) (β) then obs BB (αa) = obs BB (βa).
The following theorem states that any bulletin board implementation M that satisfies the bulletin board architecture BB, and satisfies conditions BB1 and BB2, will not give any agent an unfair advantage over another: anything agent d can learn about agent c can be learned by everyone.
Theorem 1 Let M be FTAO-compliant with interpreted architecture AI = (BB, I) where I = (A, dom, I), and satisfy conditions BB1 and BB2.For distinct agents c, d Indeed, we can show that if agent c learns something about agent d, then it is common knowledge among all agents.

Example: Auction
Figure 4 shows architecture AUC, an auction architecture for n bidders, auctioneer Auc, and bulletin board BB.As in bulletin board architecture BB, bidders directly observe bulletin board BB.Unlike BB, bidders do not send information directly to the bulletin board, but instead submit bids to the trusted auctioneer, who posts auction results to the bulletin board.Information sent by the auctioneer to the bulletin board is bounded by the filter function denoted by results.
Intuitively, the auctioneer Auc and bulletin board BB are the trusted components of the system.We specify We refer to information sent from the auctioneer to the bulletin board as an announcement.We restrict our attention to interpretations I = (A, dom, I) for AUC that contain a distinguished announcement action ANN ∈ A Auc .Intuitively, ANN is the only action that can send information to the bulletin board.According to the architecture, the maximal information for an announcement is the result of the filter function I(results).We assume that this function has the following properties: A1.Only the ANN action may send information to the bulletin board.If a = ANN then I(results)(αa) = .
A2. Announcements depend only on the previous announcement, the individual behaviors of the bidders since the previous announcement (but not on their interleaving) and the action used to make the announcement.Let α, α ∈ A * , and let Constraints A1-A3 allows us to reason about the behavior of function results.Constraint A3 is the most complex, and it intuitively requires that any pair of behaviors of B i and B j that are each individually consistent with a given behavior of B k and a particular announcement, are jointly consistent with that announcement.There are several possible definitions for the function results that satisfy the constraints, such as a function that returns the maximum bid submitted since the last announcement, or a function that returns a list of the identity and bids of all bidders in decreasing order.
We also place local constraints on the bulletin board.Various implementations of the bulletin board are compatible with these constraints.For example, the bulletin board may display the complete history of the auctioneer's announcements, or only the latest announcement plus the sequence number of that announcement, or the last k announcements plus their sequence numbers.
The auction architecture and the local constraints allow us to prove that in any implementation that satisfies the auction architecture and local constraints, no bidder gains an information advantage over other bidders: if bidder B j learns something about bidder B , then it is common knowledge among all bidders.
Theorem 3 Let machine M be FTAO-compliant with interpreted architecture AI = (AUC, I) where I = (A, dom, I), and satisfy conditions A1-A3 and BB1-BB2.Let B = {B 1 , . . ., B n } be the set of all bidders.For distinct bidders B i and If we make further assumptions about the homogeneity of bidders, and that auctioneer announcements do not reveal bidder identities, we can prove that AUC provides bidder anonymity.
B1.All bidders have the same set of actions.For any bidder B i , the set of possible actions A Bi is {a Bi | a ∈ A B }, where A B is the set of bidder actions.
We define a bidder permutation P as a permutation over the set of bidders B. Since all bidders have the same set of actions, we can apply a permutation P to sequence α, written P (α).We define P ( ) = , P (αa) = P (α)a if dom(a) ∈ B, and P (αa Bi ) = P (α)a P (Bi) if a ∈ A B .We apply permutation P to proposition X ⊆ A * by applying P to each sequence α ∈ X: P (X) = {P (α) | α ∈ X}.Intuitively, if X is a u-local proposition, for u ∈ B, then P (X) is a P (u)-local proposition.Using bidder permutations, we can state the local constraint that announcements do not depend on (and thus do not reveal) bidder identities.
Given these additional local constraints we can show that if B i believes that some proposition X may be satisfied, then B i also believes that P (X) may be satisfied, for bidder permutation P .For example, if Bill considers it possible that Bob bid $10 and Bonnie bid $20, then Bill considers it possible that Bonnie bid $10 and Bob bid $20.
8 Example: Observational barriers Our last example is a general result that expresses a sense in which a group of domains can jointly function as an auditor of all information flows from one group of domains to another.
Let A = (D, , ) be an architecture, and let U, V, G ⊆ D. Say that G is an observational barrier separating U from V in A if there exists a partition (L, R) of D, such that U ∪ G ⊆ L, and V ⊆ R, there are no edges from R to L, and the only edges leading from L to R are observational edges leading from G.That is, • for all domains u ∈ R and v ∈ L, we have u v and u v, and • for all domains u ∈ L and v ∈ R, we have u v, and if u v then u ∈ G.
The following result states that if G is an observational barrier separating U from V , then if V has information about U , then that information is also available to G.This makes precise the sense in which G is able to audit information flowing from U to V .
Theorem 5 Let AI = (A, I) be an interpreted architecture and suppose that G is an observational barrier We remark that this result needs to be stated using group knowledge rather than distributed knowledge.Since |= D V φ ⇒ K V φ, we also have M, π |= D V p ⇒ K G p.However, it is not necessarily the case that M, π |= D V p ⇒ D G p. The reason has to do with the unsuitability of the notion of distributed knowledge for asynchronous settings: it is possible for V to have observed the ordering of changes to observations at G, where, because of asynchrony, this ordering is not distributed knowledge to G.This is illustrated in the following example.
Consider the architecture A and system M depicted in Figure 5.The architecture has four domains A, B, C, D. The system has states s 0 . . .s 3 , with initial state s 0 .We suppose that there are actions A = {a, b} with dom(a) = dom(b) = A. The effect of an action is only depicted when it changes the state, e.g., we draw the edge corresponding to s 0 • a = s 1 , but omit the self-loop corresponding to s 1 • a = s 1 .Associated with each state s in the diagram is a tuple (x, y), which is to be interpreted as giving the observation made by the agents B and C at that state.In particular, obs B (s) = x and obs C (s) = y.The observations of the other agents A and D are given as follows: obs A (s) = 0 and obs D (s) = (obs B (s), obs C (s)) for all states s.It is not difficult to check that M FTAOcomplies with A.
Suppose that we take U = {A}, G = {B, C} and V = {D} and let π(p) = {a} + b {a, b} * be the proposition that there has been an occurrence of a and an occurrence of b, but with the first a preceding the first b.It is evident that G is an observational barrier separating U from V and that π(p) is U -action-local.We claim that M, π, ab |= D V p ∧ ¬D G p. For this, note that view D (ab) = (0, 0) (1, 0) (1, 1).Thus, any sequence α such that view Since D is able to synchronously observe everything that these agents observe, it knows the order of a and b.However, this is not distributed knowledge to B and C because the semantics of distributed knowledge does not combine their information until after the run is complete, and the ordering information is, because of asynchrony, lost from the views of both agents.Similar issues with distributed knowledge in asynchronous settings are discussed have been noted previously [Moses and Bloom, 1994;van der Meyden, 2008].

Conclusion
We have introduced a new type of specification of the architecture of a multi-agent system, that enables upper bounds on causal effects and lower bounds on observed information to be expressed.We have given this type of specification a formal semantics that extends previous ideas from the literature on computer security.Using this specification format, we have shown by a number of examples that it is possible to derive security properties stated in epistemic logic from highly abstract assumptions that concern the architectural structure of a system and the behavior of certain trusted components, even when other components have unknown behavior.These results contribute to the formal foundations of the MILS vision that secure systems can be constructed by composition of trusted and untrusted components in the context of a specific architectural structure.
The term "agent architecture" is much used in work on intelligent agents and multi-agent systems; this area is surveyed in Wooldridge and Jennings [1994].The emphasis in this literature is on systems built on some form of Artificial Intelligence capability.Many specific agent-oriented architectures have been proposed.Generally, these concern a much more concrete notion of ar-chitecture than we have discussed here.For example, BDI-architectures [Rao and Georgeff, 1991] posit that an agent should be designed around the idea that it has beliefs, desires and intentions, and that the dynamics for these notions may be expressed in a multi-modal logic.Our work is considerably more abstract, and intended to apply broadly to all types of computational systems, and is therefore more closely related to the notion of architecture as studied in software engineering.We refer the reader to Chong and van der Meyden [2009] for an extended discussion of related work on architectural modelling formats from software engineering and on connections between architecture and security properties.
The literature on Bayesian nets [Pearl, 2000] uses diagrams that have some similarity to our diagrams restricted to the case of edges labelled .However, Bayesian nets are interpreted over a static set of states, rather than in a dynamic systems model such as we have considered.A more temporal view of causality is taken in some work in the literature, e.g., Halpern and Pearl [2001], but here the emphasis is on defining when one event can be understood as a cause of another, rather than on specifying the causal structure of a system.
Our objective in this paper has been to develop an extended architectural format that is able to express lower bounds on information flow and to demonstrate its usefulness for enforcing security properties expressed in epistemic logic.This contribution leaves open many questions for future research.It remains to develop practical proof techniques and implementation strategies that ensure that a system complies with our extended format.For example, it would be useful to generalize the refinement techniques developed in Chong and van der Meyden [2009] to the extended format.Further examples should also be developed to validate the MILS methodology to secure systems construction and our approach to its formalization.
a deterministic transition relation, dom : A → D associates a domain with each action, and observation function obs : D × S → O describes for each state what observations can be made by each domain, for some set of observations O.

Figure 5 :
Figure 5: Observational barrier example D (ab) = view D (α) must contain both an a before any b (to cause the observation (1, 0)) and a subsequent b (to cause the observation (1, 1)), hence M, π, α |= p.This means that M, π, ab |= K V p, or equivalently, M, π, ab |= D V p, On the other hand, we have view B (ab) = 01 = view B (ba) and view C (ab) = 01 = view C (ba), so ab ≈ D G ba. Since M, π, ba |= p, this means that M, π, ab |= D G p, Intuitively, in this example, B detects the first occurrence of a and C detects the first occurrence of b.
G φ is satisfied if the group of domains G have knowledge of type K of formula φ, where K ranges over K, D, E, and C. The formula D G φ says G has distributed knowledge of φ; E G φ says every domain in G knows φ; and C G φ says G has common knowledge of φ.The operator K G represents a novel distributedknowledge-like notion that we call group knowledge.