Quantum versus classical learnability

Motivated by work on quantum black-box query complexity, we consider quantum versions of two well-studied models of learning Boolean functions: Angluin's (1988) model of exact learning from membership queries and Valiant's (1984) Probably Approximately Correct (PAC) model of learning from random examples. For each of these two learning models we establish a polynomial relationship between the number of quantum versus classical queries required for learning. Our results provide an interesting contrast to known results which show that testing black-box functions for various properties can require exponentially more classical queries than quantum queries. We also show that under a widely held computational hardness assumption there is a class of Boolean functions which is polynomial-time learnable in the quantum version but not the classical version of each learning model; thus while quantum and classical learning are equally powerful from an information theory perspective, they are different when viewed from a computational complexity perspective.


Motivation
In recent years many researchers have investigated the power of quantum computers which can query a black-box oracle for an unknown function [1,5,6,9,14,10,11,15,17,20,21,23,32,37]. The broad goal of research in this area is to understand the relationship between the number of quantum versus classical oracle queries which are required to answer various questions about the function computed by the oracle. For example, a well-known result due to Deutsch and Jozsa [17] shows that exponentially fewer queries are required in the quantum model in order to determine with certainty whether a black-box oracle computes a constant Boolean function or a function which is balanced between outputs and More recently, several researchers have studied the number of quantum oracle queries which are required to determine whether the function computed by a black-box oracle is identically zero [5,6,9,15,23,37].
A natural question which arises in this framework is the following: what is the relationship between the number of quantum versus classical oracle queries which are required in order to exactly identify the function computed by a black-box oracle? Here the goal is not to determine whether a black-box function satisfies some particular property such as ever taking a nonzero value, but rather to precisely identify an unknown black-box function from some restricted class of possible functions. The classical version of this problem has been well studied in the computational learning theory literature [2,12,22,24,25] and is known as the problem of exact learning from membership queries. The question stated above can thus be rephrased as follows: what is the relationship between the number of quantum versus classical membership queries which are required for exact learning? We answer this question in this paper.
In addition to the model of exact learning from membership queries, we also consider a quantum version of Valiant's widely studied PAC learning model which was introduced by Bshouty and Jackson [13]. While a learning algorithm in the classical PAC model has access to labeled examples drawn from some fixed probability distribution, a learning algorithm in the quantum PAC model has access to some fixed quantum superposition of labeled examples. Bshouty and Jackson gave a polynomial-time algorithm for a particular learning problem in the quantum PAC model, but did not address the general relationship between the number of quantum versus classical examples which are required for PAC learning. We answer this question as well.
"©2001 IEEE. Personal use of this material is permitted. However, permission to reprint/ republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE." Theorem 1 Let be any class of Boolean functions over and let and be such that is exact learnable from classical membership queries or from quantum membership queries. Then Our second main theorem is an analogous result for quantum versus classical PAC learnability:

Theorem 2 Let be any class of Boolean functions over and let and be such that is PAC learnable from classical examples or from quantum examples. Then
Theorems 1 and 2 are information-theoretic rather than computational in nature; they show that for any learning problem, if there is a quantum learning algorithm which uses polynomially many examples then there must also exist a classical learning algorithm which uses polynomially many examples. However, Theorems 1 and 2 do not imply that every polynomial time quantum learning algorithm must have a polynomial time classical analogue. In fact, we show that a separation exists between efficient quantum learnability and efficient clasical learnability. Under a widely held computational hardness assumption for classical computation (the hardness of factoring Blum integers), we observe that for each of the two learning models considered in this paper there is a concept class which is polynomial-time learnable in the quantum version but not in the classical version of the model.

Previous Work
Our results draw on lower bound techniques from both quantum computation and computational learning theory [2,5,6,8,12,24]. A detailed description of the relationship between our results and previous work on quantum versus classical black-box query complexity is given in Section 3.4.
In [19] Farhi et al. prove a lower bound on the number of functions which can be distinguished with quantum queries. Ronald de Wolf has noted [18] that the main result of [19] yields an alternate proof of one of the two lower bounds which we give for exact learning from quantum membership queries (Theorem 10).

Organization
We define the exact learning model and the PAC learning model and describe the quantum computation framework in Section 2. We prove the relationship between quantum and classical exact learning from membership queries (Theorem 1) in Section 3, and we prove the relationship between quantum and classical PAC learning (Theorem 2) in Section 4. Finally, in Section 5 we observe that under a widely accepted computational hardness assumption for classical computation, in each of these two learning models there is a concept class which is quantum learnable in polynomial time but not classically learnable in polynomial time.

Preliminaries
A concept over is a Boolean function over the domain or equivalently a concept can be viewed as a subset of A concept class is a collection of concepts, where is a concept over For example, might be the family of all Boolean formulae over variables which are of size at most We say that a pair is a labeled example of the concept While many different learning models have been proposed, most models follow the same basic paradigm: a learning algorithm for a concept class typically has access to (some kind of) an oracle which provides examples that are labeled according to a fixed but unknown target concept and the goal of the learning algorithm is to infer (in some sense) the target concept The two learning models which we discuss in this paper, the model of exact learning from membership queries and the PAC model, make this rough notion precise in different ways.

Classical Exact Learning from Membership Queries
The model of exact learning from membership queries was introduced by Angluin [2] and has since been widely studied [2,12,22,24,25]. In this model the learning algorithm has access to a membership oracle where is the unknown target concept. When given an input string in one time step the oracle returns the bit such an invocation is known as a membership query since the oracle's answer tells whether or not (viewing as a subset of ). The goal of the learning algorithm is to construct a hypothesis which is logically equivalent to i.e.
for all Formally, we say that an algorithm is an exact learning algorithm for using membership queries if for all for all if is given and access to then with probability at least algorithm outputs a Boolean circuit such that for all The sample complexity of a learning algorithm for is the maximum number of calls to which ever makes for any

Classical PAC Learning
The PAC (Probably Approximately Correct) model of concept learning was introduced by Valiant in [33] and has since been extensively studied [4,27]. In this model the learning algorithm has access to an example oracle where is the unknown target concept and is an unknown distribution over The oracle takes no inputs; when invoked, in one time step it returns a labeled example where is randomly selected according to the distribution The goal of the learning algorithm is to generate a hypothesis which is an -approximator for under i.e. a hypothesis such that An algorithm is a PAC learning algorithm for if the following condition holds: for all and for all for all distributions over if is given and access to then with probability at least algorithm outputs a circuit which is an -approximator for under The sample complexity of a learning algorithm for is the maximum number of calls to which ever makes for any concept and any distribution over

Quantum Computation
Detailed descriptions of the quantum computation model can be found in [7,16,28,36]; here we outline only the basics using the terminology of quantum networks as presented in [5]. A quantum network is a quantum circuit (over some standard basis augmented with one oracle gate) which acts on an -bit quantum register; the computational basis states of this register are the binary strings of length A quantum network can be viewed as a sequence of unitary transformations where each is an arbitrary unitary transformation on qubits and each is a unitary transformation which corresponds to an oracle call. 1 Such a network is said to have query complexity At every stage in the execution of the network, the current state of the register can be represented as a superposition where the are complex numbers which satisfy If this state is measured, then with probability the string is observed and the state collapses down to . After the final transformation takes place, a measurement is performed on some subset of the bits in the register and the observed value (a classical bit string) is the output of the computation.
Several points deserve mention here. First, since the information which our quantum network uses for its computation comes from the oracle calls, we may stipulate that the initial state of the quantum register is Second, as described above each can be an arbitrarily complicated unitary transformation (as long as it does not contain any oracle calls) which may require a large quantum circuit to implement. This is of small concern since we are chiefly interested in query complexity and not circuit size. Third, as defined above our quantum networks can make only one measurement at the very end of the computation; this is an inessential restriction since any algorithm which uses intermediate measurements can be modified to an algorithm which makes only one final measurement. Finally, we have not specified just how the oracle calls work; we address this point separately in Sections 3.1 and 4.1 for each type of oracle.
If and are two superpositions of basis states, then the Euclidean distance betweeen and is The total variation distance between two distributions and is defined to be The following fact (Lemma 3.2.6 of [7]), which relates the Euclidean distance between two superpositions and the total variation distance between the distributions induced by measuring the two superpositions, will be useful:

Fact 3 Let
and be two unit-length superpositions which represent possible states of a quantum register. If the Euclidean distance is at most then performing the same observation on and induces distributions and which have total variation distance at most

Quantum Membership Queries
A quantum membership oracle is the natural quantum generalization of a classical membership oracle : on input a superposition of query strings, the oracle generates the corresponding superposition of example labels. More formally, a gate maps the basis state (where and ) to the state If is a quantum network which has gates as its oracle gates, then each is the unitary transformation which maps (where and ) to . 2 Our oracle is identical to the well-studied notion of a quantum black-box oracle for [5,6,7,9,10,11,15,17,23,37].
A quantum exact learning algorithm for is a family of quantum networks where each network has a fixed architecture independent of the choice of with the following property: for all for all if 's oracle gates are instantiated as gates, then with probability at least the network outputs a representation of a (classical) Boolean circuit such that for all The quantum sample complexity of a quantum exact learning algorithm for is where is the query complexity of .

Lower Bounds on Classical and Quantum Exact Learning
Two different lower bounds are known for the number of classical membership queries which are required to exact learn any concept class. In this section we prove two analogous lower bounds on the number of quantum membership queries required to exact learn any concept class. Throughout this section for ease of notation we omit the subscript and write for A Lower Bound Based on Similarity of Concepts. Consider a set of concepts which are all "similar" in the sense that for every input almost all concepts in the set agree. Known results in learning theory state that such a concept class must require a large number of membership queries for exact learning. More formally, let be any subset of For and let denote the set of those concepts in which assign label to example i.e. Let be the fraction of such concepts in and let thus is the minimum fraction of concepts in which can be eliminated by querying on the string Let Finally, let be the minimum of across all such that Thus Intuitively, the inner corresponds to the fact that the oracle may provide a worst-case response to any query; the corresponds to the fact that the learning algorithm gets to choose the "best" query point and the outer corresponds to the fact that the learner must succeed no matter 2 Note that each only affects the first bits of a basis state. This is without loss of generality since the transformations can "permute bits" of the network. what subset of the target concept is drawn from. Thus is small if there is a large set of concepts which are all very similar in that any query eliminates only a few concepts from If this is the case then many membership queries should be required to learn formally, we have the following lemma which is a variant of Fact 2 from [12] (the proof is given in Appendix A):

Lemma 4 Any (classical) exact learning algorithm for must have sample complexity
We now develop some tools which will enable us to prove a quantum version of Lemma 4. Let be such that and let be a listing of the concepts in Let the typical concept for be the function defined as follows: for all is the bit such that (ties are broken arbitrarily; note that a tie occurs only if ). The typical concept need not belong to or even to The difference matrix is the zero/one matrix where rows are indexed by concepts in columns are indexed by strings in and iff By our choice of and the definition of each column of has at most ones, so the matrix norm of is Our quantum lower bound proof uses ideas which were first introduced by Bennett et al. [6]. Let be a fixed quantum network architecture and let be the corresponding sequence of transformations. For let be the state of the quantum register after the transformations up through have been performed (we refer to this stage of the computation as time ) if the oracle gate is As in [6], for let the query magnitude of string at time with respect to , be the sum of the squared magnitudes in of the basis states which are querying on string at time so if then The quantity can be viewed as the amount of amplitude which the network invests in the query string to at time Intuitively, the final outcome of 's computation cannot depend very much on the oracle's responses to queries which have little amplitude invested in them. Bennett  implies that there is a set of cardinality at most such that for all we have Let be any two concepts in By Fact 3, the probability that outputs a circuit equivalent to can differ by at most if 's oracle gates are as opposed to and likewise for versus It follows that the probability that outputs a circuit equivalent to can differ by at most if 's oracle gates are as opposed to but this contradicts the assumption that is a quantum exact learning algorithm for Known upper bounds on the query complexity of searching a quantum database [9,23] can easily be used to show that Theorem 7 is tight up to constant factors.
A Lower Bound Based on Concept Class Size. A second reason why a concept class can require many membership queries is its size. Angluin [2] has given the following simple bound, incomparable to the bound of Lemma 4, on the number of classical membership queries required for exact learning (the proof is given in Appendix A):

Lemma 8 Any classical exact learning algorithm for must have sample complexity
In this section we prove a variant of this lemma for the quantum model. Our proof uses ideas from [5] so we introduce some of their notation. Let For each concept let be a vector which represents as an -tuple, i.e.
where is the binary representation of ¿From this perspective we may identify with a subset of and we may view a gate as a black-box oracle for which maps basis state to Using ideas from [20,21] 2. For any we have (since the total probability across all possible observations is 1).

Let
For any let be the column vector which has a coordinate for each monic multilinear monomial over of degree at most Thus, for example, if and we have and If is a column vector in then corresponds to the degree-polynomial whose coefficients are given by the entries of For let be the column vector which corresponds to the coefficients of the polynomial Let be the matrix whose -th row is note that multiplication by defines a linear transformation from to . Since is precisely the product is a column vector in which has as its -th coordinate. Now let be the matrix whose -th column is the vector A square matrix is said to be diagonally dominant if for all Properties (1) and (2) above imply that the transpose of is diagonally dominant. It is well known that any diagonally dominant matrix must be of full rank (a proof is given in Appendix C). Since is full rank and each column of is in the image of it follows that the image under of is all of and hence Finally, since we have which proves the theorem.
The lower bound of Theorem 10 is nearly tight as witnessed by the following example: let be the collection of all parity functions over so each function in is defined by a string and The quantum algorithm which solves the well-known Deutsch-Jozsa problem [17] can be used to exactly identify and thus learn the target concept with probability 1 from a single query. It follows that the factor of in the denominator of Theorem 10 cannot be replaced by any function

Quantum and Classical Exact Learning are Equivalent
We have seen two different reasons why exact learning a concept class can require a large number of classical membership queries: the class may contain many similar concepts (i.e. is small), or the class may contain very many concepts (i.e. is large). The following lemma, which is a variant of Theorem 3.1 from [24], shows that these are the only reasons why many membership queries may be required (the proof is given in Appendix A).

Lemma 11
There is an exact learning algorithm for which has sample complexity Combining Theorem 7, Theorem 10 and Lemma 11 we obtain the following relationship between the quantum and classical sample complexity of exact learning:

Theorem 1 Let be any concept class over and let and be such that is exact learnable from classical membership queries or from quantum membership queries. Then
We note that a oracle can clearly be used to simulate an oracle, so as well.

Discussion
Theorem 1 provides an interesting contrast to several known results for black-box quantum computation. Let denote the set of all functions from to Beals et al. [5] have shown that if is any total function (i.e. is defined for every possible concept over ), then the query complexity of any quantum network which computes is polynomially related to the number of classical black-box queries required to compute Their result is interesting because it is well known [7,11,17,32] that for certain concept classes and partial functions the quantum black-box query complexity of can be exponentially smaller than the classical black-box query complexity.
Our Theorem 1 provides a sort of dual to the results of Beals et al.: their bound on query complexity holds only for the fixed concept class but for any function while our bound holds for any concept class but only for the fixed problem of exact learning. In general, the problem of computing a function from black-box queries can be viewed as an easier version of the corresponding exact learning problem: instead of having to figure out only one bit of information about the unknown concept (the value of ), for the learning problem the algorithm must identify exactly. Theorem 1 shows that for this more demanding problem, unlike the results in [7,11,17,32] there is no way of restricting the concept class so that learning becomes substantially easier in the quantum setting than in the classical setting.

The Quantum Example Oracle
Bshouty and Jackson [13] have introduced a natural quantum generalization of the standard PAC-model example oracle. While a standard PAC example oracle generates each example with probability where is a distribution over a quantum PAC example oracle generates a superposition of all labeled examples, where each labeled example appears in the superposition with amplitude proportional to the square root of More formally, a gate maps the initial basis state to the state (We leave the action of a gate undefined on other basis states, and stipulate that any quantum network which includes gates must have all gates at the "bottom of the circuit," i.e. no gate may occur on any wire between the inputs and any gate.) A quantum network with gates is said to be a QEX network with query complexity A quantum PAC learning algorithm for is a family of QEX networks with the following property: for all and for all for all distributions over if the network has all its oracle gates instantiated as gates, then with probability at least the network outputs a representation of a circuit which is an -approximator to under The quantum sample complexity of a quantum PAC algorithm is the query complexity of

Lower Bounds on Classical and Quantum PAC Learning
Throughout this section for ease of notation we omit the subscript and write for We view each concept as a subset of For we write to denote so is the number of different "dichotomies" which the concepts in induce on the points in A subset is said to be shattered by if i.e. if induces every possible dichotomy on the points in The Vapnik-Chervonenkis dimension of , VC-DIM is the size of the largest subset which is shattered by Well-known results in computational learning theory show that the Vapnik-Chervonenkis dimension of a concept class characterizes the number of calls to which are information-theoretically necessary and sufficient to PAC learn For the lower bound, the following theorem is a slight simplification of a result due to Blumer et al. ([8], Theorem 2.1.ii.b); a proof sketch is given in Appendix A.

Theorem 12 Let be any concept class and VC-DIM Then any (classical) PAC learning algorithm for must have sample complexity
We now state a quantum analogue of the classical lower bound given by Theorem 12; the proof uses ideas from error-correcting codes and is given in Appendix B.

Theorem 13 Let be any concept class and VC-DIM Then any quantum PAC learning algorithm for must have quantum sample complexity
Since the class of parity functions over has VCdimension as in Theorem 10 the in the denominator of Theorem 13 cannot be replaced by any function

Quantum and Classical PAC Learning are Equivalent
A well-known theorem due to Blumer et al. (Theorem 3.2.1.ii.a of [8]) shows that VC-DIM also upper bounds the number of calls required for (classical) PAC learning:

Theorem 14 Let be any concept class and VC-DIM
There is a classical PAC learning algorithm for which has sample complexity The proof of Theorem 14 is quite complex so we do not attempt to sketch it. As in Section 3.3, this upper bound along with our lower bound from Theorem 13 together yield:

Theorem 2 Let be any concept class over and let and be such that is PAC learnable from classical examples or from quantum examples. Then
We note that a oracle can be used to simulate the corresponding oracle by immediately performing an observation on the gate's outputs 3 (such an observation yields each example with probability ), and thus

Quantum versus Classical Efficient Learnability
We have shown that from an information-theoretic perspective, up to polynomial factors quantum learning is no more powerful than classical learning. However, we now observe that the apparant computational advantages of the quantum model yield efficient quantum learning algorithms which seem to have no efficient classical counterparts.
A Blum integer is an integer where are -bit primes each congruent to 3 modulo 4. It is widely believed that there is no polynomial-time classical algorithm which can successfully factor a randomly selected Blum integer with nonnegligible success probability.
Kearns and Valiant [26] have constructed a concept class whose PAC learnability is closely related to the problem of factoring Blum integers. In their construction each concept is uniquely defined by some Blum integer Furthermore, has the property that if then the prefix of is the binary representation of Kearns and Valiant prove that if there is a polynomial time PAC learning algorithm for then there is a polynomial time algorithm which factors Blum integers. Thus, assuming that factoring Blum integers is a computationally hard problem for classical computation, the Kearns-Valiant concept class is not efficiently PAC learnable.
On the other hand, in a celebrated result Shor [31] has exhibited a poly size quantum network which can factor any -bit integer with high success probability. Since each positive example of a concept reveals the Blum integer which defines using Shor's algorithm it is easy to obtain an efficient quantum PAC learning algorithm for the Kearns-Valiant concept class. We thus have

Observation 15
If there is no polynomial-time classical algorithm for factoring Blum integers, then there is a concept class which is efficiently quantum PAC learnable but not efficiently classically PAC learnable.
The hardness results of Kearns and Valiant were later extended by Angluin and Kharitonov [3]. Using a publickey encryption system which is secure against chosencyphertext attack (based on the assumption that factoring Blum integers is computationally hard for polynomial-time algorithms), they constructed a concept class which cannot be learned by any polynomial-time learning algorithm which makes membership queries. As with the Kearns-Valiant concept class, though, using Shor's quantum factoring algorithm it is possible to construct an efficient quantum exact learning algorithm for this concept class. Thus, for the exact learning model as well, we have:

Observation 16
If there is no polynomial-time classical algorithm for factoring Blum integers, then there is a concept class which is efficiently quantum exact learnable from membership queries but not efficiently classically exact learnable from membership queries.
Servedio [30] has recently established a stronger separation between the quantum and classical models of exact learning from membership queries than is implied by Observation 16. Using a new construction of pseudorandom functions in conjunction with Simon's quantum oracle algorithm [32], it is shown in [30] that if any one-way function exists then there is a concept class which is efficiently quantum exact learnable from membership queries but not efficiently classically exact learnable from membership queries.

Conclusion and Future Directions
While we have shown that quantum and classical learning are (up to polynomial factors) information-theoretically equivalent, many interesting questions remain about the relationship between efficient quantum and classical learnability. It would be interesting to develop efficient quantum learning algorithms for natural concept classes, such as the polynomial-time quantum algorithm of Bshouty and Jackson [13] for learning DNF formulae from uniform quantum examples.

A Bounds on Classical Sample Complexity
Proof of Lemma 4: Let be such that Consider the following adversarial strategy for answering queries: given the query string answer the bit which maximizes This strategy ensures that each response eliminates at most a fraction of the concepts in After membership queries, fewer than half of the concepts in have been eliminated, so at least two concepts have not yet been eliminated. Consequently, it is impossible for to output a hypothesis which is equivalent to the correct concept with probability greater than (Lemma 4) Proof of Lemma 8: Consider the following adversarial strategy for answering queries: if is the set of concepts which have not yet been eliminated by previous responses to queries, then given the query string answer the bit such that Under this strategy, after membership queries at least two possible target concepts will remain.

Proof of Lemma 11:
Consider the following learning algorithm : at each stage in its execution, if is the set of concepts in which have not yet been eliminated by previous responses to queries, algorithm 's next query string is the string which maximizes By following this strategy, each query response received from the oracle must eliminates at least a fraction of the set so with each query the size of the set of possible target concepts is multiplied by a factor which is at most Consequently, after queries, only a single concept will not have been eliminated; this concept must be the target concept, so can output a hypothesis which is equivalent to (Lemma 11) Proof Sketch for Theorem 12: The idea behind Theorem 12 is to consider the distribution which is uniform over some shattered set of size and assigns zero weight to points outside of Any learning algorithm which makes only calls to will have no information about the value of on at least points in moreover, since the set is shattered by any labeling is possible for these unseen points. Since the error of any hypothesis under is the fraction of points in where and the target concept disagree, a simple analysis shows that no learning algorithm which perform only calls to can have high probability (e.g.

B Proof of Theorem 13
Let be a set which is shattered by and let be the distribution which is uniform on and assigns zero weight to points outside If is a Boolean function on we say that the relative distance of and on is the fraction of points in on which and disagree. We will prove the following result which is stronger than Theorem 13: Let be a quantum network with gates such that for all if 's oracle gates are gates, then with probability at least the output of is a hypothesis such that the relative distance of and on is at most We will show that such a network must have query complexity at least Since any QEX network with query complexity can be simulated by a QMQ network with query complexity taking and will prove Theorem 13. The argument is a modification of the proof of Theorem 10 using ideas from error correcting codes. Let be a quantum network with query complexity which satisfies the following condition: for all if 's oracle gates are gates, then with probability at least the output of is a representation of a Boolean circuit such that the relative distance of and on is at most By the well-known Gilbert-Varshamov bound from coding theory (see, e.g., Theorem 5.1.7 of [34]), there exists a set of -bit strings such that for all the strings and differ in at least bit positions, where (Here is the binary entropy function.) For each let be a concept such that the -bit string is (such a concept must exist since the set is shattered by ). For let be the collection of those basis states which are such that if the final observation performed by yields a state from then the output of is a hypothesis such that and have relative distance at most on Since each pair of concepts has relative distance at least on the sets and are disjoint for all As in Section 3.2 let and let where is the -tuple representation of the concept By Lemma 9, for each there is a real-valued multilinear polynomial of degree at most such that for all the value of is precisely the probability that the final observation on yields a state from provided that the oracle gates are gates. Since, by assumption, if is the target concept then with probability at least generates a hypothesis which has relative distance at most from on the polynomials have the following properties: 1.
for all 2. For any we have that (since the 's are disjoint and the total probability across all observations is 1).

Let
and be defined as in the proof of Theorem 10. For let be the column vector which corresponds to the coefficients of the polynomial so Let be the matrix whoseth row is the vector so multiplication by is a linear transformation from to The product is a column vector in which has as its -th coordinate. Now let be the matrix whose -th column is the vector As in Theorem 10 we have that the transpose of is diagonally dominant, so is of full rank and hence Since we thus have that and the theorem is proved. (Theorem 13)

C A diagonally dominant matrix has full rank
This fact follows from the following theorem (see, e.g., Theorem 6.1.17 of [29]).

Theorem 17 (Gershgorin's Circle Theorem) Let
be a real or complex-valued matrix. Let be the disk in the complex plane whose center is and whose radius is Then every eigenvalue of lies in the union of the disks The proof is well known: if is an eigenvalue of which has corresponding eigenvector then since we have for Without loss of generality we may assume that so for some and for Thus and hence is in the disk For a diagonally dominant matrix the radius of each disk is less than its distance from the origin, which is Hence cannot be an eigenvalue of a diagonally dominant matrix, so the matrix must have full rank.