Show simple item record

dc.contributor.authorChung, Sue Yeon
dc.date.accessioned2019-08-09T09:00:13Z
dash.embargo.terms2018-05-01
dc.date.created2017-05
dc.date.issued2017-05-11
dc.date.submitted2017
dc.identifier.citationChung, Sue Yeon. 2017. Statistical Mechanics of Neural Processing of Object Manifolds. Doctoral dissertation, Harvard University, Graduate School of Arts & Sciences.
dc.identifier.urihttp://nrs.harvard.edu/urn-3:HUL.InstRepos:41141361*
dc.description.abstractInvariant object recognition is one of the most fundamental cognitive tasks performed by the brain. In the neural state space, different objects with stimulus variabilities are represented as different manifolds. In this geometrical perspective, object recognition becomes the problem of linearly separating different object manifolds. In feedforward visual hierarchy, it has been suggested that the object manifold representations are reformatted across the layers, to become more linearly separable. Thus, a complete theory of perception requires characterizing the ability of linear readout networks to classify object manifolds from variable neural responses. A theoretical understanding of the perceptron of isolated points was pioneered by Elizabeth Gardner who formulated it as a statistical mechanics problem and analyzed it using replica theory. In this thesis, we generalize the statistical mechanical analysis and establish a theory of linear classification of manifolds synthesizing statistical and geometric properties of high dimensional signals. First, we study the theory of linear classification of simple spherical manifolds, such as line segments, L2 balls, or L1 balls. We provide analytical formula for classification capacity of balls, as a function of dimension, radius, and margin. We also find that the notion of support vectors needs to be generalized, and identify different support configurations of the manifolds, which has implications in generalization error. Next, we present a Maximum Margin Manifold Machine (M4), an efficient iterative algorithm that can find a maximum margin linear binary classifier for manifolds with an uncountable set of training samples per each manifold. We provide a convergence proof with a polynomial bound on the convergence time. We further generalize M4 for non-separable manifolds with slack variables. We report that the number of training examples required to achieve the same generalization error is much smaller for M4, compared with traditional support vector machines. Next, we generalize our theory further to linear classification of random general manifolds. We start with classification capacity of random ellipsoids, and generalize to classification capacity of general smooth and non-smooth manifolds. We identify that the capacity of a manifold is determined that effective radius, R_M, and effective dimension, D_M. Finally, we show extensions to directions relevant for applications to real data. We have extended our general manifold classification theory to incorporate correlated manifolds, mixtures of manifold geometries, sparse labels and nonlinear classifications. Then, we analyze how object-based manifolds reformat in a conventional deep network (GoogLeNet). We find that the deep network indeed changes the manifolds in the direction that the capacity is increased. This thesis lays the groundwork for a computational theory of neuronal processing of objects, providing quantitative measures for linear separability of object manifolds. We hope that our theory will provide new insights into the computational principles underlying processing of sensory representations in the brain. As manifold representations of the sensory world are ubiquitous in both biological and artificial neural systems, exciting future work lies ahead.
dc.description.sponsorshipEngineering and Applied Sciences - Applied Physics
dc.format.mimetypeapplication/pdf
dc.language.isoen
dash.licenseLAA
dc.subjectReplica Theory
dc.subjectPerceptron
dc.subjectSupport Vector Machines
dc.subjectObject Manifolds
dc.subjectPerceptual Invariance
dc.subjectObject Recognition
dc.subjectStatistical Mechanics
dc.titleStatistical Mechanics of Neural Processing of Object Manifolds
dc.typeThesis or Dissertation
dash.depositing.authorChung, Sue Yeon
dash.embargo.until2018-05-01
dc.date.available2019-08-09T09:00:13Z
thesis.degree.date2017
thesis.degree.grantorGraduate School of Arts & Sciences
thesis.degree.grantorGraduate School of Arts & Sciences
thesis.degree.levelDoctoral
thesis.degree.levelDoctoral
thesis.degree.nameDoctor of Philosophy
thesis.degree.nameDoctor of Philosophy
dc.contributor.committeeMemberSompolinsky, Haim
dc.contributor.committeeMemberAdams, Ryan P.
dc.contributor.committeeMemberValiant, Leslie G.
dc.contributor.committeeMemberCox, David
dc.type.materialtext
thesis.degree.departmentEngineering and Applied Sciences - Applied Physics
thesis.degree.departmentEngineering and Applied Sciences - Applied Physics
dash.identifier.vireo
dash.author.emailsueyeonchung@gmail.com


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record