DASH
Repository logo
Communities & Collections
Browse DASH
DASH Stories
Statistics
  1. Home
  2. Harvard Division of Continuing Education
  3. DCE Theses and Dissertations
  4. Theoretical and Computational aspects of Polynomial Neural Networks: Training Stability, Algorithmic Complexity and Expressivity

Theoretical and Computational aspects of Polynomial Neural Networks: Training Stability, Algorithmic Complexity and Expressivity

Open/View files
Letourneau_PierreDavid_Thesis_May2025.pdf (560.43 KB)
Date
April 23, 2025
Author(s)
Letourneau, Pierre-David
The Harvard community has made this article openly available. Please share how this access benefits you.
Citation
Letourneau, Pierre-David. 2025. Theoretical and Computational aspects of Polynomial Neural Networks: Training Stability, Algorithmic Complexity and Expressivity. Masters Thesis, Harvard University Division of Continuing Education.
Abstract
This thesis is focused on the study of Polynomial Neural Networks (PNN) and their properties. PNNs are neural networks where the activation functions are themselves polynomials. The main result of the thesis focuses on the maximum learning rate for the stable training of polynomial neural networks in the ultra-rich regime; recent empirical bounds were observed showing a root-like (γ1/d) behavior of the maximum learning rate as a function of the richness factor γ. This thesis provides a fundamental and theoretical explanation of this observed phenomena in the cases of PNNs as well as transformer networks. Two more topics are investigated as part of this thesis: the establishment of a quasi-polynomial time algorithm for the training problem for PNNs, as well as a computational framework for the study of the expressivity of PNNs.
Terms of Use
This article is made available under the terms and conditions applicable to Other Posted Material (LAA), as set forth at Terms of Service
URI
https://dash.harvard.edu/handle/1/42719639
Subjects

maximum learning rate...

neural network expres...

non-convex optimizati...

polynomial neural net...

training complexity

ultra-rich training r...

Computer science

Applied mathematics

Artificial intelligen...

Collection
DCE Theses and Dissertations
Metrics
Repository logo
  • CONTACT US
  • OSRDS
  • ABOUT DASH
  • DEPOSITING TO DASH
  • TERMS OF USE
  • REQUEST A WAIVER
  • TAKE DOWN POLICY
  • NAME CHANGE POLICY
  • HARVARD OA POLICIES
  • DASH SUBMITTER AGREEMENT
  • Accessibility
  • Privacy
Creative Commons License BY
Creative Commons Attribution 4.0 International License

Except where otherwise noted, this work is subject to a Creative Commons Attribution 4.0 International License,
which allows anyone to share and adapt our material as long as proper attribution is given.
For details and exceptions, see the Harvard Library Copyright Policy
©2024 Presidents and Fellows of Harvard College.