Detection of Sparse and Weak Effects in High-Dimensional Supervised Learning Problems, Applied to Human Microbiome Data
Year: 2020
Type: dissertation
Abstract: This project studies the signal detection and identification problem in high-dimensional noisy data and the possibility of using it on microbiome data. An extensive simulation study was performed on generated data using as well as a microbiome dataset collected on patients with Parkinson's disease, using Donoho and Jin's Higher criticism, Jager and Wellner's phi-divergence-based goodness-of-fit-test and Stepanova and Pavlenko's CsCsHM statistic . We present some novel approaches based on established theory that perform better than existing methods and show that it is possible to use the signal identification framework to detect differentially abundant features in microbiome data. Although the novel approaches produce good results, they lack substantial mathematical foundations and should be avoided if theoretical rigour is needed. We also conclude that while we have found that it is possible to use signal identification methods to find abundant features in microbiome data, further refinement is necessary before it can be properly used in research.
Author: Fred Lindahl
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W3171553001