A generalized sparse regression model with adjustment of pedigree structure for variant detection from next generation sequencing data

Next-generation sequencing technologies have been providing more comprehensive descriptions of rare and common sequence variants. Many powerful association tests have been developed for identifying significant individual common variants and genetic regions likely harboring rare and common variants. Single marker tests bear poor statistical powers to identify rare variant associations. Set-based tests sacrifice single marker resolution and require the set size to be much smaller than the sample size. Existing sparse regression algorithms can identify susceptible variants from a large set, even if its size far exceeds the sample size. Such algorithms are developed for analyzing sequence data of unrelated individuals and thus can be invalid in the presence of relatedness. Relatedness and population structure are two ubiquitous confounders in sequencing studies, especially those of admixed minorities. We hereby propose a flexible sparse regression model to jointly rectify relatedness, population structure and traditional covariates. Under this framework, we develop unweighted and weighted L_p (0<p<1) regularization algorithms for selecting a sparse set of susceptible genetic variants. Both the joint adjustment of confounders and the L_p regularizations proved effective for preventing the flow of false positives while maximizing the power to select susceptible genetic variants. Under extensive simulations, our new algorithms outperformed conventional sparse regression algorithms by identifying more causal variants while maintaining lower false discovery rates. In particular, our algorithms appeared to be equally efficient for identifying susceptible rare and common variants. Our algorithms should be useful to a large range of next generation sequencing studies.