Nonparametric estimation of correlation functions in longitudinal and spatial data, with application to colon carcinogenesis experiments

Abstract

This reprint differs from the original in pagination and typographic detail. 1 2 Y. LI ET AL.have strong correlations, which are stationary in time or distance lags. The structure for the observation at a particular time or location within one subject can be very general, for example, a vector or even a function.Our study arises from a colon carcinogenesis experiment. The biomarker that we are interested in is p27, which is a life cycle protein that affects cell apoptosis, proliferation and differentiation. An important goal of the study is to understand the function of p27 in the early stage of the cancer development process. In the experiment, 12 rats were administered azoxymethane (AOM), which is a colon specific carcinogen. After 24 hours, the rats were terminated and a segment of colon tissue was excised from each rat. About 20 colonic crypts were randomly picked along a linear slice on the colon segment. The physical distances between the crypts were measured. Then, within each crypt, we measured cells at different depths within the crypts, and then the expression level of p27 was measured for each cell within the chosen crypts. In this data set, crypts are naturally functional data (Ramsay and Silverman [13]), in that the responses within a crypt are coordinated by cell depths. There is a literature about similar data, for example, Morris et al. [11].However, in this paper we will be focused on a very different perspective. In this application the spatial correlation between crypts is of biological interest, because it helps answer the question: if we observe a crypt with high p27 expression, how likely are the neighboring crypts to have high p27 expression? We will phrase much of our discussion in terms of this example, but as seen in later sections, we have a quite general structure that includes time series as a special case. In that context, the asymptotic theory is as the number of "time series locations," that is, crypts, increases to infinity.Although motivated by a very specific problem, nonparametric covariance/correlation estimators are worth being investigated in their own right. They can be used in a statistical analysis as: (a) an exploratory device to help formulate a parametric model, (b) an intermediate tool to do spatial prediction (kriging), (c) a diagnostic for parametric models and (d) a robust tool to test correlation. Understanding the theoretical properties of the nonparametric estimator is important under any of these situations. A limiting distribution theory would be especially valuable for purpose (d).There is previous work on the subject of nonparametric covariance estimation. Hall, Fisher and Hoffmann [7] developed an asymptotic convergence rate of a kernel covariance estimator in a time series setting. They required not only an increasing time domain, but increasingly denser observations. Diggle and Verbyla [5] suggested a kernel-weighted local linear regression estimator for estimating the nonstationary variogram in longitudinal data, without developing asymptotic theory. Gua...

Generalized Functional Linear Models With Semiparametric Single-Index Interactions

“…The colon carcinogenesis data that we are using are similar to those analyzed in Morris et al (2001, 2003), Li et al (2007) and Baladandayuthapani et al (2008). The biomarker of interest in this experiment is p27, which is a protein that inhibits the cell cycle.…”

“…We have m = 25–30 observations (cells) on each function. In the literature, it has been noted that there is spatial correlation among the crypts within the same rat (Li et al, 2007, Baladandayuthapani et al, 2008). In this experiment, we sampled crypts sufficiently far apart so that the spatial correlations are negligible, and thus we can assume that the crypts are independent.…”

Semiparametric regression during 2003–2007

“…They use penalized regression splines and develop a novel method for smoothing parameter selection that overcomes the well-known biases of cross-validation with correlated data. Li et al [163] show how to estimate a correlation function in longitudinal and spatial data. Both papers give applications to colon carcinogenesis experiments.…”

Modeling clinical outcome using multiple correlated functional biomarkers: A Bayesian approach

“…Of note, colon crypt data similar to those in the MAP II study have been analyzed by Morris et al 13 , Apanasovich et al 14 , Li et al 15 , and Baladandayuthapani et al 16 among others. These studies, however, are fundamentally different from ours in that their primary goal was to model and estimate the biomarker functions along the length of crypts and/or test correlations among these biomarker functions, and crypt-level biomarker values are used as the outcome of interest.…”