s-news
[Top] [All Lists]

Re: array with NA values

To: s-news@wubios.wustl.edu
Subject: Re: array with NA values
From: Tim Hesterberg <timh@insightful.com>
Date: 6 Apr 2006 15:05:24 -0700
In-reply-to: <SE2KEXCH01BEzJvlZGd00001109@se2kexch01.insightful.com> (message from Tim Hesterberg on 6 Apr 2006 14:44:31 -0700)
References: <20060405110338.83318.qmail@web33902.mail.mud.yahoo.com> <SE2KEXCH01BEzJvlZGd00001109@se2kexch01.insightful.com>
My apologies; I used my own function "omit.na" in the example below.
Thanks to Chris Barker for noticing this.

omit.na <- function(x) x[!is.na(x)]

Tim Hesterberg


>If you are doing many operations with the same data, you might
>want to convert your data into a list with missing values omitted.
>Each element of the list would be a vector containing non-missing values.
>Optionally, you can give the list dimensions.
>
>x <- array(1:1000, c(10,10,10))
>set.seed(0)
>x[runif(1000) > .4] <- NA
>
>temp <- x
>dim(temp) <-  c(10, prod(dim(x)[2:3]))
>x2 <- lapply(1:ncol(temp), function(j, x) omit.na(x[,j]), x = temp)
>x2  # list of length 100, containing non-missing values for each
>
># Now can turn x2 into a matrix/list hybrid if you like,
># to let you use apply on rows or columns
>dim(x2) <- dim(x)[2:3]
>
># example using apply:
>apply(x2, 2, function(x) length(unlist(x)))
>
>Tim Hesterberg
>
>>Dear all,
>>
>>I have numerous 3-dimenional arrays and each array contains no entire row or 
>>column with NA values. However, 50-60% of the values in the array are NA 
>>values. I perform various operations on the array using the apply statement. 
>>However, because the arrays are quite large the calculations take a while to 
>>finish. Within the apply statement I reduce the number of calculations on the 
>>vector by using the !is.na() statement. Is there a faster way to run the 
>>apply command without the need to read in an enitire vector from the array 
>>which contains a large amount of NA values, i.e. should I store the data in 
>>another format to avoid having an array with a large amount of NA values.
>>
>>For example, if I have a 3-dimensional 1000x1000x1000 and use the apply 
>>statement on dimensions 2 and 3, I'm reading in vectors of length 1000 in the 
>>apply statement and most of the 1000 values might be NAs. Can this be avoided?
>>
>>I would greatly appreciate any help on this.
>>Dave
>


<Prev in Thread] Current Thread [Next in Thread>