Subject: Marine Geospatial Ecology Tools (MGET) help
Text archives
| From: | Liza Hoos <> |
|---|---|
| To: | "" <> |
| Subject: | [mget-help] glm, predict glm from table |
| Date: | Fri, 21 Sep 2012 17:59:14 -0400 |
Hello,
I am trying to run a GLM using MGET tools, but I keep getting an error when I use the Predict GLM from table tool. The error reads: "<type 'exceptions.ValueError'>: too many values to unpack Failed to execute (Predict GLM From Table)."
If I leave the "input table (optional)" field blank, the tool runs without problems. However, when I specify an input table it gives me this error (above, and see attachment for all details). Do you know what could be causing this problem? The table only has 1436 entries and about 15 fields.
Thanks so much!
Liza
Executing (Predict GLM From Table): GLMPredictFromArcGISTable
Z:\sharks\blacktip_bll\glm\blacktip_bll_glm4 bll_blacktip # # ""TestData" =
1" true # Z:\sharks\blacktip_bll\glm\blacktipglm4.emf tpr fpr true
Z:\sharks\blacktip_bll\glm\blacktipglmSS4 1000 3000 3000 10 white
Start Time: Fri Sep 21 17:17:47 2012
Running script GLMPredictFromArcGISTable...
GeoEco will now use ArcGIS Geoprocessor object 0x1EAE22A8 for ArcGIS
operations.
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
describe(*args=('C:\\Python26\\ArcGIS10.0\\lib\\site-packages\\GeoEco\\ArcGISToolbox\\Rasters\\dummyint',))...
ArcGIS Geoprocessor object 0x1EAE22A8: describe returned <geoprocessing
describe data object object at 0x1ED30F50>
ArcGIS Geoprocessor object 0x1EAE22A8: Get ParameterCount returned 19
This python script was invoked from an ArcGIS application.
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(0,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned
'Z:\\sharks\\blacktip_bll\\glm\\blacktip_bll_glm4'
Parameter inputModelFile = u'Z:\\sharks\\blacktip_bll\\glm\\blacktip_bll_glm4'
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(1,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned
'bll_blacktip'
Parameter inputTable = u'bll_blacktip'
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(2,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned ''
Parameter predictedValuesField = None
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(3,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned ''
Parameter cutoff = None
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(4,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned
'"TestData" = 1'
Parameter where = u'"TestData" = 1'
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(5,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned 'true'
Parameter ignoreOutOfRangeValues = True
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(6,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned ''
Parameter noDataValue = None
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(7,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned
'Z:\\sharks\\blacktip_bll\\glm\\blacktipglm4.emf'
Parameter outputPlotFile = u'Z:\\sharks\\blacktip_bll\\glm\\blacktipglm4.emf'
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(8,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned 'tpr'
Parameter measure1 = u'tpr'
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(9,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned 'fpr'
Parameter measure2 = u'fpr'
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(10,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned 'true'
Parameter colorize = True
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(11,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned
'Z:\\sharks\\blacktip_bll\\glm\\blacktipglmSS4'
Parameter outputSummaryFile = u'Z:\\sharks\\blacktip_bll\\glm\\blacktipglmSS4'
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(12,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned '1000'
Parameter res = 1000.0
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(13,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned '3000'
Parameter width = 3000.0
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(14,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned '3000'
Parameter height = 3000.0
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(15,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned '10'
Parameter pointSize = 10.0
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(16,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned 'white'
Parameter bg = u'white'
ArcGIS Geoprocessor object 0x1EAE22A8: Get OverwriteOutput returned 1
Parameter overwriteExisting = True
Executing method GeoEco.Statistics.Modeling.GLM.PredictFromArcGISTable.
Checking software dependency: R ROCR package.
R interpreter: Evaluating statements: library(utils)
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
R interpreter: Evaluating statements: library(tools)
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
R interpreter: Evaluating statements: .packages()
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
The R ROCR package has already been loaded into the R interpreter.
Checking software dependency: R e1071 package.
R interpreter: Evaluating statements: library(utils)
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
R interpreter: Evaluating statements: library(tools)
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
R interpreter: Evaluating statements: .packages()
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
The R e1071 package has already been loaded into the R interpreter.
Checking software dependency: R caret package.
R interpreter: Evaluating statements: library(utils)
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
R interpreter: Evaluating statements: library(tools)
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
R interpreter: Evaluating statements: .packages()
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
The R caret package has already been loaded into the R interpreter.
The file Z:\sharks\blacktip_bll\glm\blacktip_bll_glm4 exists.
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
exists(*args=('bll_blacktip',))...
ArcGIS Geoprocessor object 0x1EAE22A8: exists returned 1
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
describe(*args=('bll_blacktip',))...
ArcGIS Geoprocessor object 0x1EAE22A8: describe returned <geoprocessing
describe data object object at 0x1ED57F38>
ArcGIS Describe object 0x1ED57F38: Get DataType returned 'FeatureLayer'
The ArcGIS table or table view bll_blacktip exists.
The file Z:\sharks\blacktip_bll\glm\blacktipglm4.emf does not exist.
Directory Z:\sharks\blacktip_bll\glm will not be created because it already
exists.
The file Z:\sharks\blacktip_bll\glm\blacktipglmSS4 does not exist.
Directory Z:\sharks\blacktip_bll\glm will not be created because it already
exists.
R interpreter: Evaluating statements:
load("Z:\\sharks\\blacktip_bll\\glm\\blacktip_bll_glm4")
R interpreter: The last statement returned [u'model', u'xVar', u'yVar',
u'zVar', u'mVar', u'coordinateSystem']
R interpreter: Evaluating statements: exists("model")
R interpreter: The last statement returned True
R interpreter: Evaluating statements: tolower(class(model))
R interpreter: The last statement returned [u'glm', u'lm']
R interpreter: Evaluating statements: exists("rPackage")
R interpreter: The last statement returned False
R interpreter: Evaluating statements: class(model)[1]
R interpreter: The last statement returned u'glm'
Loaded glm from Z:\sharks\blacktip_bll\glm\blacktip_bll_glm4.
R interpreter: Evaluating statements: exists("rPackage")
R interpreter: The last statement returned False
The file C:\Python26\ArcGIS10.0\lib\site-packages\GeoEco\Statistics\Utils.r
exists.
R interpreter: Evaluating statements: # Utils.r
#
# Copyright (C) 2011 Jason J. Roberts
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License (available in the file LICENSE.TXT)
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
USA.
CheckForCancel <- function(timeLastChecked=NULL)
{
# If timeLastChecked is NULL, we are being called for the first
# time. If there happens to be a file named
# "GeoEcoCancelRequested_" followed by the process ID that was
# somehow left over from a previous process, delete it so we don't
# misinterpret it as an immediate cancel request. (This should
# never happen, but we do it anyway for safety.)
if (is.null(timeLastChecked))
{
tempDir <- Sys.getenv("TEMP")
if (tempDir == "")
tempDir <- Sys.getenv("TMP")
if (tempDir != "" && file.exists(file.path(tempDir,
sprintf("GeoEcoCancelReqested_%i", Sys.getpid()))))
unlink(file.path(tempDir, sprintf("GeoEcoCancelReqested_%i",
Sys.getpid())))
}
# If five seconds or more have elapsed since we last checked for a
# cancel request, check for one now.
if (is.null(timeLastChecked) || proc.time()[3] >= timeLastChecked + 5)
{
# Write a message with a special string. When the GeoEco
# Python function that handles messages recieves this string,
# it will check for a cancel request (and not log the
# message).
message("CHECK_FOR_CANCEL_REQUEST")
# Look in the TEMP directory file a file named
# "GeoEcoCancelRequested_" followed by the process ID. If we
# find it, call stop with a message indicating we were
# cancelled.
tempDir <- Sys.getenv("TEMP")
if (tempDir == "")
tempDir <- Sys.getenv("TMP")
if (tempDir != "" && file.exists(file.path(tempDir,
sprintf("GeoEcoCancelReqested_%i", Sys.getpid()))))
{
unlink(file.path(tempDir, sprintf("GeoEcoCancelReqested_%i",
Sys.getpid())))
stop("Cancel requested", call.=FALSE)
}
# We were not cancelled. Return the current time.
return(proc.time()[3])
}
# Less than five seconds have elapsed since we last checked for a
# cancel request. Just return the time we last checked.
return(timeLastChecked)
}
R interpreter: The last statement returned R closure function at 0x1ED1AAC0
The file
C:\Python26\ArcGIS10.0\lib\site-packages\GeoEco\Statistics\PredictModel.r
exists.
R interpreter: Evaluating statements: # PredictLMForArcGISRasters.r
#
# Copyright (C) 2008 Jason J. Roberts and Ben D. Best
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License (available in the file LICENSE.TXT)
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
USA.
ckappa <- function (r)
{
r <- na.omit(r)
r1 <- r[, 1]
r2 <- r[, 2]
n1 <- as.character(r1)
n2 <- as.character(r2)
lev <- levels(as.factor(c(n1, n2)))
p <- length(lev)
tab <- matrix(nrow = p, ncol = p)
dimnames(tab) <- list(levels(as.factor(c(n1, n2))),
levels(as.factor(c(n1,
n2))))
dim1 <- dimnames(tab)[[1]]
dim2 <- dimnames(tab)[[2]]
tabi <- table(n1, n2)
dimi1 <- dimnames(tabi)[[1]]
dimi2 <- dimnames(tabi)[[2]]
for (i in 1:p) for (j in 1:p) {
if ((sum(dim1[i] == dimi1) == 1) & (sum(dim2[j] == dimi2) ==
1))
tab[i, j] <- tabi[dim1[i], dim2[j]]
else tab[i, j] <- 0
}
tsum <- sum(tab)
ttab <- tab/tsum
tm1 <- apply(ttab, 1, sum)
tm2 <- apply(ttab, 2, sum)
agreeP <- sum(diag(ttab))
chanceP <- sum(tm1 * tm2)
kappa2 <- (agreeP - chanceP)/(1 - chanceP)
result <- list(table = tab, kappa = kappa2)
result
}
PrepareForPrediction <- function(model, trainingData, rPackage)
{
library(stats) # Not sure if this is still needed here.
# Look up the response variable and the minimum and maximum values of it
in
# the training data.
if (!is.null(rPackage) && rPackage == "party")
{
responseVariable <-
names(model@responses@variables)[1]
responseVariable <- gsub(" +", "", responseVariable)
if (grep("factor(.+)", responseVariable))
responseVariable <- substr(responseVariable, 8,
nchar(responseVariable) - 1)
stop("Support for the party package is not fully implemented: cannot
look up terms of fitted party model.", .call=FALSE)
}
else
{
responseVariable <- all.vars(model$terms)[1]
allPredictors <-
all.vars(model$terms)[2:length(all.vars(model$terms))]
}
minResponseValue <- min(trainingData[,responseVariable])
maxResponseValue <- max(trainingData[,responseVariable])
# Look up the min and max values for each predictor from the data
# that was used to fit the model.
minFittedPredictorValues <- list()
maxFittedPredictorValues <- list()
for (predictor in allPredictors)
{
minFittedPredictorValues[[predictor]] <-
min(c(trainingData[[predictor]]), na.rm=TRUE)
maxFittedPredictorValues[[predictor]] <-
max(c(trainingData[[predictor]]), na.rm=TRUE)
}
isBinaryClassification <- minResponseValue == 0 && maxResponseValue == 1
&&
(!is.null(rPackage) && rPackage == "rpart" &&
model$method == "class" ||
!is.null(rPackage) && rPackage ==
"randomForest" && model$type == "classification" ||
!is.null(rPackage) && rPackage == "party" &&
(model@responses@is_nominal[1]
||
model@responses@is_ordinal[1])
||
(!is.null(rPackage) && rPackage %in% c("mgcv",
"gam") || is.null(rPackage)) && model$family$family %in% c("binomial",
"quasibinomial", "negbin"))
isNonbinaryClassification <- !isBinaryClassification &&
(!is.null(rPackage) && rPackage == "rpart"
&& model$method == "class" ||
!is.null(rPackage) && rPackage ==
"randomForest" && model$type == "classification" ||
!is.null(rPackage) && rPackage == "party" &&
(model@responses@is_nominal[1]
||
model@responses@is_ordinal[1]))
# Build a vector listing the categorical predictors.
if (!is.null(rPackage) && rPackage %in% c("rpart", "randomForest"))
categoricalPredictors <- labels(attr(model$terms,
"dataClasses"))[attr(model$terms, "dataClasses") %in% c("factor", "ordered")]
else if (!is.null(rPackage) && rPackage == "party")
categoricalPredictors <-
names(model@data@get("input"))[lapply(model@data@get("input"),
class) %in% c("factor", "ordered")]
else if (!is.null(rPackage) && rPackage == "nlme")
categoricalPredictors <- c(labels(attr(model$terms,
"dataClasses"))[attr(model$terms, "dataClasses") %in% c("factor",
"ordered")], names(model$modelStruct$reStruct)) # For lme models, use
categorical predictors from fixed effects and all grouping variables from
random effects
else
categoricalPredictors <- names(model$xlevels)
categoricalPredictors <- categoricalPredictors[categoricalPredictors !=
responseVariable]
categoricalPredictors <- categoricalPredictors[categoricalPredictors !=
sprintf("factor(%s)", responseVariable)]
categoricalPredictorLevels <- list()
for (predictor in categoricalPredictors)
if (!is.null(rPackage) && rPackage %in% c("rpart", "nlme",
"randomForest", "party"))
categoricalPredictorLevels[[predictor]] <-
levels(trainingData[[predictor]])
else
categoricalPredictorLevels[[predictor]] <-
model$xlevels[[predictor]]
# Initialize some variables we will use to control warnings we may
# issue about out of range values, etc.
warnedAboutMissingLevels <- list()
for (predictor in categoricalPredictors)
warnedAboutMissingLevels[[predictor]] <- FALSE
allValuesForPredictorAreNA <- list()
for (predictor in allPredictors)
allValuesForPredictorAreNA[[predictor]] <- TRUE
allValuesAreNA <- TRUE
return(list(responseVariable, allPredictors, minResponseValue,
maxResponseValue, isBinaryClassification, isNonbinaryClassification,
minFittedPredictorValues, maxFittedPredictorValues, categoricalPredictors,
categoricalPredictorLevels, warnedAboutMissingLevels,
allValuesForPredictorAreNA, allValuesAreNA))
}
PredictModel <- function(model, rPackage, cutoff, ignoreOutOfRangeValues,
returnPredictedErrors, allPredictors, responseVariable, minResponseValue,
maxResponseValue, minFittedPredictorValues, maxFittedPredictorValues,
categoricalPredictors, categoricalPredictorLevels, warnedAboutMissingLevels,
allValuesForPredictorAreNA, allValuesAreNA, predictorValues)
{
# Determine which records have NA for one or more predictors.
anyAreNA <- logical(length(predictorValues[[1]])) # For each
record, TRUE if any predictor is NA
allAreNA <- logical(length(predictorValues[[1]])) | TRUE # For each
record, TRUE if all predictors are NA
for (predictor in allPredictors)
{
# For predictors that are categorical variables (i.e. factors), we
# cannot do predictions using values that were not considered by the
# model fitting process. If there are any of these, treat them as NA
# and warn the user.
#
# TODO: rpart and cforest models may be able to work around this with
# "surrogate splits". Figure out how that works and modify this if
# needed.
if (predictor %in% categoricalPredictors)
{
isMissingLevel <- !(predictorValues[[predictor]] %in%
categoricalPredictorLevels[[predictor]])
predictorValues[[predictor]][isMissingLevel] <- NA
if (any(isMissingLevel) && !warnedAboutMissingLevels[[predictor]])
{
warning(paste("The categorical variable named", predictor,
"contains values that were not present in the data used to fit the model. No
predictions can be made for these values."), call.=FALSE)
warnedAboutMissingLevels[[predictor]] <- TRUE
}
}
# For predictors that are continuous variables, if the caller wants us
# to ignore values that are outside the range of values used to fit
the
# model, set these to NA.
else if (ignoreOutOfRangeValues)
predictorValues[[predictor]][predictorValues[[predictor]] <
minFittedPredictorValues[[predictor]] | predictorValues[[predictor]] >
maxFittedPredictorValues[[predictor]]] <- NA
# Determine which records have NA for this predictor.
isNA <- is.na(predictorValues[[predictor]])
# Keep track of whether all of the values for this predictor are NA
for
# the entire prediction, which may span multiple calls to us.
allValuesForPredictorAreNA[[predictor]] <-
allValuesForPredictorAreNA[[predictor]] && all(isNA)
# For each record, we need to keep track of whether any or all of the
# predictors are NA. Incorporate this predictor into that calculation.
anyAreNA <- anyAreNA | isNA
allAreNA <- allAreNA & isNA
}
# Do the prediction. The prediction code varies according to the kind of
# model that was fitted. First handle rpart models.
predictedError <- NULL
if (!is.null(rPackage) && rPackage == "rpart")
{
# Do the prediction. For "class" models (i.e. classification trees),
get
# the result as integers, applying the cutoff to binary models. Note
# that rpart can perform predictions on rows where some of the
# predictors are NA, so long as not all of them are NA.
if (model$method == "class")
if (minResponseValue == 0 && maxResponseValue == 1)
{
predictedResponse <-
as.vector(suppressWarnings(predict(model, newdata=predictorValues,
type="prob")[,2]))
if (!is.null(cutoff))
predictedResponse <- as.integer(predictedResponse >=
cutoff)
}
else
predictedResponse <-
as.integer(as.vector(suppressWarnings(predict(model, newdata=predictorValues,
type="class"))))
else
predictedResponse <- as.vector(suppressWarnings(predict(model,
newdata=predictorValues, type="vector")))
# The rpart documentation implies that rpart can do predictions if
# *some* of the predictors are NA but not if *all* of the predictors
are
# NA. We therefore expect that cells that have NA for all predictors
# would result in a predicted value of NA. But rpart does not do that.
#
# So: Set all cells having NA for all predictors to NA.
predictedResponse[allAreNA] <- NA
}
# Handle random forest models fitted with randomForest or party.
else if (!is.null(rPackage) && rPackage %in% c("randomForest", "party"))
{
# For binary classification forests, get the predicted probability of
# the second class and apply the cutoff.
if (rPackage == "randomForest" && model$type == "classification" &&
minResponseValue == 0 && maxResponseValue == 1)
{
predictedResponse <- as.vector(suppressWarnings(predict(model,
newdata=predictorValues, type="prob")[,2]))
if (!is.null(cutoff))
predictedResponse <- as.integer(predictedResponse >= cutoff)
}
else if (rPackage == "party" &&
(model@responses@is_nominal[1]
||
model@responses@is_ordinal[1])
&& minResponseValue == 0 && maxResponseValue == 1)
{
predictedResponse <- sapply(suppressWarnings(predict(model,
newdata=predictorValues, type="prob")), "[", i = 2)
if (!is.null(cutoff))
predictedResponse <- as.integer(predictedResponse >= cutoff)
}
# For other models (regression forests or classification forests with
# more than two classes), get the predicted response. For
classification
# forests, coerce it to an integer.
else
{
predictedResponse <- suppressWarnings(predict(model,
newdata=predictorValues, type="response"))
if (is.factor(predictedResponse))
predictedResponse <- as.integer(as.vector(predictedResponse))
else
predictedResponse <- as.vector(predictedResponse)
}
}
# Handle nlme (not currently supported).
else if (!is.null(rPackage) && rPackage == "nlme")
{
stop("Prediction is not implemented for nlme models yet. Please
contact the MGET development team for assistance.", call.=FALSE)
}
# Handle GLM and GAM.
else
{
# The predict.gam function from the R gam library will fail if we pass
# in NA for covariates that were fitted with the loess smoother (lo).
To
# work around this, set the predictors to dummy values if they are NA.
# For the dummy values, use the minimum values of the predictors used
to
# fit the model. (If we pass in values outside the range of values
used
# to fit the model, loess will report a warning on a per-record basis,
# drowning the user in warnings.) These dummy predictor values will
# result in bogus predictions of the response variable. Later, we will
# convert these back to NA.
for (predictor in allPredictors)
predictorValues[[predictor]][anyAreNA] <-
minFittedPredictorValues[[predictor]]
# Do the prediction.
predictions <- suppressWarnings(predict(model,
newdata=predictorValues, type="response", se.fit=returnPredictedErrors))
# If the caller requested predicted errors, extract both the predicted
# response and the errors. Otherwise just extract the response.
if (returnPredictedErrors)
{
predictedResponse <- as.vector(predictions$fit)
predictedError <- as.vector(predictions$se.fit)
}
else
predictedResponse <- as.vector(predictions)
# Set rows that had NA for one or more predictor variables back to NA.
# This will overwrite bogus predictions that resulted from dummy
values.
predictedResponse[anyAreNA] <- NA
if (returnPredictedErrors)
predictedError[anyAreNA] <- NA
# If the caller provided a cutoff, classify the response into
integers.
if (!is.null(cutoff))
predictedResponse <- as.integer(predictedResponse >= cutoff)
}
# Keep track of whether the predicted values were NA for the entire
# prediction, which may span multiple calls to us.
allValuesAreNA <- allValuesAreNA && all(is.na(predictedResponse))
# Return the predicted response and error.
return(list(predictedResponse, predictedError, warnedAboutMissingLevels,
allValuesForPredictorAreNA, allValuesAreNA))
}
PrintPredictionWarnings <- function(rPackage, allPredictors,
allValuesForPredictorAreNA, allValuesAreNA)
{
# For non-rpart models, warn the user about any predictors were NA for all
# records or cells.
if (is.null(rPackage) || rPackage != "rpart")
{
naPredictors <- character()
for (predictor in allPredictors)
if (allValuesForPredictorAreNA[[predictor]])
naPredictors <- append(naPredictors, predictor,
length(naPredictors))
if (length(naPredictors) > 0)
if (length(naPredictors) == 1)
warning(paste("All of the values provided for the predictor
variable", naPredictors[1], " were NULL or NoData. A response can only be
predicted when all of the predictor variables have data. As a result, all of
the predicted values will be database NULL values (if you are predicting
values from a table) or NoData values (if you are predicting values from
rasters)."), call.=FALSE)
else
warning(paste("All of the values provided for the following
predictor variables were NULL or NoData: ", paste(naPredictors, collapse=",
"), ". A response can only be predicted when all of the predictor variables
have data. As a result, all of the predicted values will be database NULL
values (if you are predicting values from a table) or NoData values (if you
are predicting values from rasters).", sep=""), call.=FALSE)
# Warn the user if no predictions could be performed (even though the
# predictors each had data for at least one record or cell).
else if (allValuesAreNA)
warning("All of the predictions resulted in \"NA\" values and
will be stored as database NULL values (if you are predicting values from a
table) or NoData values (if you are predicting values from rasters). This
usually happens when predictor values are missing (they are NULL or NoData)
or not within the range used to fit the model. If this result is unexpected,
check the predictor values you provided as input and make sure they are
correct.", call.=FALSE)
}
# For rpart models, only warn the user if all predicted values were NA.
else if (allValuesAreNA)
warning("All of the predictions resulted in \"NA\" values and will be
stored as database NULL values (if you are predicting values from a table) or
NoData values (if you are predicting values from rasters). This usually
happens when predictor values are missing (they are NULL or NoData) or not
within the range used to fit the model. If this result is unexpected, check
the predictor values you provided as input and make sure they are correct.",
call.=FALSE)
}
PredictModelForArcGISRasters <- function(model, trainingData, rPackage,
rastersForPredictors, constantsForPredictors, outputResponseFile,
outputErrorFile=NULL, ignoreOutOfRangeValues=FALSE, cutoff=NULL)
{
library(stats)
# Initialize variables needed to do a prediction.
if (is.null(rastersForPredictors) || length(rastersForPredictors) <= 0)
stop("You must specify at least one predictor raster.", call.=FALSE)
variables <- PrepareForPrediction(model, trainingData, rPackage)
responseVariable <- variables[[1]]
allPredictors <- variables[[2]]
minResponseValue <- variables[[3]]
maxResponseValue <- variables[[4]]
isBinaryClassification <- variables[[5]]
isNonbinaryClassification <- variables[[6]]
minFittedPredictorValues <- variables[[7]]
maxFittedPredictorValues <- variables[[8]]
categoricalPredictors <- variables[[9]]
categoricalPredictorLevels <- variables[[10]]
warnedAboutMissingLevels <- variables[[11]]
allValuesForPredictorAreNA <- variables[[12]]
allValuesAreNA <- variables[[13]]
# Verify that the caller provided all of the necessary predictors.
for (predictor in allPredictors)
if (!(predictor %in% labels(rastersForPredictors)) &&
(!is.null(constantsForPredictors) && !(predictor %in%
labels(constantsForPredictors))))
stop(sprintf("The model included a predictor named \"%s\" but you
did not specify a raster or constant value for it. Please do that and try
again.", predictor), call.=FALSE)
# Load the rgdal library, which we use to read the predictor rasters.
library(rgdal)
# Obtain the characteristics of the predictor rasters. For safety, verify
# that they all have the same dimensions. The Python function that
generated
# them should guarantee this, but it is cheap to check them here.
rasterInfo <- open.SpatialGDAL(rastersForPredictors[1], silent=TRUE)
tryCatch(
{
if (length(rastersForPredictors) > 1)
for (i in 2:length(rastersForPredictors))
{
rasterInfo2 <- open.SpatialGDAL(rastersForPredictors[i],
silent=TRUE)
tryCatch(
{
if
(any(rasterInfo2@
!=
rasterInfo@))
stop("All of the predictor rasters must have the same
number of rows and columns.", call.=FALSE)
}, finally=close(rasterInfo2))
}
rasterCols <-
as.integer(round(rasterInfo@[1]))
rasterRows <-
as.integer(round(rasterInfo@[2]))
rasterXLLCorner <-
rasterInfo@bbox["x","min"]
rasterYLLCorner <-
rasterInfo@bbox["y","min"]
rasterCellSize <-
rasterInfo@grid@cellsize[1]
}, finally=close(rasterInfo))
# Determine the data type and NoData value of the output rasters.
# If a cutoff is provided (indicating that the output should be a
# binary classification) or the model is a classification with
# more than two classes, use the most compact integer data type
# and corresponding NoData value. Otherwise (for regression models
# or classification models with two classes that but no cutoff was
# provided), use 32-bit floating point as the data type and
# -3.4028235e+038 as the NoData value (the 32-bit IEEE-754
# floating-point negative number that is farthest from 0;
# traditionally used by ArcGIS for NoData).
if (!is.null(cutoff) || isNonbinaryClassification)
{
if (minResponseValue > -128 && maxResponseValue <= 127)
{
dataType <- "signedint"
nbytes <- 1
noDataValue <- as.integer(-128)
}
else if (minResponseValue >= -128 && maxResponseValue < 127)
{
dataType <- "signedint"
nbytes <- 1
noDataValue <- as.integer(127)
}
else if (minResponseValue > 0 && maxResponseValue <= 255)
{
dataType <- "unsignedint"
nbytes <- 1
noDataValue <- as.integer(0)
}
else if (minResponseValue >= 0 && maxResponseValue < 255)
{
dataType <- "unsignedint"
nbytes <- 1
noDataValue <- as.integer(255)
}
else if (minResponseValue > -32768 && maxResponseValue <= 32767)
{
dataType <- "signedint"
nbytes <- 2
noDataValue <- as.integer(-32768)
}
else if (minResponseValue >= -32768 && maxResponseValue < 32767)
{
dataType <- "signedint"
nbytes <- 2
noDataValue <- as.integer(32767)
}
else if (minResponseValue > 0 && maxResponseValue <= 65535)
{
dataType <- "unsignedint"
nbytes <- 2
noDataValue <- as.integer(0)
}
else if (minResponseValue >= 0 && maxResponseValue < 65535)
{
dataType <- "unsignedint"
nbytes <- 2
noDataValue <- as.integer(65535)
}
else if (minResponseValue > -2147483648 && maxResponseValue <=
2147483647)
{
dataType <- "signedint"
nbytes <- 4
noDataValue <- as.integer(-2147483648)
}
else if (minResponseValue >= -2147483648 && maxResponseValue <
2147483647)
{
dataType <- "signedint"
nbytes <- 4
noDataValue <- as.integer(2147483647)
}
else if (minResponseValue > 0)
{
dataType <- "unsignedint"
nbytes <- 4
noDataValue <- as.integer(0)
}
else
{
dataType <- "unsignedint"
nbytes <- 4
noDataValue <- as.integer(4294967296)
}
}
else
{
if (isBinaryClassification && is.null(cutoff))
warning("This appears to be a binary classification model but no
cutoff value was provided. The output will be a continuous floating-point
value ranging from 0 to 1. To obtain a binary integer value (0 or 1), please
provide a cutoff and try again.", call.=FALSE)
dataType <- "float"
nbytes <- 4
noDataValue <- -3.4028235e+038
}
# Create the .hdr files that describe the binaries we will write.
header <- file(paste(outputResponseFile, ".hdr", sep=""), "wb")
tryCatch(
{
cat("ncols ", rasterCols, "\r\n", sep="", file=header)
cat("nrows ", rasterRows, "\r\n", sep="", file=header)
cat("cellsize ", formatC(rasterCellSize, format="g", digits=15,
width=-1), "\r\n", sep="", file=header)
cat("xllcorner ", formatC(rasterXLLCorner, format="g", digits=15,
width=-1), "\r\n", sep="", file=header)
cat("yllcorner ", formatC(rasterYLLCorner, format="g", digits=15,
width=-1), "\r\n", sep="", file=header)
if (dataType == "float")
cat("nodata_value ", formatC(noDataValue, format="g", digits=8,
width=-1), "\r\n", sep="", file=header)
else
cat("nodata_value ", noDataValue, "\r\n", sep="", file=header)
cat("nbits ", as.integer(nbytes * 8), "\r\n", sep="", file=header)
cat("pixeltype ", dataType, "\r\n", sep="", file=header)
if (.Platform$endian == "little")
cat("byteorder lsbfirst\r\n", file=header)
else
cat("byteorder msbfirst\r\n", file=header)
}, finally=close(header))
if (!is.null(outputErrorFile))
{
header <- file(paste(outputErrorFile, ".hdr", sep=""), "wb")
tryCatch(
{
cat("ncols ", rasterCols, "\r\n", sep="", file=header)
cat("nrows ", rasterRows, "\r\n", sep="", file=header)
cat("cellsize ", formatC(rasterCellSize, format="g", digits=15,
width=-1), "\r\n", sep="", file=header)
cat("xllcorner ", formatC(rasterXLLCorner, format="g", digits=15,
width=-1), "\r\n", sep="", file=header)
cat("yllcorner ", formatC(rasterYLLCorner, format="g", digits=15,
width=-1), "\r\n", sep="", file=header)
cat("nodata_value ", formatC(noDataValue, format="g", digits=8,
width=-1), "\r\n", sep="", file=header)
cat("nbits 32\r\n", file=header)
cat("pixeltype float\r\n", file=header)
if (.Platform$endian == "little")
cat("byteorder lsbfirst\r\n", file=header)
else
cat("byteorder msbfirst\r\n", file=header)
}, finally=close(header))
}
# Open the output binaries.
responseFile <- file(paste(outputResponseFile, ".bil", sep=""), "wb")
tryCatch(
{
if (!is.null(outputErrorFile))
errorFile <- file(paste(outputErrorFile, ".bil", sep=""), "wb")
tryCatch(
{
# Open each predictor raster as a GDAL dataset.
datasets <- character()
tryCatch(
{
datasets <- list()
for (predictor in labels(rastersForPredictors))
datasets[[predictor]] <-
GDAL.open(rastersForPredictors[[predictor]])
# For each row of the predictor rasters, do a prediction and
# write the results to the output files. We do this one row
at a
# time to minimize the amount of memory required to run the
# tool. This is slower than reading everything into memory and
# doing the prediction all in one shot (1.5x slower in my
# limited tests) but it is well worth it, since doing it in
one
# shot can require too much memory.
timeCancelLastChecked <- CheckForCancel()
for (row in 1:rasterRows)
{
timeCancelLastChecked <-
CheckForCancel(timeCancelLastChecked)
# If we have processed 1% of the rows, estimate
# the time required to process the rest. If it
# exceeds 1 minute, report a warning message to
# alert the user that he will have to wait a
# while.
if (row == 2)
started = Sys.time()
else if (row == round(rasterRows / 100) + 1 &&
as.numeric((Sys.time() - started) * 99, units="mins") > 1)
warning(sprintf("This prediction is estimated to
require %0.1f minutes to complete.", as.numeric((Sys.time() - started),
units="mins") * 99), call.=FALSE)
# Read this row from each predictor raster.
predictorValues <- list()
for (predictor in labels(rastersForPredictors))
predictorValues[[predictor]] <-
getRasterData(datasets[[predictor]], offset=c(row-1,0),
region.dim=c(1,rasterCols), as.is=TRUE)
# For predictors that have constant values, allocate
vectors
# that repeat those values the appropriate number of
times.
if (!is.null(constantsForPredictors))
for (predictor in labels(constantsForPredictors))
predictorValues[[predictor]] <-
rep(constantsForPredictors[[predictor]],
length(predictorValues[[labels(rastersForPredictors)[1]]]))
# Do the prediction.
result <- PredictModel(model, rPackage, cutoff,
ignoreOutOfRangeValues, !is.null(outputErrorFile), allPredictors,
responseVariable, minResponseValue, maxResponseValue,
minFittedPredictorValues, maxFittedPredictorValues, categoricalPredictors,
categoricalPredictorLevels, warnedAboutMissingLevels,
allValuesForPredictorAreNA, allValuesAreNA, predictorValues)
predictedResponse <- result[[1]]
predictedError <- result[[2]]
warnedAboutMissingLevels <- result[[3]]
allValuesForPredictorAreNA <- result[[4]]
allValuesAreNA <- result[[5]]
# Convert NAs to NoData values.
predictedResponse[is.na(predictedResponse)] <- noDataValue
if (!is.null(outputErrorFile))
predictedError[is.na(predictedError)] <- noDataValue
# Write the row to the binary files.
writeBin(predictedResponse, responseFile, size=nbytes)
if (!is.null(outputErrorFile))
writeBin(predictedError, errorFile, size=nbytes)
}
}, finally=for (ds in datasets) GDAL.close(ds))
}, finally=if (!is.null(outputErrorFile)) close(errorFile))
}, finally=close(responseFile))
# Warn the user about any predictors were NoData for all cells, etc.
PrintPredictionWarnings(rPackage, allPredictors,
allValuesForPredictorAreNA, allValuesAreNA)
}
PredictModelForDataframe <- function(model, rPackage, trainingData,
newData=NULL, ignoreOutOfRangeValues=FALSE, cutoff=NULL, outputPlotFile=NULL,
measure1="tpr", measure2="fpr", colorize=TRUE, outputSummaryFile=NULL,
res=1000.0, width=3000.0, height=3000.0, pointSize=10.0, bg="white")
{
library(stats)
# Initialize variables needed to do a prediction.
variables <- PrepareForPrediction(model, trainingData, rPackage)
responseVariable <- variables[[1]]
allPredictors <- variables[[2]]
minResponseValue <- variables[[3]]
maxResponseValue <- variables[[4]]
isBinaryClassification <- variables[[5]]
isNonbinaryClassification <- variables[[6]]
minFittedPredictorValues <- variables[[7]]
maxFittedPredictorValues <- variables[[8]]
categoricalPredictors <- variables[[9]]
categoricalPredictorLevels <- variables[[10]]
warnedAboutMissingLevels <- variables[[11]]
allValuesForPredictorAreNA <- variables[[12]]
allValuesAreNA <- variables[[13]]
# Verify that the caller provided all of the necessary predictors.
useTrainingData <- is.null(newData)
if (useTrainingData)
newData <- trainingData
else
for (predictor in allPredictors)
if (!(predictor %in% labels(newData)[[2]]))
stop(sprintf("The model included a predictor named \"%s\" but
newData does not contain a column for it. Please add that column and try
again.", predictor), call.=FALSE)
# Do the prediction. For binary models, force the probability to be
returned.
predictedResponse <- PredictModel(model, rPackage, NULL,
ignoreOutOfRangeValues, FALSE, allPredictors, responseVariable,
minResponseValue, maxResponseValue, minFittedPredictorValues,
maxFittedPredictorValues, categoricalPredictors, categoricalPredictorLevels,
warnedAboutMissingLevels, allValuesForPredictorAreNA, allValuesAreNA,
newData)[[1]]
# If the data include the response variable, compute summary statistics.
if (responseVariable %in% labels(newData)[[2]])
{
if (useTrainingData)
modelSummary <- paste("MODEL PERFORMANCE SUMMARY:\n",
"==========================\n",
"\n",
"Statistics calculated from the training
data.\n",
sep="")
else
modelSummary <- paste("MODEL PERFORMANCE SUMMARY:\n",
"==========================\n",
sep="")
# If it is a binary classification model and the caller did not
provide
# a cutoff, calculate one now using the ROCR package.
cutoffAutomaticallySelected <- FALSE
if (isBinaryClassification && is.null(cutoff))
{
library(ROCR)
pred <- prediction(predictedResponse, newData[[responseVariable]])
perf <- performance(pred, 'sens', 'spec')
bestCutoff <- which.max(mapply(sum,
unlist(),
unlist(),
-1))
cutoff <-
pred@cutoffs[[1]][bestCutoff]
cutoffAutomaticallySelected <- TRUE
}
response <- data.frame(actual=newData[[responseVariable]],
predicted=predictedResponse)
response <- na.omit(response)
# For binary classification models, compute summary statistics using
# ROCR.
if (isBinaryClassification)
{
library(ROCR)
pred <- prediction(response$predicted, response$actual)
# Do the following calculations in a try, so that the whole
function
# does not fail for certain input data. The only case I know of is
# the prbe calculation can fail with:
#
# Error in
.performance.precision.recall.break.even.point(predictions = c(1L, : Not
enough distinct predictions to compute precision/recall intersections.
auc <- NA
mxe <- NA
prbe <- NA
rmse <- NA
try({ auc <- performance(pred, "auc")@y.values[[1]] },
silent=TRUE)
try({ mxe <- performance(pred, "mxe")@y.values[[1]] },
silent=TRUE)
try({ prbe <- unlist(performance(pred, "prbe")@y.values) },
silent=TRUE)
try({ rmse <- performance(pred, "rmse")@y.values[[1]] },
silent=TRUE)
if (length(prbe) > 0)
prbe <- prbe[length(prbe)]
else
prbe <- NA
modelSummary <- paste(modelSummary,
"\n",
sprintf("Area under the ROC curve (auc)
= %.3f\n", auc),
sprintf("Mean cross-entropy (mxe)
= %.3f\n", mxe),
sprintf("Precision-recall break-even point
(prbe) = %.3f\n", prbe),
sprintf("Root-mean square error (rmse)
= %.3f\n", rmse),
sep="")
tn = length(which((response$predicted < cutoff) &
(response$actual == 0)))
fn = length(which((response$predicted < cutoff) &
(response$actual != 0)))
tp = length(which((response$predicted >= cutoff) &
(response$actual == 1)))
fp = length(which((response$predicted >= cutoff) &
(response$actual != 1)))
if (cutoffAutomaticallySelected)
modelSummary <- paste(modelSummary,
"\n",
sprintf("Cutoff selected by maximizing
the Youden index = %.3f\n", cutoff),
sep="")
else
modelSummary <- paste(modelSummary,
"\n",
sprintf("User-specified cutoff =
%.3f\n", cutoff),
sep="")
modelSummary <- paste(modelSummary,
"\n",
"Confusion matrix for that cutoff:\n",
"\n",
" Actual 1 Actual 0
Total\n",
sprintf("Predicted 1 %9i %9i %9i\n", tp,
fp, tp+fp),
sprintf("Predicted 0 %9i %9i %9i\n", fn,
tn, tn+fn),
sprintf(" Total %9i %9i %9i\n", tp+fn,
fp+tn, tp+fn+fp+tn),
sep="")
tn <- as.double(tn)
fn <- as.double(fn)
tp <- as.double(tp)
fp <- as.double(fp)
acc <- (tp+tn)/(tp+fp+tn+fn)
modelSummary <- paste(modelSummary,
"\n",
"Model performance statistics for that
cutoff:\n",
"\n",
sprintf("Accuracy (acc)
= %.3f\n", acc),
sprintf("Error rate (err)
= %.3f\n", (fp+fn)/(tp+fp+tn+fn)),
sprintf("Rate of positive predictions (rpp)
= %.3f\n", (tp+fp)/(tp+fp+tn+fn)),
sprintf("Rate of negative predictions (rnp)
= %.3f\n", (tn+fn)/(tp+fp+tn+fn)),
"\n",
sprintf("True positive rate (tpr, or
sensitivity) = %.3f\n", tp/(tp+fn)),
sprintf("False positive rate (fpr, or
fallout) = %.3f\n", fp/(fp+tn)),
sprintf("True negative rate (tnr, or
specificity) = %.3f\n", tn/(fp+tn)),
sprintf("False negative rate (fnr, or miss)
= %.3f\n", fn/(tp+fn)),
"\n",
sprintf("Positive prediction value (ppv, or
precision) = %.3f\n", tp/(tp+fp)),
sprintf("Negative prediction value (npv)
= %.3f\n", tn/(tn+fn)),
sprintf("Prediction-conditioned fallout
(pcfall) = %.3f\n", fp/(tp+fp)),
sprintf("Prediction-conditioned miss
(pcmiss) = %.3f\n", fn/(tn+fn)),
"\n",
sprintf("Matthews correlation coefficient
(mcc) = %.3f\n", (tp*tn -
fp*fn)/sqrt((tp+fn)*(fp+tn)*(tp+fp)*(fn+tn))),
sprintf("Odds ratio (odds)
= %.3f\n", (tp*tn)/(fn*fp)),
sprintf("SAR
= %.3f\n", (acc + auc + rmse)/3),
sep="")
# Create the performance plot.
if (!is.null(outputPlotFile))
{
perf <- performance(pred, measure1, measure2)
if (tolower(substring(outputPlotFile,
nchar(outputPlotFile)-3, nchar(outputPlotFile))) == ".emf")
win.metafile(outputPlotFile, width=width, height=height,
pointsize=pointSize)
else
png(outputPlotFile, res=res, width=width, height=height,
pointsize=pointSize, bg=bg)
tryCatch({
if (measure1 == "tpr" && measure2 == "fpr") # Handle
ROC plots specially
{
plot(perf, colorize=colorize, lwd=5)
abline(0, 1, lty=2, lwd=0.5, col="grey25")
tpr = tp/(tp+fn)
fpr = fp/(fp+tn)
if (colorize)
points(x=fpr, y=tpr, cex=1.5, lwd=2)
else
points(x=fpr, y=tpr, pch=21, cex=1.5, lwd=2,
bg=bg)
text(x=fpr+0.03, y=tpr-0.03, labels=sprintf("Cutoff =
%.3f", cutoff), adj=c(0,1))
}
else
plot(perf, colorize=colorize && measure2 != "cutoff")
}, finally={ graphics.off() })
}
# Using the cutoff, classify the continuous (0..1) response to
# binary values (0 or 1), and calculate and report Cohen's kappa.
predictedResponse <- as.integer(predictedResponse >= cutoff)
response <- data.frame(actual=newData[[responseVariable]],
predicted=predictedResponse)
response <- na.omit(response)
modelSummary <- paste(modelSummary,
"\n",
sprintf("Cohen's kappa (K)
= %.3f\n", ckappa(response)$kappa),
sep="")
}
# For other classification models, compute summary statistics using
# caret.
else if (isNonbinaryClassification)
{
library(caret)
factorLevels <- unique(c(response$predicted, response$actual))
modelSummary <- paste(modelSummary,
"\n",
paste(capture.output(print(confusionMatrix(factor(response$predicted,
levels=factorLevels), factor(response$actual, levels=factorLevels),
dnn=c("Predicted", "Actual")))), sep="", collapse="\n"), "\n",
sep="")
}
# For regression models, compute % variance explained (a.k.a. Efron's
# pseudo R-squared).
else
{
varianceExplained <- 1 - sum((response$actual -
response$predicted)**2) / sum((response$actual - mean(response$actual))**2)
mse <- mean((response$actual - response$predicted)**2)
rmse <- sqrt(mse)
r <- cor(response$actual, response$predicted, method="pearson")
tau <- cor(response$actual, response$predicted, method="kendall")
rho <- cor(response$actual, response$predicted, method="spearman")
modelSummary <- paste(modelSummary,
"\n",
sprintf("Number of records = %i\n",
length(response$actual)),
"\n",
sprintf("Variance Explained =
%.1f%%\n", varianceExplained * 100),
sprintf("Mean Squared Error = %f\n",
mse),
sprintf("Root Mean Squared Error = %f\n",
rmse),
"\n",
sprintf("Pearson's r = %.3f\n", r),
sprintf("Kendall's tau = %.3f\n", tau),
sprintf("Spearman's rho = %.3f\n", rho),
sep="")
}
# Print the summary statistics.
message("")
writeLines(modelSummary)
message("")
# If the caller provided a summary file path, write the summary
# statistics to the file.
if (!is.null(outputSummaryFile))
{
f <- file(outputSummaryFile, "wt")
tryCatch(writeLines(modelSummary, f), finally=close(f))
}
}
return(list(predictedResponse, cutoff))
}
R interpreter: The last statement returned R closure function at 0x1ED1AA70
R interpreter: Evaluating statements: exists("rPackage")
R interpreter: The last statement returned False
R interpreter: Evaluating statements: exists("rPackage")
R interpreter: The last statement returned False
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
exists(*args=('bll_blacktip',))...
ArcGIS Geoprocessor object 0x1EAE22A8: exists returned 1
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
describe(*args=('bll_blacktip',))...
ArcGIS Geoprocessor object 0x1EAE22A8: describe returned <geoprocessing
describe data object object at 0x1ED57608>
ArcGIS Describe object 0x1ED57608: Get Fields returned <geoprocessing list
object object at 0x1ED570F8>
ArcGIS Describe object 0x1ED57608: Get Fields returned <geoprocessing list
object object at 0x1ED570F8>
ArcGIS Describe object 0x1ED57608: Get Fields returned <geoprocessing list
object object at 0x1ED570F8>
ArcGIS Fields object 0x1ED570F8: Invoking next(*args=())...
ArcGIS Fields object 0x1ED570F8: next returned <geoprocessing describe field
object object at 0x1ED572D8>
ArcGIS Describe object 0x1ED57608: Get DataType returned 'FeatureLayer'
ArcGIS Describe object 0x1ED57608: Get CatalogPath returned
'Z:\\sharks\\blacktip_bll\\blacktip_bll.gdb\\bll_blacktip'
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
describe(*args=('Z:\\sharks\\blacktip_bll\\blacktip_bll.gdb\\bll_blacktip',))...
ArcGIS Geoprocessor object 0x1EAE22A8: describe returned <geoprocessing
describe data object object at 0x1ED57E18>
ArcGIS Describe object 0x1ED57E18: Get DataType returned 'FeatureClass'
ArcGIS Describe object 0x1ED57608: Get CatalogPath returned
'Z:\\sharks\\blacktip_bll\\blacktip_bll.gdb\\bll_blacktip'
ArcGIS Describe object 0x1ED57608: Get OIDFieldName returned 'OBJECTID'
ArcGIS Describe object 0x1ED57608: Get OIDFieldName returned 'OBJECTID'
ArcGIS Describe object 0x1ED57608: Get OIDFieldName returned 'OBJECTID'
ArcGIS Describe object 0x1ED57608: Get OIDFieldName returned 'OBJECTID'
ArcGIS Describe object 0x1ED57608: Get ShapeFieldName returned 'Shape'
ArcGIS Describe object 0x1ED57608: Get ShapeFieldName returned 'Shape'
ArcGIS Describe object 0x1ED57608: Get ShapeFieldName returned 'Shape'
ArcGIS Describe object 0x1ED57608: Get ShapeFieldName returned 'Shape'
ArcGIS Describe object 0x1ED57608: Get ShapeType returned 'Point'
ArcGIS Describe object 0x1ED57608: Get HasZ returned False
ArcGIS Describe object 0x1ED57608: Get HasZ returned False
ArcGIS Describe object 0x1ED57608: Get SpatialReference returned
<geoprocessing spatial reference object object at 0x1ED57E18>
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
CreateSpatialReference_management(*args=(<geoprocessing spatial reference
object object at 0x1ED57E18>,))...
Executing: CreateSpatialReference
"PROJCS['North_America_Lambert_Conformal_Conic',GEOGCS['GCS_North_American_1983',DATUM['D_North_American_1983',SPHEROID['GRS_1980',6378137.0,298.257222101]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],PROJECTION['Lambert_Conformal_Conic'],PARAMETER['False_Easting',0.0],PARAMETER['False_Northing',0.0],PARAMETER['Central_Meridian',-79.5],PARAMETER['Standard_Parallel_1',27.5],PARAMETER['Standard_Parallel_2',41.5],PARAMETER['Latitude_Of_Origin',20.0],UNIT['Meter',1.0]];-37658500
-26849000 9999.99370426336;-100000 10000;-100000
10000;9.9999998928979E-04;0.001;0.001;IsHighPrecision" # # # # # 0
Start Time: Fri Sep 21 17:17:49 2012
Spatial Reference = North_America_Lambert_Conformal_Conic
XY Domain (XMin,YMin XMax,YMax) = -37658500,-26849000
900682834044,900693643544
Z Domain (Min,Max) = -100000,900719825474.099
M Domain (Min,Max) = -100000,900719825474.099
Succeeded at Fri Sep 21 17:17:49 2012 (Elapsed Time: 0.00 seconds)
ArcGIS Geoprocessor object 0x1EAE22A8: CreateSpatialReference_management
returned
"PROJCS['North_America_Lambert_Conformal_Conic',GEOGCS['GCS_North_American_1983',DATUM['D_North_American_1983',SPHEROID['GRS_1980',6378137.0,298.257222101]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],PROJECTION['Lambert_Conformal_Conic'],PARAMETER['False_Easting',0.0],PARAMETER['False_Northing',0.0],PARAMETER['Central_Meridian',-79.5],PARAMETER['Standard_Parallel_1',27.5],PARAMETER['Standard_Parallel_2',41.5],PARAMETER['Latitude_Of_Origin',20.0],UNIT['Meter',1.0]];-37658500
-26849000 9999.99370426336;-100000 10000;-100000
10000;9.9999998928979E-04;0.001;0.001;IsHighPrecision"
ArcGIS Describe object 0x1ED57608: Get Fields returned <geoprocessing list
object object at 0x1ED57548>
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57950>
ArcGIS Field object 0x1ED57950: Get Type returned 'OID'
ArcGIS Field object 0x1ED57950: Get IsNullable returned 'FALSE'
ArcGIS Field object 0x1ED57950: Get Name returned 'OBJECTID'
ArcGIS Field object 0x1ED57950: Get Length returned 4
ArcGIS Field object 0x1ED57950: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57848>
ArcGIS Field object 0x1ED57848: Get Type returned 'Geometry'
ArcGIS Field object 0x1ED57848: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57848: Get Name returned 'Shape'
ArcGIS Field object 0x1ED57848: Get Name returned 'Shape'
ArcGIS Field object 0x1ED57848: Get Name returned 'Shape'
ArcGIS Field object 0x1ED57848: Get Length returned 0
ArcGIS Field object 0x1ED57848: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57950>
ArcGIS Field object 0x1ED57950: Get Type returned 'Double'
ArcGIS Field object 0x1ED57950: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57950: Get Name returned 'DD_LAT'
ArcGIS Field object 0x1ED57950: Get Name returned 'DD_LAT'
ArcGIS Field object 0x1ED57950: Get Name returned 'DD_LAT'
ArcGIS Field object 0x1ED57950: Get Length returned 8
ArcGIS Field object 0x1ED57950: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED579C8>
ArcGIS Field object 0x1ED579C8: Get Type returned 'Double'
ArcGIS Field object 0x1ED579C8: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED579C8: Get Name returned 'DD_LONG'
ArcGIS Field object 0x1ED579C8: Get Name returned 'DD_LONG'
ArcGIS Field object 0x1ED579C8: Get Name returned 'DD_LONG'
ArcGIS Field object 0x1ED579C8: Get Length returned 8
ArcGIS Field object 0x1ED579C8: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57B00>
ArcGIS Field object 0x1ED57B00: Get Type returned 'Double'
ArcGIS Field object 0x1ED57B00: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57B00: Get Name returned 'Bathy'
ArcGIS Field object 0x1ED57B00: Get Name returned 'Bathy'
ArcGIS Field object 0x1ED57B00: Get Name returned 'Bathy'
ArcGIS Field object 0x1ED57B00: Get Length returned 8
ArcGIS Field object 0x1ED57B00: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57488>
ArcGIS Field object 0x1ED57488: Get Type returned 'Date'
ArcGIS Field object 0x1ED57488: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57488: Get Name returned 'DATE'
ArcGIS Field object 0x1ED57488: Get Name returned 'DATE'
ArcGIS Field object 0x1ED57488: Get Name returned 'DATE'
ArcGIS Field object 0x1ED57488: Get Length returned 8
ArcGIS Field object 0x1ED57488: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED572D8>
ArcGIS Field object 0x1ED572D8: Get Type returned 'Double'
ArcGIS Field object 0x1ED572D8: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED572D8: Get Name returned 'SST_8day'
ArcGIS Field object 0x1ED572D8: Get Name returned 'SST_8day'
ArcGIS Field object 0x1ED572D8: Get Name returned 'SST_8day'
ArcGIS Field object 0x1ED572D8: Get Length returned 8
ArcGIS Field object 0x1ED572D8: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57560>
ArcGIS Field object 0x1ED57560: Get Type returned 'Double'
ArcGIS Field object 0x1ED57560: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57560: Get Name returned 'Chla_8day_SW'
ArcGIS Field object 0x1ED57560: Get Name returned 'Chla_8day_SW'
ArcGIS Field object 0x1ED57560: Get Name returned 'Chla_8day_SW'
ArcGIS Field object 0x1ED57560: Get Length returned 8
ArcGIS Field object 0x1ED57560: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57320>
ArcGIS Field object 0x1ED57320: Get Type returned 'Double'
ArcGIS Field object 0x1ED57320: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57320: Get Name returned 'Slope'
ArcGIS Field object 0x1ED57320: Get Name returned 'Slope'
ArcGIS Field object 0x1ED57320: Get Name returned 'Slope'
ArcGIS Field object 0x1ED57320: Get Length returned 8
ArcGIS Field object 0x1ED57320: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57F38>
ArcGIS Field object 0x1ED57F38: Get Type returned 'Double'
ArcGIS Field object 0x1ED57F38: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57F38: Get Name returned 'Dist2Shr'
ArcGIS Field object 0x1ED57F38: Get Name returned 'Dist2Shr'
ArcGIS Field object 0x1ED57F38: Get Name returned 'Dist2Shr'
ArcGIS Field object 0x1ED57F38: Get Length returned 8
ArcGIS Field object 0x1ED57F38: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED578C0>
ArcGIS Field object 0x1ED578C0: Get Type returned 'Double'
ArcGIS Field object 0x1ED578C0: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED578C0: Get Name returned 'Bathy2'
ArcGIS Field object 0x1ED578C0: Get Name returned 'Bathy2'
ArcGIS Field object 0x1ED578C0: Get Name returned 'Bathy2'
ArcGIS Field object 0x1ED578C0: Get Length returned 8
ArcGIS Field object 0x1ED578C0: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED570F8>
ArcGIS Field object 0x1ED570F8: Get Type returned 'SmallInteger'
ArcGIS Field object 0x1ED570F8: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED570F8: Get Name returned 'month_1'
ArcGIS Field object 0x1ED570F8: Get Name returned 'month_1'
ArcGIS Field object 0x1ED570F8: Get Name returned 'month_1'
ArcGIS Field object 0x1ED570F8: Get Length returned 2
ArcGIS Field object 0x1ED570F8: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED572F0>
ArcGIS Field object 0x1ED572F0: Get Type returned 'SmallInteger'
ArcGIS Field object 0x1ED572F0: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED572F0: Get Name returned 'Blacktip_Pres'
ArcGIS Field object 0x1ED572F0: Get Name returned 'Blacktip_Pres'
ArcGIS Field object 0x1ED572F0: Get Name returned 'Blacktip_Pres'
ArcGIS Field object 0x1ED572F0: Get Length returned 2
ArcGIS Field object 0x1ED572F0: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57CB0>
ArcGIS Field object 0x1ED57CB0: Get Type returned 'SmallInteger'
ArcGIS Field object 0x1ED57CB0: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57CB0: Get Name returned 'isevaldata_1'
ArcGIS Field object 0x1ED57CB0: Get Name returned 'isevaldata_1'
ArcGIS Field object 0x1ED57CB0: Get Name returned 'isevaldata_1'
ArcGIS Field object 0x1ED57CB0: Get Length returned 2
ArcGIS Field object 0x1ED57CB0: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57E78>
ArcGIS Field object 0x1ED57E78: Get Type returned 'SmallInteger'
ArcGIS Field object 0x1ED57E78: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57E78: Get Name returned 'TestData'
ArcGIS Field object 0x1ED57E78: Get Name returned 'TestData'
ArcGIS Field object 0x1ED57E78: Get Name returned 'TestData'
ArcGIS Field object 0x1ED57E78: Get Length returned 2
ArcGIS Field object 0x1ED57E78: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57F80>
ArcGIS Field object 0x1ED57F80: Get Type returned 'Double'
ArcGIS Field object 0x1ED57F80: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57F80: Get Name returned 'bll_glm'
ArcGIS Field object 0x1ED57F80: Get Name returned 'bll_glm'
ArcGIS Field object 0x1ED57F80: Get Name returned 'bll_glm'
ArcGIS Field object 0x1ED57F80: Get Length returned 8
ArcGIS Field object 0x1ED57F80: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned None
ArcGISTable 0x1EC4FED0: Retrieved lazy properties of ArcGIS FeatureLayer
"bll_blacktip" of FeatureClass
"Z:\sharks\blacktip_bll\blacktip_bll.gdb\bll_blacktip":
ArcGISDataType=FeatureLayer, ArcGISPhysicalDataType=FeatureClass,
MaxStringLength=None, HasOID=True, OIDFieldName=u'OBJECTID',
GeometryType=u'Point', GeometryFieldName=u'Shape',
SpatialReference=u'PROJCS["North_America_Lambert_Conformal_Conic",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.017453292519943295]],PROJECTION["Lambert_Conformal_Conic"],PARAMETER["False_Easting",0.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",-79.5],PARAMETER["Standard_Parallel_1",27.5],PARAMETER["Standard_Parallel_2",41.5],PARAMETER["Latitude_Of_Origin",20.0],UNIT["Meter",1.0],PARAMETER["scale_factor",1.0]]'.
ArcGISTable 0x1EC4FED0: Field 0: Name=OBJECTID, DataType=oid, Length=4,
Precision=0, IsNullable=False, IsSettable=False.
ArcGISTable 0x1EC4FED0: Field 1: Name=Shape, DataType=geometry, Length=0,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 2: Name=DD_LAT, DataType=float64, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 3: Name=DD_LONG, DataType=float64, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 4: Name=Bathy, DataType=float64, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 5: Name=DATE, DataType=datetime, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 6: Name=SST_8day, DataType=float64, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 7: Name=Chla_8day_SW, DataType=float64,
Length=8, Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 8: Name=Slope, DataType=float64, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 9: Name=Dist2Shr, DataType=float64, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 10: Name=Bathy2, DataType=float64, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 11: Name=month_1, DataType=int16, Length=2,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 12: Name=Blacktip_Pres, DataType=int16,
Length=2, Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 13: Name=isevaldata_1, DataType=int16,
Length=2, Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 14: Name=TestData, DataType=int16, Length=2,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 15: Name=bll_glm, DataType=float64, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
R interpreter: Evaluating statements: PrepareForPrediction(model, model$data,
NULL)
R interpreter: The last statement returned [u'Blacktip_Pres',
[u'Chla_8day_SW', u'Slope', u'Dist2Shr', u'Bathy2', u'SST_8day', u'Month_1'],
0, 1, True, False, {'Bathy2': -677.0, 'Chla_8day_SW': 0.11578000336885452,
'Dist2Shr': 0.0, 'Month_1': 1, 'SST_8day': 17.475000381469727, 'Slope': 0.0},
{'Bathy2': -2.0, 'Chla_8day_SW': 16.118820190429688, 'Dist2Shr':
157651.484375, 'Month_1': 11, 'SST_8day': 30.075000762939453, 'Slope':
1.4985077381134033}, u'factor(Month_1)', {'factor(Month_1)': [u'1', u'2',
u'3', u'6', u'7', u'8', u'9', u'10', u'11']}, {'factor(Month_1)': False},
{'Bathy2': True, 'Chla_8day_SW': True, 'Dist2Shr': True, 'Month_1': True,
'SST_8day': True, 'Slope': True}, True]
R interpreter: Evaluating statements: if (exists("rastersForPredictors"))
rm("rastersForPredictors")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("model")) rm("model")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("trainingData"))
rm("trainingData")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("rPackage")) rm("rPackage")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("xVar")) rm("xVar")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("yVar")) rm("yVar")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("zVar")) rm("zVar")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("mVar")) rm("mVar")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("coordinateSystem"))
rm("coordinateSystem")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("f20985982305"))
rm("f20985982305", pos=globalenv())
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("data20985982305"))
rm("data20985982305", pos=globalenv())
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("fam20985982305"))
rm("fam20985982305", pos=globalenv())
R interpreter: The last statement returned None
ValueError: too many values to unpack
---------- BEGINNING OF DEBUGGING INFORMATION ----------
Traceback (most recent call last):
File
"C:\Python26\ArcGIS10.0\Lib\site-packages\GeoEco\ArcGISToolbox\Scripts\GLMPredictFromArcGISTable.py",
line 5, in <module>
ExecuteMethodFromCommandLineAsArcGISTool('GeoEco.Statistics.Modeling',
'GLM', 'PredictFromArcGISTable')
File "C:\Python26\ArcGIS10.0\lib\site-packages\GeoEco\ArcGISScripts.py",
line 212, in ExecuteMethodFromCommandLineAsArcGISTool
Logger.LogExceptionAsError()
File "<string>", line 1, in <module>
File
"C:\Python26\ArcGIS10.0\lib\site-packages\GeoEco\Statistics\Modeling.py",
line 126, in PredictFromArcGISTable
return _PredictFromArcGISTable('glm', _(u'Fit GLM'), inputModelFile,
inputTable, predictedValuesField, cutoff, where, ignoreOutOfRangeValues,
noDataValue, outputPlotFile, measure1, measure2, colorize, outputSummaryFile,
res, width, height, pointSize, bg, overwriteExisting)
File
"C:\Python26\ArcGIS10.0\lib\site-packages\GeoEco\Statistics\Modeling.py",
line 1332, in _PredictFromArcGISTable
responseVariable, allPredictors, minResponseValue, maxResponseValue,
minFittedPredictorValues, maxFittedPredictorValues, categoricalPredictors,
categoricalPredictorLevels, warnedAboutMissingLevels,
allValuesForPredictorAreNA, allValuesAreNA = r('PrepareForPrediction(model,
%s, %s)' % (trainingData, rPackage))
ValueError: too many values to unpack
Local variables for stack frame: File
"C:\Python26\ArcGIS10.0\lib\site-packages\GeoEco\Statistics\Modeling.py",
line 1480, in _PredictFromArcGISTable:
ArcGISTable = <class 'GeoEco.Datasets.ArcGIS.ArcGISTable'>
bg = u'white'
colorize = True
cutoff = None
dataFrameName = None
fields = []
fitToolName = u'Fit GLM'
gotPointFeatures = True
height = 3.0
ignoreOutOfRangeValues = True
inputModelFile = u'Z:\\sharks\\blacktip_bll\\glm\\blacktip_bll_glm4'
inputTable = u'bll_blacktip'
mColumnName = None
measure1 = u'tpr'
measure2 = u'fpr'
modelType = 'glm'
noDataValue = None
outputPlotFile = u'Z:\\sharks\\blacktip_bll\\glm\\blacktipglm4.emf'
outputSummaryFile = u'Z:\\sharks\\blacktip_bll\\glm\\blacktipglmSS4'
overwriteExisting = True
pointSize = 10.0
predictedValuesField = None
r = <GeoEco.R.RInterpreter object at 0x1EC4FEB0>
rPackage = 'NULL'
res = 1000.0
table = <GeoEco.Datasets.ArcGIS.ArcGISTable object at 0x1EC4FED0>
trainingData = 'model$data'
where = u'"TestData" = 1'
width = 3.0
xColumnName = None
yColumnName = None
zColumnName = None
Global variables for stack frame: File
"C:\Python26\ArcGIS10.0\lib\site-packages\GeoEco\Statistics\Modeling.py",
line 1480, in _PredictFromArcGISTable:
AddArgumentMetadata = <function AddArgumentMetadata at 0x1E17F770>
AddClassMetadata = <function AddClassMetadata at 0x1E17F670>
AddMethodMetadata = <function AddMethodMetadata at 0x1E17F730>
AddModuleMetadata = <function AddModuleMetadata at 0x1E17F630>
AddPropertyMetadata = <function AddPropertyMetadata at 0x1E17F6B0>
AddResultMetadata = <function AddResultMetadata at 0x1E17F7F0>
AnyObjectTypeMetadata = <class 'GeoEco.Types.AnyObjectTypeMetadata'>
ArcGIS91DatabaseConnection = <class
'GeoEco.DatabaseAccess.ArcGIS.ArcGIS91DatabaseConnection'>
ArcGIS91SelectCursor = <class
'GeoEco.DatabaseAccess.ArcGIS.ArcGIS91SelectCursor'>
ArcGISDependency = <class 'GeoEco.ArcGIS.ArcGISDependency'>
ArcGISExtensionDependency = <class
'GeoEco.ArcGIS.ArcGISExtensionDependency'>
ArcGISFeatureClassTypeMetadata = <class
'GeoEco.Types.ArcGISFeatureClassTypeMetadata'>
ArcGISFeatureLayerTypeMetadata = <class
'GeoEco.Types.ArcGISFeatureLayerTypeMetadata'>
ArcGISFieldTypeMetadata = <class 'GeoEco.Types.ArcGISFieldTypeMetadata'>
ArcGISGeoDatasetTypeMetadata = <class
'GeoEco.Types.ArcGISGeoDatasetTypeMetadata'>
ArcGISRaster = <class 'GeoEco.DataManagement.ArcGISRasters.ArcGISRaster'>
ArcGISRasterCatalogTypeMetadata = <class
'GeoEco.Types.ArcGISRasterCatalogTypeMetadata'>
ArcGISRasterLayerTypeMetadata = <class
'GeoEco.Types.ArcGISRasterLayerTypeMetadata'>
ArcGISRasterTypeMetadata = <class 'GeoEco.Types.ArcGISRasterTypeMetadata'>
ArcGISTableTypeMetadata = <class 'GeoEco.Types.ArcGISTableTypeMetadata'>
ArcGISTableViewTypeMetadata = <class
'GeoEco.Types.ArcGISTableViewTypeMetadata'>
ArcGISWorkspaceTypeMetadata = <class
'GeoEco.Types.ArcGISWorkspaceTypeMetadata'>
ArgumentMetadata = <class 'GeoEco.Metadata.ArgumentMetadata'>
BooleanTypeMetadata = <class 'GeoEco.Types.BooleanTypeMetadata'>
ClassInstanceTypeMetadata = <class 'GeoEco.Types.ClassInstanceTypeMetadata'>
ClassMetadata = <class 'GeoEco.Metadata.ClassMetadata'>
ClassOrClassInstanceTypeMetadata = <class
'GeoEco.Types.ClassOrClassInstanceTypeMetadata'>
ClassTypeMetadata = <class 'GeoEco.Types.ClassTypeMetadata'>
CoordinateSystemTypeMetadata = <class
'GeoEco.Types.CoordinateSystemTypeMetadata'>
CopyArgumentMetadata = <function CopyArgumentMetadata at 0x1E17F830>
CopyPropertyMetadata = <function CopyPropertyMetadata at 0x1E17F6F0>
CopyResultMetadata = <function CopyResultMetadata at 0x1E17F870>
DateTimeTypeMetadata = <class 'GeoEco.Types.DateTimeTypeMetadata'>
DictionaryTypeMetadata = <class 'GeoEco.Types.DictionaryTypeMetadata'>
DirectoryTypeMetadata = <class 'GeoEco.Types.DirectoryTypeMetadata'>
DynamicDocString = <class 'GeoEco.DynamicDocString.DynamicDocString'>
EnvelopeTypeMetadata = <class 'GeoEco.Types.EnvelopeTypeMetadata'>
File = <class 'GeoEco.DataManagement.Files.File'>
FileTypeMetadata = <class 'GeoEco.Types.FileTypeMetadata'>
FloatTypeMetadata = <class 'GeoEco.Types.FloatTypeMetadata'>
GAM = <class 'GeoEco.Statistics.Modeling.GAM'>
GLM = <class 'GeoEco.Statistics.Modeling.GLM'>
GeoprocessorManager = <class 'GeoEco.ArcGIS.GeoprocessorManager'>
IntegerTypeMetadata = <class 'GeoEco.Types.IntegerTypeMetadata'>
LinearMixedModel = <class 'GeoEco.Statistics.Modeling.LinearMixedModel'>
LinearUnitTypeMetadata = <class 'GeoEco.Types.LinearUnitTypeMetadata'>
ListTypeMetadata = <class 'GeoEco.Types.ListTypeMetadata'>
Logger = <class 'GeoEco.Logging.Logger'>
MapAlgebraExpressionTypeMetadata = <class
'GeoEco.Types.MapAlgebraExpressionTypeMetadata'>
Metadata = <class 'GeoEco.Metadata.Metadata'>
MethodMetadata = <class 'GeoEco.Metadata.MethodMetadata'>
ModelEvaluation = <class 'GeoEco.Statistics.Modeling.ModelEvaluation'>
ModuleMetadata = <class 'GeoEco.Metadata.ModuleMetadata'>
NoneTypeMetadata = <class 'GeoEco.Types.NoneTypeMetadata'>
NumPyArrayTypeMetadata = <class 'GeoEco.Types.NumPyArrayTypeMetadata'>
PointTypeMetadata = <class 'GeoEco.Types.PointTypeMetadata'>
PropertyMetadata = <class 'GeoEco.Metadata.PropertyMetadata'>
PythonModuleDependency = <class
'GeoEco.Dependencies.PythonModuleDependency'>
R = <class 'GeoEco.R.R'>
RDependency = <class 'GeoEco.R.RDependency'>
RPackageDependency = <class 'GeoEco.R.RPackageDependency'>
RandomForestModel = <class 'GeoEco.Statistics.Modeling.RandomForestModel'>
ResultMetadata = <class 'GeoEco.Metadata.ResultMetadata'>
SQLWhereClauseTypeMetadata = <class
'GeoEco.Types.SQLWhereClauseTypeMetadata'>
SequenceTypeMetadata = <class 'GeoEco.Types.SequenceTypeMetadata'>
ShapefileTypeMetadata = <class 'GeoEco.Types.ShapefileTypeMetadata'>
SpatialReferenceTypeMetadata = <class
'GeoEco.Types.SpatialReferenceTypeMetadata'>
StoredObjectTypeMetadata = <class 'GeoEco.Types.StoredObjectTypeMetadata'>
TableFieldTypeMetadata = <class 'GeoEco.Types.TableFieldTypeMetadata'>
TemporaryDirectory = <class
'GeoEco.DataManagement.Directories.TemporaryDirectory'>
TextFileTypeMetadata = <class 'GeoEco.Types.TextFileTypeMetadata'>
TreeModel = <class 'GeoEco.Statistics.Modeling.TreeModel'>
TupleTypeMetadata = <class 'GeoEco.Types.TupleTypeMetadata'>
TypeMetadata = <class 'GeoEco.Metadata.TypeMetadata'>
UnicodeStringTypeMetadata = <class 'GeoEco.Types.UnicodeStringTypeMetadata'>
_ = <function _gettext at 0x1DE6F770>
_FormulaDescription = u'The formula must be in the format expected by the R
glm function::\n\n response ~ term1 + term2 + ... + termN\n\nresponse is
the table field that will be modeled as the response\nvariable and the terms
are the table fields that will serve as the\npredictor variables. The field
names are case sensitive. If any field\nused in the formula is NULL for a
given row, that row will not be used\nin fitting the model.\n\nFor example,
if you have a field Presence that indicates the presence\nor absence of a
species (1 or 0) and you want to model it in terms of\nsampled environmental
covariates stored in the SST, ChlDensity, and\nDepth fields, you would use
the formula::\n\n Presence ~ SST + ChlDensity + Depth\n\nBy default, all
terms are treated as continuous variables. To indicate\nthat a term should be
treated as a categorical variable, use the\nfactor function. For example, if
SubstrateType is an integer code that\nshould be treated as categorical::\n\n
Presence ~ SST + ChlDensity + Depth + factor(SubstrateType)\n\nThe model
terms may also use these operators:\n\n* The : operator denotes the
interaction of variables a and b. For\n example: a:b.\n\n* The * operator
denotes "crossing". For example, a*b is identical to\n a+b+a:b.\n\n* The ^
operator denotes crossing to the Nth degree. For example,\n (a+b+c)^2 is
identical to (a+b+c)*(a+b+c) which in turn expands to a\n formula containing
the main effects for a, b and c together with\n their second-order
interactions.\n\n* The %in% operator indicates that the terms on its left are
nested\n within those on the right. For example a + b %in% a expands to
the\n formula a + a:b.\n\n* The - operator (minus) removes the specified
terms, so that\n (a+b+c)^2 - a:b is identical to a + b + c + b:c + a:c. It
can also\n used to remove the intercept term: y ~ x - 1 is a line through
the\n origin. A model with no intercept can be also specified as y ~ x + 0\n
or y ~ 0 + x.\n\nWhile formulae usually involve just variable names, they
can also\ninvolve arithmetic expressions. The formula log(y) ~ a + log(x)
is\nquite legal. When such arithmetic expressions involve operators
which\nare also used symbolically in model formulae, there can be
confusion\nbetween arithmetic and symbolic operator use.\n\nTo avoid this
confusion, the function I() can be used to bracket those\nportions of a model
formula where the operators are used in their\narithmetic sense. For example,
in the formula y ~ a + I(b+c), the term\nb+c is to be interpreted as the sum
of b and c.\n\nPlease see the topics "glm" and "formula" in the R
documentation for\nmore information.'
_LoadDataFrameForModelFitting = <function _LoadDataFrameForModelFitting at
0x1EB1B7B0>
_PredictFromArcGISRasters = <function _PredictFromArcGISRasters at
0x1EB37570>
_PredictFromArcGISTable = <function _PredictFromArcGISTable at 0x1EB37530>
_PreparePredictorRasters = <function _PreparePredictorRasters at 0x1EB375B0>
__all__ = ['GLM', 'GAM', 'TreeModel', 'LinearMixedModel',
'RandomForestModel', 'ModelEvaluation']
__doc__ = u'Provides methods for modeling and prediction.\n'
__file__ =
'C:\\Python26\\ArcGIS10.0\\lib\\site-packages\\GeoEco\\Statistics\\Modeling.pyc'
__name__ = 'GeoEco.Statistics.Modeling'
__package__ = 'GeoEco.Statistics'
copy = <module 'copy' from 'C:\Python26\ArcGIS10.0\Lib\copy.pyc'>
glob = <module 'glob' from 'C:\Python26\ArcGIS10.0\Lib\glob.pyc'>
os = <module 'os' from 'C:\Python26\ArcGIS10.0\Lib\os.pyc'>
random = <module 'random' from 'C:\Python26\ArcGIS10.0\Lib\random.pyc'>
sys = <module 'sys' (built-in)>
types = <module 'types' from 'C:\Python26\ArcGIS10.0\Lib\types.pyc'>
Enviornment variables:
AGSDESKTOPJAVA = 'C:\\Program Files\\ArcGIS\\Desktop10.0\\'
ALLUSERSPROFILE = 'C:\\Documents and Settings\\All Users'
APPDATA = 'C:\\Documents and Settings\\ehh10\\Application Data'
ARCHOME = 'C:\\Program Files\\ArcGIS\\Desktop10.0\\'
ARCHOME_USER = 'C:\\Program Files\\ArcGIS\\Desktop10.0\\'
ARCPWD = 'Z:\\sharks\\blacktip_bll'
CLIENTNAME = 'COMPUTERLIZA'
CLUSTERLOG = 'C:\\WINDOWS\\Cluster\\cluster.log'
COMMONPROGRAMFILES = 'C:\\Program Files\\Common Files'
COMPUTERNAME = 'ESTEBAN'
COMSPEC = 'C:\\WINDOWS\\system32\\cmd.exe'
DSETPATH = 'C:\\Program Files\\Dell\\DSET\\'
FP_NO_HOST_CHECK = 'NO'
HOMEDRIVE = 'C:'
HOMEPATH = '\\Documents and Settings\\ehh10'
LOGONSERVER = '\\\\RIDLEY'
NUMBER_OF_PROCESSORS = '4'
OS = 'Windows_NT'
PATH = 'C:\\Program Files\\Windows Resource
Kits\\Tools\\;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\Program
Files\\MATLAB\\MATLAB Component
Runtime\\v77\\runtime\\win32;C:\\Python25\\Scripts;C:\\Program
Files\\TortoiseSVN\\bin;E:\\MATLAB\\R2012a\\runtime\\win32;E:\\MATLAB\\R2012a\\bin;C:\\Program
Files\\Microsoft SQL Server\\90\\Tools\\binn\\;C:\\Program
Files\\R\\R-2.15.1\\bin\\i386;C:\\Program Files\\R\\R-2.15.1\\modules\\i386'
PATHEXT = '.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH'
PROCESSOR_ARCHITECTURE = 'x86'
PROCESSOR_IDENTIFIER = 'x86 Family 6 Model 23 Stepping 6, GenuineIntel'
PROCESSOR_LEVEL = '6'
PROCESSOR_REVISION = '1706'
PROGRAMFILES = 'C:\\Program Files'
RPY_RHOME = 'C:\\Program Files\\R\\R-2.15.1'
RPY_RUSER = 'C:\\Documents and Settings\\ehh10\\My Documents'
RPY_RVER = '2151'
RPY_RVERSION = '2.15.1'
R_HOME = 'C:\\Program Files\\R\\R-2.15.1'
SESSIONNAME = 'RDP-Tcp#10'
SYSTEMDRIVE = 'C:'
SYSTEMROOT = 'C:\\WINDOWS'
TEMP = 'C:\\DOCUME~1\\ehh10\\LOCALS~1\\Temp\\3'
TMP = 'C:\\DOCUME~1\\ehh10\\LOCALS~1\\Temp\\3'
USERDOMAIN = 'MGEL'
USERNAME = 'ehh10'
USERPROFILE = 'C:\\Documents and Settings\\ehh10'
WINDIR = 'C:\\WINDOWS'
Other variables:
GeoEco.__version__ = u'0.8a43'
sys.argv =
['C:\\Python26\\ArcGIS10.0\\Lib\\site-packages\\GeoEco\\ArcGISToolbox\\Scripts\\GLMPredictFromArcGISTable.py',
'Z:\\sharks\\blacktip_bll\\glm\\blacktip_bll_glm4', 'bll_blacktip', '#',
'#', '"TestData" = 1', 'true', '#',
'Z:\\sharks\\blacktip_bll\\glm\\blacktipglm4.emf', 'tpr', 'fpr', 'true',
'Z:\\sharks\\blacktip_bll\\glm\\blacktipglmSS4', '1000', '3000', '3000',
'10', 'white', 'bll_blacktip', '0.491230300279']
sys.version = 2.6.5 (r265:79096, Mar 19 2010, 21:48:26) [MSC v.1500 32 bit
(Intel)]
sys.version_info = (2, 6, 5, 'final', 0)
sys.platform = win32
sys.getwindowsversion() = (5, 2, 3790, 2, 'Service Pack 2')
os.getcwd() = Z:\sharks\blacktip_bll
sys.path =
['C:\\Python26\\ArcGIS10.0\\Lib\\site-packages\\GeoEco\\ArcGISToolbox\\Scripts',
'C:\\WINDOWS\\system32\\python26.zip', u'C:\\Program
Files\\ArcGIS\\Desktop10.0\\arcpy', 'C:\\Python26\\ArcGIS10.0\\Lib',
'C:\\Python26\\ArcGIS10.0\\DLLs', 'C:\\Python26\\ArcGIS10.0\\Lib\\lib-tk',
'C:\\Program Files\\ArcGIS\\Desktop10.0\\ArcToolbox\\Toolboxes', 'C:\\Program
Files\\ArcGIS\\Desktop10.0\\bin', 'C:\\Python26\\ArcGIS10.0',
'C:\\Python26\\ArcGIS10.0\\lib\\site-packages', 'C:\\Program
Files\\ArcGIS\\Desktop10.0\\arcpy', 'C:\\Program
Files\\ArcGIS\\Desktop10.0\\ArcToolbox\\Scripts',
'C:\\Python26\\ArcGIS10.0\\lib\\site-packages\\win32',
'C:\\Python26\\ArcGIS10.0\\lib\\site-packages\\win32\\lib',
'C:\\Python26\\ArcGIS10.0\\lib\\site-packages\\Pythonwin']
Loaded modules: ConfigParser, GeoEco, GeoEco.ArcGIS, GeoEco.ArcGISScripts,
GeoEco.AssimilatedModules, GeoEco.AssimilatedModules.GeoEco,
GeoEco.AssimilatedModules.osgeo, GeoEco.AssimilatedModules.osgeo.imp,
GeoEco.AssimilatedModules.osgeo.os, GeoEco.AssimilatedModules.osgeo.osr,
GeoEco.AssimilatedModules.osgeo.sys, GeoEco.AssimilatedModules.rpy,
GeoEco.AssimilatedModules.rpy.UserDict,
GeoEco.AssimilatedModules.rpy.__future__,
GeoEco.AssimilatedModules.rpy._rpy2151, GeoEco.AssimilatedModules.rpy.atexit,
GeoEco.AssimilatedModules.rpy.codecs, GeoEco.AssimilatedModules.rpy.glob,
GeoEco.AssimilatedModules.rpy.numpy, GeoEco.AssimilatedModules.rpy.os,
GeoEco.AssimilatedModules.rpy.re, GeoEco.AssimilatedModules.rpy.rpy,
GeoEco.AssimilatedModules.rpy.rpy_io,
GeoEco.AssimilatedModules.rpy.rpy_options,
GeoEco.AssimilatedModules.rpy.rpy_tools,
GeoEco.AssimilatedModules.rpy.rpy_version, GeoEco.AssimilatedModules.rpy.sys,
GeoEco.AssimilatedModules.rpy.threading, GeoEco.AssimilatedModules.rpy.time,
GeoEco.AssimilatedModules.rpy.win32api,
GeoEco.AssimilatedModules.rpy.win32com,
GeoEco.AssimilatedModules.rpy.win32con, GeoEco.BatchProcessing,
GeoEco.DataManagement, GeoEco.DataManagement.ArcGISRasters,
GeoEco.DataManagement.BinaryRasters, GeoEco.DataManagement.Directories,
GeoEco.DataManagement.Fields, GeoEco.DataManagement.Files,
GeoEco.DataManagement.GeoEco, GeoEco.DataManagement.Queue,
GeoEco.DataManagement.array, GeoEco.DataManagement.bz2,
GeoEco.DataManagement.datetime, GeoEco.DataManagement.errno,
GeoEco.DataManagement.glob, GeoEco.DataManagement.gzip,
GeoEco.DataManagement.inspect, GeoEco.DataManagement.locale,
GeoEco.DataManagement.math, GeoEco.DataManagement.operator,
GeoEco.DataManagement.os, GeoEco.DataManagement.random,
GeoEco.DataManagement.re, GeoEco.DataManagement.shutil,
GeoEco.DataManagement.sys, GeoEco.DataManagement.tarfile,
GeoEco.DataManagement.tempfile, GeoEco.DataManagement.threading,
GeoEco.DataManagement.time, GeoEco.DataManagement.types,
GeoEco.DataManagement.urlparse, GeoEco.DataManagement.zipfile,
GeoEco.DatabaseAccess, GeoEco.DatabaseAccess.ArcGIS,
GeoEco.DatabaseAccess.GeoEco, GeoEco.DatabaseAccess.InMemory,
GeoEco.DatabaseAccess.datetime, GeoEco.DatabaseAccess.logging,
GeoEco.DatabaseAccess.os, GeoEco.DatabaseAccess.repr,
GeoEco.DatabaseAccess.time, GeoEco.DatabaseAccess.types, GeoEco.Datasets,
GeoEco.Datasets.ArcGIS, GeoEco.Datasets.Collections, GeoEco.Datasets.GDAL,
GeoEco.Datasets.GeoEco, GeoEco.Datasets.atexit, GeoEco.Datasets.bisect,
GeoEco.Datasets.codecs, GeoEco.Datasets.copy, GeoEco.Datasets.datetime,
GeoEco.Datasets.locale, GeoEco.Datasets.logging, GeoEco.Datasets.math,
GeoEco.Datasets.os, GeoEco.Datasets.re, GeoEco.Datasets.shutil,
GeoEco.Datasets.types, GeoEco.Datasets.weakref, GeoEco.Dependencies,
GeoEco.DynamicDocString, GeoEco.Exceptions, GeoEco.GeoArray, GeoEco.GeoEco,
GeoEco.Internationalization, GeoEco.Logging, GeoEco.Metadata,
GeoEco.MetadataUtils, GeoEco.R, GeoEco.Statistics, GeoEco.Statistics.GeoEco,
GeoEco.Statistics.Modeling, GeoEco.Statistics.copy, GeoEco.Statistics.glob,
GeoEco.Statistics.os, GeoEco.Statistics.random, GeoEco.Statistics.sys,
GeoEco.Statistics.types, GeoEco.StringIO, GeoEco.Types,
GeoEco.arcgisscripting, GeoEco.atexit, GeoEco.bisect, GeoEco.codecs,
GeoEco.copy, GeoEco.datetime, GeoEco.gc, GeoEco.gettext, GeoEco.glob,
GeoEco.inspect, GeoEco.locale, GeoEco.logging, GeoEco.math, GeoEco.new,
GeoEco.numpy, GeoEco.operator, GeoEco.os, GeoEco.platform, GeoEco.pywintypes,
GeoEco.random, GeoEco.re, GeoEco.repr, GeoEco.sys, GeoEco.time,
GeoEco.traceback, GeoEco.types, GeoEco.weakref, GeoEco.win32api,
GeoEco.win32con, GeoEco.xml, Queue, SocketServer, StringIO, UserDict,
_GeoEcoArcGISHelper, __builtin__, __future__, __main__, _abcoll, _bisect,
_codecs, _collections, _ctypes, _functools, _gdal, _heapq, _locale, _osr,
_random, _socket, _sre, _ssl, _struct, _warnings, _weakref, _win32sysloader,
abc, arcgis, arcgisscripting, arcpy, arcpy._base, arcpy._ga, arcpy._graph,
arcpy._management, arcpy._mapping, arcpy.analysis, arcpy.arc,
arcpy.arcgisscripting, arcpy.arcobjects, arcpy.arcobjects._base,
arcpy.arcobjects.arcobjectconversion, arcpy.arcobjects.arcobjects,
arcpy.arcobjects.collections, arcpy.arcobjects.functools,
arcpy.arcobjects.geometries, arcpy.arcobjects.itertools,
arcpy.arcobjects.mixins, arcpy.arcobjects.operator, arcpy.arcobjects.os,
arcpy.arcobjects.types, arcpy.arcobjects.weakref, arcpy.arcobjects.xml,
arcpy.arcpy, arcpy.cartography, arcpy.conversion, arcpy.ddd, arcpy.edit,
arcpy.fabric, arcpy.functools, arcpy.ga, arcpy.geocoding,
arcpy.geoprocessing, arcpy.geoprocessing._base,
arcpy.geoprocessing.arcgisscripting, arcpy.geoprocessing.functools,
arcpy.geoprocessing.thread, arcpy.geoprocessing.weakref, arcpy.inspect,
arcpy.interop, arcpy.logging, arcpy.lr, arcpy.management, arcpy.mapping,
arcpy.md, arcpy.na, arcpy.os, arcpy.pprint, arcpy.re, arcpy.sa,
arcpy.sa.CompoundParameter, arcpy.sa.Functions, arcpy.sa.ParameterClasses,
arcpy.sa.Utils, arcpy.sa.arcgisscripting, arcpy.sa.arcpy, arcpy.sa.datetime,
arcpy.sa.types, arcpy.schematics, arcpy.server, arcpy.stats, arcpy.sys,
arcpy.ta, arcpy.toolbox, arcpy.types, arcpy.utils, array, atexit, binascii,
bisect, bz2, cPickle, cStringIO, codecs, collections, copy, copy_reg, ctypes,
ctypes._ctypes, ctypes._endian, ctypes.ctypes, ctypes.os, ctypes.struct,
ctypes.sys, datetime, dis, encodings, encodings.__builtin__,
encodings.aliases, encodings.codecs, encodings.cp1252, encodings.encodings,
encodings.utf_8, errno, exceptions, fnmatch, functools, gc, genericpath,
gettext, glob, gzip, heapq, imp, inspect, itertools, keyword, linecache,
locale, logging, logging.ConfigParser, logging.SocketServer, logging.atexit,
logging.cPickle, logging.cStringIO, logging.codecs, logging.config,
logging.handlers, logging.logging, logging.os, logging.re, logging.socket,
logging.stat, logging.string, logging.struct, logging.sys, logging.thread,
logging.threading, logging.time, logging.traceback, logging.types, math, new,
nt, ntpath, numpy, numpy.__builtin__, numpy.__config__, numpy._import_tools,
numpy.add_newdocs, numpy.compat, numpy.compat._inspect, numpy.compat.types,
numpy.core, numpy.core.__builtin__, numpy.core._dotblas,
numpy.core._internal, numpy.core._sort, numpy.core.arrayprint,
numpy.core.cPickle, numpy.core.ctypes, numpy.core.defchararray,
numpy.core.fromnumeric, numpy.core.function_base, numpy.core.getlimits,
numpy.core.info, numpy.core.machar, numpy.core.memmap, numpy.core.multiarray,
numpy.core.numeric, numpy.core.numerictypes, numpy.core.numpy, numpy.core.os,
numpy.core.re, numpy.core.records, numpy.core.scalarmath,
numpy.core.shape_base, numpy.core.sys, numpy.core.types, numpy.core.umath,
numpy.core.warnings, numpy.ctypes, numpy.ctypeslib, numpy.fft,
numpy.fft.fftpack, numpy.fft.fftpack_lite, numpy.fft.helper, numpy.fft.info,
numpy.fft.numpy, numpy.fft.types, numpy.lib, numpy.lib.__builtin__,
numpy.lib.__future__, numpy.lib._compiled_base, numpy.lib._datasource,
numpy.lib._iotools, numpy.lib.arraysetops, numpy.lib.arrayterator,
numpy.lib.cPickle, numpy.lib.cStringIO, numpy.lib.financial,
numpy.lib.format, numpy.lib.function_base, numpy.lib.index_tricks,
numpy.lib.info, numpy.lib.io, numpy.lib.itertools, numpy.lib.math,
numpy.lib.numpy, numpy.lib.operator, numpy.lib.os, numpy.lib.polynomial,
numpy.lib.re, numpy.lib.scimath, numpy.lib.shape_base, numpy.lib.shutil,
numpy.lib.stride_tricks, numpy.lib.sys, numpy.lib.twodim_base,
numpy.lib.type_check, numpy.lib.types, numpy.lib.ufunclike, numpy.lib.utils,
numpy.lib.warnings, numpy.linalg, numpy.linalg.info,
numpy.linalg.lapack_lite, numpy.linalg.linalg, numpy.linalg.numpy, numpy.ma,
numpy.ma.cPickle, numpy.ma.core, numpy.ma.extras, numpy.ma.itertools,
numpy.ma.numpy, numpy.ma.operator, numpy.ma.warnings, numpy.matrixlib,
numpy.matrixlib.defmatrix, numpy.matrixlib.numpy, numpy.matrixlib.sys,
numpy.numpy, numpy.os, numpy.polynomial, numpy.polynomial.__future__,
numpy.polynomial.chebyshev, numpy.polynomial.exceptions,
numpy.polynomial.numpy, numpy.polynomial.polynomial,
numpy.polynomial.polytemplate, numpy.polynomial.polyutils,
numpy.polynomial.string, numpy.polynomial.warnings, numpy.random,
numpy.random.info, numpy.random.mtrand, numpy.random.numpy, numpy.sys,
numpy.testing, numpy.testing.decorators, numpy.testing.nosetester,
numpy.testing.numpy, numpy.testing.numpytest, numpy.testing.operator,
numpy.testing.os, numpy.testing.re, numpy.testing.sys,
numpy.testing.traceback, numpy.testing.types, numpy.testing.unittest,
numpy.testing.utils, numpy.testing.warnings, numpy.version, opcode, operator,
os, os.path, platform, posixpath, pprint, pythoncom, pythoncom.__univgw,
pywintypes, random, re, repr, select, shutil, site, socket, sre_compile,
sre_constants, sre_parse, stat, string, strop, struct, swig_runtime_data4,
sys, tarfile, tempfile, thread, threading, time, token, tokenize, traceback,
types, unittest, urlparse, warnings, weakref, win32api, win32com,
win32com.gen_py, win32com.gen_py.win32com, win32com.os, win32com.pythoncom,
win32com.shell, win32com.shell.shell, win32com.shell.shellcon,
win32com.shell.win32com, win32com.sys, win32com.win32api, win32com.win32com,
win32con, xml, xml.dom, xml.dom.domreg, xml.dom.minicompat, xml.dom.xml,
xml.etree, xml.etree.ElementPath, xml.etree.ElementTree, xml.etree.re,
xml.etree.string, xml.etree.sys, zipfile, zipimport, zlib
---------- END OF DEBUGGING INFORMATION ----------
GeoEco is releasing its reference to ArcGIS Geoprocessor object 0x1EAE22A8
and will no longer use it for ArcGIS operations.
<type 'exceptions.ValueError'>: too many values to unpack
Failed to execute (Predict GLM From Table).
Failed at Fri Sep 21 17:17:49 2012 (Elapsed Time: 2.00 seconds)
I am trying to run a GLM using MGET tools, but I keep getting an error when I use the Predict GLM from table tool. The error reads: "<type 'exceptions.ValueError'>: too many values to unpack Failed to execute (Predict GLM From Table)."
If I leave the "input table (optional)" field blank, the tool runs without problems. However, when I specify an input table it gives me this error (above, and see attachment for all details). Do you know what could be causing this problem? The table only has 1436 entries and about 15 fields.
Thanks so much!
Liza
Executing (Predict GLM From Table): GLMPredictFromArcGISTable
Z:\sharks\blacktip_bll\glm\blacktip_bll_glm4 bll_blacktip # # ""TestData" =
1" true # Z:\sharks\blacktip_bll\glm\blacktipglm4.emf tpr fpr true
Z:\sharks\blacktip_bll\glm\blacktipglmSS4 1000 3000 3000 10 white
Start Time: Fri Sep 21 17:17:47 2012
Running script GLMPredictFromArcGISTable...
GeoEco will now use ArcGIS Geoprocessor object 0x1EAE22A8 for ArcGIS
operations.
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
describe(*args=('C:\\Python26\\ArcGIS10.0\\lib\\site-packages\\GeoEco\\ArcGISToolbox\\Rasters\\dummyint',))...
ArcGIS Geoprocessor object 0x1EAE22A8: describe returned <geoprocessing
describe data object object at 0x1ED30F50>
ArcGIS Geoprocessor object 0x1EAE22A8: Get ParameterCount returned 19
This python script was invoked from an ArcGIS application.
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(0,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned
'Z:\\sharks\\blacktip_bll\\glm\\blacktip_bll_glm4'
Parameter inputModelFile = u'Z:\\sharks\\blacktip_bll\\glm\\blacktip_bll_glm4'
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(1,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned
'bll_blacktip'
Parameter inputTable = u'bll_blacktip'
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(2,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned ''
Parameter predictedValuesField = None
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(3,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned ''
Parameter cutoff = None
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(4,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned
'"TestData" = 1'
Parameter where = u'"TestData" = 1'
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(5,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned 'true'
Parameter ignoreOutOfRangeValues = True
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(6,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned ''
Parameter noDataValue = None
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(7,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned
'Z:\\sharks\\blacktip_bll\\glm\\blacktipglm4.emf'
Parameter outputPlotFile = u'Z:\\sharks\\blacktip_bll\\glm\\blacktipglm4.emf'
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(8,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned 'tpr'
Parameter measure1 = u'tpr'
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(9,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned 'fpr'
Parameter measure2 = u'fpr'
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(10,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned 'true'
Parameter colorize = True
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(11,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned
'Z:\\sharks\\blacktip_bll\\glm\\blacktipglmSS4'
Parameter outputSummaryFile = u'Z:\\sharks\\blacktip_bll\\glm\\blacktipglmSS4'
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(12,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned '1000'
Parameter res = 1000.0
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(13,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned '3000'
Parameter width = 3000.0
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(14,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned '3000'
Parameter height = 3000.0
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(15,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned '10'
Parameter pointSize = 10.0
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
getparameterastext(*args=(16,))...
ArcGIS Geoprocessor object 0x1EAE22A8: getparameterastext returned 'white'
Parameter bg = u'white'
ArcGIS Geoprocessor object 0x1EAE22A8: Get OverwriteOutput returned 1
Parameter overwriteExisting = True
Executing method GeoEco.Statistics.Modeling.GLM.PredictFromArcGISTable.
Checking software dependency: R ROCR package.
R interpreter: Evaluating statements: library(utils)
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
R interpreter: Evaluating statements: library(tools)
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
R interpreter: Evaluating statements: .packages()
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
The R ROCR package has already been loaded into the R interpreter.
Checking software dependency: R e1071 package.
R interpreter: Evaluating statements: library(utils)
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
R interpreter: Evaluating statements: library(tools)
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
R interpreter: Evaluating statements: .packages()
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
The R e1071 package has already been loaded into the R interpreter.
Checking software dependency: R caret package.
R interpreter: Evaluating statements: library(utils)
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
R interpreter: Evaluating statements: library(tools)
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
R interpreter: Evaluating statements: .packages()
R interpreter: The last statement returned [u'caret', u'foreach', u'cluster',
u'reshape', u'plyr', u'lattice', u'e1071', u'class', u'ROCR', u'gplots',
u'MASS', u'KernSmooth', u'grid', u'caTools', u'bitops', u'stats', u'gdata',
u'gtools', u'tools', u'graphics', ...]
The R caret package has already been loaded into the R interpreter.
The file Z:\sharks\blacktip_bll\glm\blacktip_bll_glm4 exists.
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
exists(*args=('bll_blacktip',))...
ArcGIS Geoprocessor object 0x1EAE22A8: exists returned 1
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
describe(*args=('bll_blacktip',))...
ArcGIS Geoprocessor object 0x1EAE22A8: describe returned <geoprocessing
describe data object object at 0x1ED57F38>
ArcGIS Describe object 0x1ED57F38: Get DataType returned 'FeatureLayer'
The ArcGIS table or table view bll_blacktip exists.
The file Z:\sharks\blacktip_bll\glm\blacktipglm4.emf does not exist.
Directory Z:\sharks\blacktip_bll\glm will not be created because it already
exists.
The file Z:\sharks\blacktip_bll\glm\blacktipglmSS4 does not exist.
Directory Z:\sharks\blacktip_bll\glm will not be created because it already
exists.
R interpreter: Evaluating statements:
load("Z:\\sharks\\blacktip_bll\\glm\\blacktip_bll_glm4")
R interpreter: The last statement returned [u'model', u'xVar', u'yVar',
u'zVar', u'mVar', u'coordinateSystem']
R interpreter: Evaluating statements: exists("model")
R interpreter: The last statement returned True
R interpreter: Evaluating statements: tolower(class(model))
R interpreter: The last statement returned [u'glm', u'lm']
R interpreter: Evaluating statements: exists("rPackage")
R interpreter: The last statement returned False
R interpreter: Evaluating statements: class(model)[1]
R interpreter: The last statement returned u'glm'
Loaded glm from Z:\sharks\blacktip_bll\glm\blacktip_bll_glm4.
R interpreter: Evaluating statements: exists("rPackage")
R interpreter: The last statement returned False
The file C:\Python26\ArcGIS10.0\lib\site-packages\GeoEco\Statistics\Utils.r
exists.
R interpreter: Evaluating statements: # Utils.r
#
# Copyright (C) 2011 Jason J. Roberts
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License (available in the file LICENSE.TXT)
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
USA.
CheckForCancel <- function(timeLastChecked=NULL)
{
# If timeLastChecked is NULL, we are being called for the first
# time. If there happens to be a file named
# "GeoEcoCancelRequested_" followed by the process ID that was
# somehow left over from a previous process, delete it so we don't
# misinterpret it as an immediate cancel request. (This should
# never happen, but we do it anyway for safety.)
if (is.null(timeLastChecked))
{
tempDir <- Sys.getenv("TEMP")
if (tempDir == "")
tempDir <- Sys.getenv("TMP")
if (tempDir != "" && file.exists(file.path(tempDir,
sprintf("GeoEcoCancelReqested_%i", Sys.getpid()))))
unlink(file.path(tempDir, sprintf("GeoEcoCancelReqested_%i",
Sys.getpid())))
}
# If five seconds or more have elapsed since we last checked for a
# cancel request, check for one now.
if (is.null(timeLastChecked) || proc.time()[3] >= timeLastChecked + 5)
{
# Write a message with a special string. When the GeoEco
# Python function that handles messages recieves this string,
# it will check for a cancel request (and not log the
# message).
message("CHECK_FOR_CANCEL_REQUEST")
# Look in the TEMP directory file a file named
# "GeoEcoCancelRequested_" followed by the process ID. If we
# find it, call stop with a message indicating we were
# cancelled.
tempDir <- Sys.getenv("TEMP")
if (tempDir == "")
tempDir <- Sys.getenv("TMP")
if (tempDir != "" && file.exists(file.path(tempDir,
sprintf("GeoEcoCancelReqested_%i", Sys.getpid()))))
{
unlink(file.path(tempDir, sprintf("GeoEcoCancelReqested_%i",
Sys.getpid())))
stop("Cancel requested", call.=FALSE)
}
# We were not cancelled. Return the current time.
return(proc.time()[3])
}
# Less than five seconds have elapsed since we last checked for a
# cancel request. Just return the time we last checked.
return(timeLastChecked)
}
R interpreter: The last statement returned R closure function at 0x1ED1AAC0
The file
C:\Python26\ArcGIS10.0\lib\site-packages\GeoEco\Statistics\PredictModel.r
exists.
R interpreter: Evaluating statements: # PredictLMForArcGISRasters.r
#
# Copyright (C) 2008 Jason J. Roberts and Ben D. Best
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License (available in the file LICENSE.TXT)
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
USA.
ckappa <- function (r)
{
r <- na.omit(r)
r1 <- r[, 1]
r2 <- r[, 2]
n1 <- as.character(r1)
n2 <- as.character(r2)
lev <- levels(as.factor(c(n1, n2)))
p <- length(lev)
tab <- matrix(nrow = p, ncol = p)
dimnames(tab) <- list(levels(as.factor(c(n1, n2))),
levels(as.factor(c(n1,
n2))))
dim1 <- dimnames(tab)[[1]]
dim2 <- dimnames(tab)[[2]]
tabi <- table(n1, n2)
dimi1 <- dimnames(tabi)[[1]]
dimi2 <- dimnames(tabi)[[2]]
for (i in 1:p) for (j in 1:p) {
if ((sum(dim1[i] == dimi1) == 1) & (sum(dim2[j] == dimi2) ==
1))
tab[i, j] <- tabi[dim1[i], dim2[j]]
else tab[i, j] <- 0
}
tsum <- sum(tab)
ttab <- tab/tsum
tm1 <- apply(ttab, 1, sum)
tm2 <- apply(ttab, 2, sum)
agreeP <- sum(diag(ttab))
chanceP <- sum(tm1 * tm2)
kappa2 <- (agreeP - chanceP)/(1 - chanceP)
result <- list(table = tab, kappa = kappa2)
result
}
PrepareForPrediction <- function(model, trainingData, rPackage)
{
library(stats) # Not sure if this is still needed here.
# Look up the response variable and the minimum and maximum values of it
in
# the training data.
if (!is.null(rPackage) && rPackage == "party")
{
responseVariable <-
names(model@responses@variables)[1]
responseVariable <- gsub(" +", "", responseVariable)
if (grep("factor(.+)", responseVariable))
responseVariable <- substr(responseVariable, 8,
nchar(responseVariable) - 1)
stop("Support for the party package is not fully implemented: cannot
look up terms of fitted party model.", .call=FALSE)
}
else
{
responseVariable <- all.vars(model$terms)[1]
allPredictors <-
all.vars(model$terms)[2:length(all.vars(model$terms))]
}
minResponseValue <- min(trainingData[,responseVariable])
maxResponseValue <- max(trainingData[,responseVariable])
# Look up the min and max values for each predictor from the data
# that was used to fit the model.
minFittedPredictorValues <- list()
maxFittedPredictorValues <- list()
for (predictor in allPredictors)
{
minFittedPredictorValues[[predictor]] <-
min(c(trainingData[[predictor]]), na.rm=TRUE)
maxFittedPredictorValues[[predictor]] <-
max(c(trainingData[[predictor]]), na.rm=TRUE)
}
isBinaryClassification <- minResponseValue == 0 && maxResponseValue == 1
&&
(!is.null(rPackage) && rPackage == "rpart" &&
model$method == "class" ||
!is.null(rPackage) && rPackage ==
"randomForest" && model$type == "classification" ||
!is.null(rPackage) && rPackage == "party" &&
(model@responses@is_nominal[1]
||
model@responses@is_ordinal[1])
||
(!is.null(rPackage) && rPackage %in% c("mgcv",
"gam") || is.null(rPackage)) && model$family$family %in% c("binomial",
"quasibinomial", "negbin"))
isNonbinaryClassification <- !isBinaryClassification &&
(!is.null(rPackage) && rPackage == "rpart"
&& model$method == "class" ||
!is.null(rPackage) && rPackage ==
"randomForest" && model$type == "classification" ||
!is.null(rPackage) && rPackage == "party" &&
(model@responses@is_nominal[1]
||
model@responses@is_ordinal[1]))
# Build a vector listing the categorical predictors.
if (!is.null(rPackage) && rPackage %in% c("rpart", "randomForest"))
categoricalPredictors <- labels(attr(model$terms,
"dataClasses"))[attr(model$terms, "dataClasses") %in% c("factor", "ordered")]
else if (!is.null(rPackage) && rPackage == "party")
categoricalPredictors <-
names(model@data@get("input"))[lapply(model@data@get("input"),
class) %in% c("factor", "ordered")]
else if (!is.null(rPackage) && rPackage == "nlme")
categoricalPredictors <- c(labels(attr(model$terms,
"dataClasses"))[attr(model$terms, "dataClasses") %in% c("factor",
"ordered")], names(model$modelStruct$reStruct)) # For lme models, use
categorical predictors from fixed effects and all grouping variables from
random effects
else
categoricalPredictors <- names(model$xlevels)
categoricalPredictors <- categoricalPredictors[categoricalPredictors !=
responseVariable]
categoricalPredictors <- categoricalPredictors[categoricalPredictors !=
sprintf("factor(%s)", responseVariable)]
categoricalPredictorLevels <- list()
for (predictor in categoricalPredictors)
if (!is.null(rPackage) && rPackage %in% c("rpart", "nlme",
"randomForest", "party"))
categoricalPredictorLevels[[predictor]] <-
levels(trainingData[[predictor]])
else
categoricalPredictorLevels[[predictor]] <-
model$xlevels[[predictor]]
# Initialize some variables we will use to control warnings we may
# issue about out of range values, etc.
warnedAboutMissingLevels <- list()
for (predictor in categoricalPredictors)
warnedAboutMissingLevels[[predictor]] <- FALSE
allValuesForPredictorAreNA <- list()
for (predictor in allPredictors)
allValuesForPredictorAreNA[[predictor]] <- TRUE
allValuesAreNA <- TRUE
return(list(responseVariable, allPredictors, minResponseValue,
maxResponseValue, isBinaryClassification, isNonbinaryClassification,
minFittedPredictorValues, maxFittedPredictorValues, categoricalPredictors,
categoricalPredictorLevels, warnedAboutMissingLevels,
allValuesForPredictorAreNA, allValuesAreNA))
}
PredictModel <- function(model, rPackage, cutoff, ignoreOutOfRangeValues,
returnPredictedErrors, allPredictors, responseVariable, minResponseValue,
maxResponseValue, minFittedPredictorValues, maxFittedPredictorValues,
categoricalPredictors, categoricalPredictorLevels, warnedAboutMissingLevels,
allValuesForPredictorAreNA, allValuesAreNA, predictorValues)
{
# Determine which records have NA for one or more predictors.
anyAreNA <- logical(length(predictorValues[[1]])) # For each
record, TRUE if any predictor is NA
allAreNA <- logical(length(predictorValues[[1]])) | TRUE # For each
record, TRUE if all predictors are NA
for (predictor in allPredictors)
{
# For predictors that are categorical variables (i.e. factors), we
# cannot do predictions using values that were not considered by the
# model fitting process. If there are any of these, treat them as NA
# and warn the user.
#
# TODO: rpart and cforest models may be able to work around this with
# "surrogate splits". Figure out how that works and modify this if
# needed.
if (predictor %in% categoricalPredictors)
{
isMissingLevel <- !(predictorValues[[predictor]] %in%
categoricalPredictorLevels[[predictor]])
predictorValues[[predictor]][isMissingLevel] <- NA
if (any(isMissingLevel) && !warnedAboutMissingLevels[[predictor]])
{
warning(paste("The categorical variable named", predictor,
"contains values that were not present in the data used to fit the model. No
predictions can be made for these values."), call.=FALSE)
warnedAboutMissingLevels[[predictor]] <- TRUE
}
}
# For predictors that are continuous variables, if the caller wants us
# to ignore values that are outside the range of values used to fit
the
# model, set these to NA.
else if (ignoreOutOfRangeValues)
predictorValues[[predictor]][predictorValues[[predictor]] <
minFittedPredictorValues[[predictor]] | predictorValues[[predictor]] >
maxFittedPredictorValues[[predictor]]] <- NA
# Determine which records have NA for this predictor.
isNA <- is.na(predictorValues[[predictor]])
# Keep track of whether all of the values for this predictor are NA
for
# the entire prediction, which may span multiple calls to us.
allValuesForPredictorAreNA[[predictor]] <-
allValuesForPredictorAreNA[[predictor]] && all(isNA)
# For each record, we need to keep track of whether any or all of the
# predictors are NA. Incorporate this predictor into that calculation.
anyAreNA <- anyAreNA | isNA
allAreNA <- allAreNA & isNA
}
# Do the prediction. The prediction code varies according to the kind of
# model that was fitted. First handle rpart models.
predictedError <- NULL
if (!is.null(rPackage) && rPackage == "rpart")
{
# Do the prediction. For "class" models (i.e. classification trees),
get
# the result as integers, applying the cutoff to binary models. Note
# that rpart can perform predictions on rows where some of the
# predictors are NA, so long as not all of them are NA.
if (model$method == "class")
if (minResponseValue == 0 && maxResponseValue == 1)
{
predictedResponse <-
as.vector(suppressWarnings(predict(model, newdata=predictorValues,
type="prob")[,2]))
if (!is.null(cutoff))
predictedResponse <- as.integer(predictedResponse >=
cutoff)
}
else
predictedResponse <-
as.integer(as.vector(suppressWarnings(predict(model, newdata=predictorValues,
type="class"))))
else
predictedResponse <- as.vector(suppressWarnings(predict(model,
newdata=predictorValues, type="vector")))
# The rpart documentation implies that rpart can do predictions if
# *some* of the predictors are NA but not if *all* of the predictors
are
# NA. We therefore expect that cells that have NA for all predictors
# would result in a predicted value of NA. But rpart does not do that.
#
# So: Set all cells having NA for all predictors to NA.
predictedResponse[allAreNA] <- NA
}
# Handle random forest models fitted with randomForest or party.
else if (!is.null(rPackage) && rPackage %in% c("randomForest", "party"))
{
# For binary classification forests, get the predicted probability of
# the second class and apply the cutoff.
if (rPackage == "randomForest" && model$type == "classification" &&
minResponseValue == 0 && maxResponseValue == 1)
{
predictedResponse <- as.vector(suppressWarnings(predict(model,
newdata=predictorValues, type="prob")[,2]))
if (!is.null(cutoff))
predictedResponse <- as.integer(predictedResponse >= cutoff)
}
else if (rPackage == "party" &&
(model@responses@is_nominal[1]
||
model@responses@is_ordinal[1])
&& minResponseValue == 0 && maxResponseValue == 1)
{
predictedResponse <- sapply(suppressWarnings(predict(model,
newdata=predictorValues, type="prob")), "[", i = 2)
if (!is.null(cutoff))
predictedResponse <- as.integer(predictedResponse >= cutoff)
}
# For other models (regression forests or classification forests with
# more than two classes), get the predicted response. For
classification
# forests, coerce it to an integer.
else
{
predictedResponse <- suppressWarnings(predict(model,
newdata=predictorValues, type="response"))
if (is.factor(predictedResponse))
predictedResponse <- as.integer(as.vector(predictedResponse))
else
predictedResponse <- as.vector(predictedResponse)
}
}
# Handle nlme (not currently supported).
else if (!is.null(rPackage) && rPackage == "nlme")
{
stop("Prediction is not implemented for nlme models yet. Please
contact the MGET development team for assistance.", call.=FALSE)
}
# Handle GLM and GAM.
else
{
# The predict.gam function from the R gam library will fail if we pass
# in NA for covariates that were fitted with the loess smoother (lo).
To
# work around this, set the predictors to dummy values if they are NA.
# For the dummy values, use the minimum values of the predictors used
to
# fit the model. (If we pass in values outside the range of values
used
# to fit the model, loess will report a warning on a per-record basis,
# drowning the user in warnings.) These dummy predictor values will
# result in bogus predictions of the response variable. Later, we will
# convert these back to NA.
for (predictor in allPredictors)
predictorValues[[predictor]][anyAreNA] <-
minFittedPredictorValues[[predictor]]
# Do the prediction.
predictions <- suppressWarnings(predict(model,
newdata=predictorValues, type="response", se.fit=returnPredictedErrors))
# If the caller requested predicted errors, extract both the predicted
# response and the errors. Otherwise just extract the response.
if (returnPredictedErrors)
{
predictedResponse <- as.vector(predictions$fit)
predictedError <- as.vector(predictions$se.fit)
}
else
predictedResponse <- as.vector(predictions)
# Set rows that had NA for one or more predictor variables back to NA.
# This will overwrite bogus predictions that resulted from dummy
values.
predictedResponse[anyAreNA] <- NA
if (returnPredictedErrors)
predictedError[anyAreNA] <- NA
# If the caller provided a cutoff, classify the response into
integers.
if (!is.null(cutoff))
predictedResponse <- as.integer(predictedResponse >= cutoff)
}
# Keep track of whether the predicted values were NA for the entire
# prediction, which may span multiple calls to us.
allValuesAreNA <- allValuesAreNA && all(is.na(predictedResponse))
# Return the predicted response and error.
return(list(predictedResponse, predictedError, warnedAboutMissingLevels,
allValuesForPredictorAreNA, allValuesAreNA))
}
PrintPredictionWarnings <- function(rPackage, allPredictors,
allValuesForPredictorAreNA, allValuesAreNA)
{
# For non-rpart models, warn the user about any predictors were NA for all
# records or cells.
if (is.null(rPackage) || rPackage != "rpart")
{
naPredictors <- character()
for (predictor in allPredictors)
if (allValuesForPredictorAreNA[[predictor]])
naPredictors <- append(naPredictors, predictor,
length(naPredictors))
if (length(naPredictors) > 0)
if (length(naPredictors) == 1)
warning(paste("All of the values provided for the predictor
variable", naPredictors[1], " were NULL or NoData. A response can only be
predicted when all of the predictor variables have data. As a result, all of
the predicted values will be database NULL values (if you are predicting
values from a table) or NoData values (if you are predicting values from
rasters)."), call.=FALSE)
else
warning(paste("All of the values provided for the following
predictor variables were NULL or NoData: ", paste(naPredictors, collapse=",
"), ". A response can only be predicted when all of the predictor variables
have data. As a result, all of the predicted values will be database NULL
values (if you are predicting values from a table) or NoData values (if you
are predicting values from rasters).", sep=""), call.=FALSE)
# Warn the user if no predictions could be performed (even though the
# predictors each had data for at least one record or cell).
else if (allValuesAreNA)
warning("All of the predictions resulted in \"NA\" values and
will be stored as database NULL values (if you are predicting values from a
table) or NoData values (if you are predicting values from rasters). This
usually happens when predictor values are missing (they are NULL or NoData)
or not within the range used to fit the model. If this result is unexpected,
check the predictor values you provided as input and make sure they are
correct.", call.=FALSE)
}
# For rpart models, only warn the user if all predicted values were NA.
else if (allValuesAreNA)
warning("All of the predictions resulted in \"NA\" values and will be
stored as database NULL values (if you are predicting values from a table) or
NoData values (if you are predicting values from rasters). This usually
happens when predictor values are missing (they are NULL or NoData) or not
within the range used to fit the model. If this result is unexpected, check
the predictor values you provided as input and make sure they are correct.",
call.=FALSE)
}
PredictModelForArcGISRasters <- function(model, trainingData, rPackage,
rastersForPredictors, constantsForPredictors, outputResponseFile,
outputErrorFile=NULL, ignoreOutOfRangeValues=FALSE, cutoff=NULL)
{
library(stats)
# Initialize variables needed to do a prediction.
if (is.null(rastersForPredictors) || length(rastersForPredictors) <= 0)
stop("You must specify at least one predictor raster.", call.=FALSE)
variables <- PrepareForPrediction(model, trainingData, rPackage)
responseVariable <- variables[[1]]
allPredictors <- variables[[2]]
minResponseValue <- variables[[3]]
maxResponseValue <- variables[[4]]
isBinaryClassification <- variables[[5]]
isNonbinaryClassification <- variables[[6]]
minFittedPredictorValues <- variables[[7]]
maxFittedPredictorValues <- variables[[8]]
categoricalPredictors <- variables[[9]]
categoricalPredictorLevels <- variables[[10]]
warnedAboutMissingLevels <- variables[[11]]
allValuesForPredictorAreNA <- variables[[12]]
allValuesAreNA <- variables[[13]]
# Verify that the caller provided all of the necessary predictors.
for (predictor in allPredictors)
if (!(predictor %in% labels(rastersForPredictors)) &&
(!is.null(constantsForPredictors) && !(predictor %in%
labels(constantsForPredictors))))
stop(sprintf("The model included a predictor named \"%s\" but you
did not specify a raster or constant value for it. Please do that and try
again.", predictor), call.=FALSE)
# Load the rgdal library, which we use to read the predictor rasters.
library(rgdal)
# Obtain the characteristics of the predictor rasters. For safety, verify
# that they all have the same dimensions. The Python function that
generated
# them should guarantee this, but it is cheap to check them here.
rasterInfo <- open.SpatialGDAL(rastersForPredictors[1], silent=TRUE)
tryCatch(
{
if (length(rastersForPredictors) > 1)
for (i in 2:length(rastersForPredictors))
{
rasterInfo2 <- open.SpatialGDAL(rastersForPredictors[i],
silent=TRUE)
tryCatch(
{
if
(any(rasterInfo2@
!=
rasterInfo@))
stop("All of the predictor rasters must have the same
number of rows and columns.", call.=FALSE)
}, finally=close(rasterInfo2))
}
rasterCols <-
as.integer(round(rasterInfo@[1]))
rasterRows <-
as.integer(round(rasterInfo@[2]))
rasterXLLCorner <-
rasterInfo@bbox["x","min"]
rasterYLLCorner <-
rasterInfo@bbox["y","min"]
rasterCellSize <-
rasterInfo@grid@cellsize[1]
}, finally=close(rasterInfo))
# Determine the data type and NoData value of the output rasters.
# If a cutoff is provided (indicating that the output should be a
# binary classification) or the model is a classification with
# more than two classes, use the most compact integer data type
# and corresponding NoData value. Otherwise (for regression models
# or classification models with two classes that but no cutoff was
# provided), use 32-bit floating point as the data type and
# -3.4028235e+038 as the NoData value (the 32-bit IEEE-754
# floating-point negative number that is farthest from 0;
# traditionally used by ArcGIS for NoData).
if (!is.null(cutoff) || isNonbinaryClassification)
{
if (minResponseValue > -128 && maxResponseValue <= 127)
{
dataType <- "signedint"
nbytes <- 1
noDataValue <- as.integer(-128)
}
else if (minResponseValue >= -128 && maxResponseValue < 127)
{
dataType <- "signedint"
nbytes <- 1
noDataValue <- as.integer(127)
}
else if (minResponseValue > 0 && maxResponseValue <= 255)
{
dataType <- "unsignedint"
nbytes <- 1
noDataValue <- as.integer(0)
}
else if (minResponseValue >= 0 && maxResponseValue < 255)
{
dataType <- "unsignedint"
nbytes <- 1
noDataValue <- as.integer(255)
}
else if (minResponseValue > -32768 && maxResponseValue <= 32767)
{
dataType <- "signedint"
nbytes <- 2
noDataValue <- as.integer(-32768)
}
else if (minResponseValue >= -32768 && maxResponseValue < 32767)
{
dataType <- "signedint"
nbytes <- 2
noDataValue <- as.integer(32767)
}
else if (minResponseValue > 0 && maxResponseValue <= 65535)
{
dataType <- "unsignedint"
nbytes <- 2
noDataValue <- as.integer(0)
}
else if (minResponseValue >= 0 && maxResponseValue < 65535)
{
dataType <- "unsignedint"
nbytes <- 2
noDataValue <- as.integer(65535)
}
else if (minResponseValue > -2147483648 && maxResponseValue <=
2147483647)
{
dataType <- "signedint"
nbytes <- 4
noDataValue <- as.integer(-2147483648)
}
else if (minResponseValue >= -2147483648 && maxResponseValue <
2147483647)
{
dataType <- "signedint"
nbytes <- 4
noDataValue <- as.integer(2147483647)
}
else if (minResponseValue > 0)
{
dataType <- "unsignedint"
nbytes <- 4
noDataValue <- as.integer(0)
}
else
{
dataType <- "unsignedint"
nbytes <- 4
noDataValue <- as.integer(4294967296)
}
}
else
{
if (isBinaryClassification && is.null(cutoff))
warning("This appears to be a binary classification model but no
cutoff value was provided. The output will be a continuous floating-point
value ranging from 0 to 1. To obtain a binary integer value (0 or 1), please
provide a cutoff and try again.", call.=FALSE)
dataType <- "float"
nbytes <- 4
noDataValue <- -3.4028235e+038
}
# Create the .hdr files that describe the binaries we will write.
header <- file(paste(outputResponseFile, ".hdr", sep=""), "wb")
tryCatch(
{
cat("ncols ", rasterCols, "\r\n", sep="", file=header)
cat("nrows ", rasterRows, "\r\n", sep="", file=header)
cat("cellsize ", formatC(rasterCellSize, format="g", digits=15,
width=-1), "\r\n", sep="", file=header)
cat("xllcorner ", formatC(rasterXLLCorner, format="g", digits=15,
width=-1), "\r\n", sep="", file=header)
cat("yllcorner ", formatC(rasterYLLCorner, format="g", digits=15,
width=-1), "\r\n", sep="", file=header)
if (dataType == "float")
cat("nodata_value ", formatC(noDataValue, format="g", digits=8,
width=-1), "\r\n", sep="", file=header)
else
cat("nodata_value ", noDataValue, "\r\n", sep="", file=header)
cat("nbits ", as.integer(nbytes * 8), "\r\n", sep="", file=header)
cat("pixeltype ", dataType, "\r\n", sep="", file=header)
if (.Platform$endian == "little")
cat("byteorder lsbfirst\r\n", file=header)
else
cat("byteorder msbfirst\r\n", file=header)
}, finally=close(header))
if (!is.null(outputErrorFile))
{
header <- file(paste(outputErrorFile, ".hdr", sep=""), "wb")
tryCatch(
{
cat("ncols ", rasterCols, "\r\n", sep="", file=header)
cat("nrows ", rasterRows, "\r\n", sep="", file=header)
cat("cellsize ", formatC(rasterCellSize, format="g", digits=15,
width=-1), "\r\n", sep="", file=header)
cat("xllcorner ", formatC(rasterXLLCorner, format="g", digits=15,
width=-1), "\r\n", sep="", file=header)
cat("yllcorner ", formatC(rasterYLLCorner, format="g", digits=15,
width=-1), "\r\n", sep="", file=header)
cat("nodata_value ", formatC(noDataValue, format="g", digits=8,
width=-1), "\r\n", sep="", file=header)
cat("nbits 32\r\n", file=header)
cat("pixeltype float\r\n", file=header)
if (.Platform$endian == "little")
cat("byteorder lsbfirst\r\n", file=header)
else
cat("byteorder msbfirst\r\n", file=header)
}, finally=close(header))
}
# Open the output binaries.
responseFile <- file(paste(outputResponseFile, ".bil", sep=""), "wb")
tryCatch(
{
if (!is.null(outputErrorFile))
errorFile <- file(paste(outputErrorFile, ".bil", sep=""), "wb")
tryCatch(
{
# Open each predictor raster as a GDAL dataset.
datasets <- character()
tryCatch(
{
datasets <- list()
for (predictor in labels(rastersForPredictors))
datasets[[predictor]] <-
GDAL.open(rastersForPredictors[[predictor]])
# For each row of the predictor rasters, do a prediction and
# write the results to the output files. We do this one row
at a
# time to minimize the amount of memory required to run the
# tool. This is slower than reading everything into memory and
# doing the prediction all in one shot (1.5x slower in my
# limited tests) but it is well worth it, since doing it in
one
# shot can require too much memory.
timeCancelLastChecked <- CheckForCancel()
for (row in 1:rasterRows)
{
timeCancelLastChecked <-
CheckForCancel(timeCancelLastChecked)
# If we have processed 1% of the rows, estimate
# the time required to process the rest. If it
# exceeds 1 minute, report a warning message to
# alert the user that he will have to wait a
# while.
if (row == 2)
started = Sys.time()
else if (row == round(rasterRows / 100) + 1 &&
as.numeric((Sys.time() - started) * 99, units="mins") > 1)
warning(sprintf("This prediction is estimated to
require %0.1f minutes to complete.", as.numeric((Sys.time() - started),
units="mins") * 99), call.=FALSE)
# Read this row from each predictor raster.
predictorValues <- list()
for (predictor in labels(rastersForPredictors))
predictorValues[[predictor]] <-
getRasterData(datasets[[predictor]], offset=c(row-1,0),
region.dim=c(1,rasterCols), as.is=TRUE)
# For predictors that have constant values, allocate
vectors
# that repeat those values the appropriate number of
times.
if (!is.null(constantsForPredictors))
for (predictor in labels(constantsForPredictors))
predictorValues[[predictor]] <-
rep(constantsForPredictors[[predictor]],
length(predictorValues[[labels(rastersForPredictors)[1]]]))
# Do the prediction.
result <- PredictModel(model, rPackage, cutoff,
ignoreOutOfRangeValues, !is.null(outputErrorFile), allPredictors,
responseVariable, minResponseValue, maxResponseValue,
minFittedPredictorValues, maxFittedPredictorValues, categoricalPredictors,
categoricalPredictorLevels, warnedAboutMissingLevels,
allValuesForPredictorAreNA, allValuesAreNA, predictorValues)
predictedResponse <- result[[1]]
predictedError <- result[[2]]
warnedAboutMissingLevels <- result[[3]]
allValuesForPredictorAreNA <- result[[4]]
allValuesAreNA <- result[[5]]
# Convert NAs to NoData values.
predictedResponse[is.na(predictedResponse)] <- noDataValue
if (!is.null(outputErrorFile))
predictedError[is.na(predictedError)] <- noDataValue
# Write the row to the binary files.
writeBin(predictedResponse, responseFile, size=nbytes)
if (!is.null(outputErrorFile))
writeBin(predictedError, errorFile, size=nbytes)
}
}, finally=for (ds in datasets) GDAL.close(ds))
}, finally=if (!is.null(outputErrorFile)) close(errorFile))
}, finally=close(responseFile))
# Warn the user about any predictors were NoData for all cells, etc.
PrintPredictionWarnings(rPackage, allPredictors,
allValuesForPredictorAreNA, allValuesAreNA)
}
PredictModelForDataframe <- function(model, rPackage, trainingData,
newData=NULL, ignoreOutOfRangeValues=FALSE, cutoff=NULL, outputPlotFile=NULL,
measure1="tpr", measure2="fpr", colorize=TRUE, outputSummaryFile=NULL,
res=1000.0, width=3000.0, height=3000.0, pointSize=10.0, bg="white")
{
library(stats)
# Initialize variables needed to do a prediction.
variables <- PrepareForPrediction(model, trainingData, rPackage)
responseVariable <- variables[[1]]
allPredictors <- variables[[2]]
minResponseValue <- variables[[3]]
maxResponseValue <- variables[[4]]
isBinaryClassification <- variables[[5]]
isNonbinaryClassification <- variables[[6]]
minFittedPredictorValues <- variables[[7]]
maxFittedPredictorValues <- variables[[8]]
categoricalPredictors <- variables[[9]]
categoricalPredictorLevels <- variables[[10]]
warnedAboutMissingLevels <- variables[[11]]
allValuesForPredictorAreNA <- variables[[12]]
allValuesAreNA <- variables[[13]]
# Verify that the caller provided all of the necessary predictors.
useTrainingData <- is.null(newData)
if (useTrainingData)
newData <- trainingData
else
for (predictor in allPredictors)
if (!(predictor %in% labels(newData)[[2]]))
stop(sprintf("The model included a predictor named \"%s\" but
newData does not contain a column for it. Please add that column and try
again.", predictor), call.=FALSE)
# Do the prediction. For binary models, force the probability to be
returned.
predictedResponse <- PredictModel(model, rPackage, NULL,
ignoreOutOfRangeValues, FALSE, allPredictors, responseVariable,
minResponseValue, maxResponseValue, minFittedPredictorValues,
maxFittedPredictorValues, categoricalPredictors, categoricalPredictorLevels,
warnedAboutMissingLevels, allValuesForPredictorAreNA, allValuesAreNA,
newData)[[1]]
# If the data include the response variable, compute summary statistics.
if (responseVariable %in% labels(newData)[[2]])
{
if (useTrainingData)
modelSummary <- paste("MODEL PERFORMANCE SUMMARY:\n",
"==========================\n",
"\n",
"Statistics calculated from the training
data.\n",
sep="")
else
modelSummary <- paste("MODEL PERFORMANCE SUMMARY:\n",
"==========================\n",
sep="")
# If it is a binary classification model and the caller did not
provide
# a cutoff, calculate one now using the ROCR package.
cutoffAutomaticallySelected <- FALSE
if (isBinaryClassification && is.null(cutoff))
{
library(ROCR)
pred <- prediction(predictedResponse, newData[[responseVariable]])
perf <- performance(pred, 'sens', 'spec')
bestCutoff <- which.max(mapply(sum,
unlist(),
unlist(),
-1))
cutoff <-
pred@cutoffs[[1]][bestCutoff]
cutoffAutomaticallySelected <- TRUE
}
response <- data.frame(actual=newData[[responseVariable]],
predicted=predictedResponse)
response <- na.omit(response)
# For binary classification models, compute summary statistics using
# ROCR.
if (isBinaryClassification)
{
library(ROCR)
pred <- prediction(response$predicted, response$actual)
# Do the following calculations in a try, so that the whole
function
# does not fail for certain input data. The only case I know of is
# the prbe calculation can fail with:
#
# Error in
.performance.precision.recall.break.even.point(predictions = c(1L, : Not
enough distinct predictions to compute precision/recall intersections.
auc <- NA
mxe <- NA
prbe <- NA
rmse <- NA
try({ auc <- performance(pred, "auc")@y.values[[1]] },
silent=TRUE)
try({ mxe <- performance(pred, "mxe")@y.values[[1]] },
silent=TRUE)
try({ prbe <- unlist(performance(pred, "prbe")@y.values) },
silent=TRUE)
try({ rmse <- performance(pred, "rmse")@y.values[[1]] },
silent=TRUE)
if (length(prbe) > 0)
prbe <- prbe[length(prbe)]
else
prbe <- NA
modelSummary <- paste(modelSummary,
"\n",
sprintf("Area under the ROC curve (auc)
= %.3f\n", auc),
sprintf("Mean cross-entropy (mxe)
= %.3f\n", mxe),
sprintf("Precision-recall break-even point
(prbe) = %.3f\n", prbe),
sprintf("Root-mean square error (rmse)
= %.3f\n", rmse),
sep="")
tn = length(which((response$predicted < cutoff) &
(response$actual == 0)))
fn = length(which((response$predicted < cutoff) &
(response$actual != 0)))
tp = length(which((response$predicted >= cutoff) &
(response$actual == 1)))
fp = length(which((response$predicted >= cutoff) &
(response$actual != 1)))
if (cutoffAutomaticallySelected)
modelSummary <- paste(modelSummary,
"\n",
sprintf("Cutoff selected by maximizing
the Youden index = %.3f\n", cutoff),
sep="")
else
modelSummary <- paste(modelSummary,
"\n",
sprintf("User-specified cutoff =
%.3f\n", cutoff),
sep="")
modelSummary <- paste(modelSummary,
"\n",
"Confusion matrix for that cutoff:\n",
"\n",
" Actual 1 Actual 0
Total\n",
sprintf("Predicted 1 %9i %9i %9i\n", tp,
fp, tp+fp),
sprintf("Predicted 0 %9i %9i %9i\n", fn,
tn, tn+fn),
sprintf(" Total %9i %9i %9i\n", tp+fn,
fp+tn, tp+fn+fp+tn),
sep="")
tn <- as.double(tn)
fn <- as.double(fn)
tp <- as.double(tp)
fp <- as.double(fp)
acc <- (tp+tn)/(tp+fp+tn+fn)
modelSummary <- paste(modelSummary,
"\n",
"Model performance statistics for that
cutoff:\n",
"\n",
sprintf("Accuracy (acc)
= %.3f\n", acc),
sprintf("Error rate (err)
= %.3f\n", (fp+fn)/(tp+fp+tn+fn)),
sprintf("Rate of positive predictions (rpp)
= %.3f\n", (tp+fp)/(tp+fp+tn+fn)),
sprintf("Rate of negative predictions (rnp)
= %.3f\n", (tn+fn)/(tp+fp+tn+fn)),
"\n",
sprintf("True positive rate (tpr, or
sensitivity) = %.3f\n", tp/(tp+fn)),
sprintf("False positive rate (fpr, or
fallout) = %.3f\n", fp/(fp+tn)),
sprintf("True negative rate (tnr, or
specificity) = %.3f\n", tn/(fp+tn)),
sprintf("False negative rate (fnr, or miss)
= %.3f\n", fn/(tp+fn)),
"\n",
sprintf("Positive prediction value (ppv, or
precision) = %.3f\n", tp/(tp+fp)),
sprintf("Negative prediction value (npv)
= %.3f\n", tn/(tn+fn)),
sprintf("Prediction-conditioned fallout
(pcfall) = %.3f\n", fp/(tp+fp)),
sprintf("Prediction-conditioned miss
(pcmiss) = %.3f\n", fn/(tn+fn)),
"\n",
sprintf("Matthews correlation coefficient
(mcc) = %.3f\n", (tp*tn -
fp*fn)/sqrt((tp+fn)*(fp+tn)*(tp+fp)*(fn+tn))),
sprintf("Odds ratio (odds)
= %.3f\n", (tp*tn)/(fn*fp)),
sprintf("SAR
= %.3f\n", (acc + auc + rmse)/3),
sep="")
# Create the performance plot.
if (!is.null(outputPlotFile))
{
perf <- performance(pred, measure1, measure2)
if (tolower(substring(outputPlotFile,
nchar(outputPlotFile)-3, nchar(outputPlotFile))) == ".emf")
win.metafile(outputPlotFile, width=width, height=height,
pointsize=pointSize)
else
png(outputPlotFile, res=res, width=width, height=height,
pointsize=pointSize, bg=bg)
tryCatch({
if (measure1 == "tpr" && measure2 == "fpr") # Handle
ROC plots specially
{
plot(perf, colorize=colorize, lwd=5)
abline(0, 1, lty=2, lwd=0.5, col="grey25")
tpr = tp/(tp+fn)
fpr = fp/(fp+tn)
if (colorize)
points(x=fpr, y=tpr, cex=1.5, lwd=2)
else
points(x=fpr, y=tpr, pch=21, cex=1.5, lwd=2,
bg=bg)
text(x=fpr+0.03, y=tpr-0.03, labels=sprintf("Cutoff =
%.3f", cutoff), adj=c(0,1))
}
else
plot(perf, colorize=colorize && measure2 != "cutoff")
}, finally={ graphics.off() })
}
# Using the cutoff, classify the continuous (0..1) response to
# binary values (0 or 1), and calculate and report Cohen's kappa.
predictedResponse <- as.integer(predictedResponse >= cutoff)
response <- data.frame(actual=newData[[responseVariable]],
predicted=predictedResponse)
response <- na.omit(response)
modelSummary <- paste(modelSummary,
"\n",
sprintf("Cohen's kappa (K)
= %.3f\n", ckappa(response)$kappa),
sep="")
}
# For other classification models, compute summary statistics using
# caret.
else if (isNonbinaryClassification)
{
library(caret)
factorLevels <- unique(c(response$predicted, response$actual))
modelSummary <- paste(modelSummary,
"\n",
paste(capture.output(print(confusionMatrix(factor(response$predicted,
levels=factorLevels), factor(response$actual, levels=factorLevels),
dnn=c("Predicted", "Actual")))), sep="", collapse="\n"), "\n",
sep="")
}
# For regression models, compute % variance explained (a.k.a. Efron's
# pseudo R-squared).
else
{
varianceExplained <- 1 - sum((response$actual -
response$predicted)**2) / sum((response$actual - mean(response$actual))**2)
mse <- mean((response$actual - response$predicted)**2)
rmse <- sqrt(mse)
r <- cor(response$actual, response$predicted, method="pearson")
tau <- cor(response$actual, response$predicted, method="kendall")
rho <- cor(response$actual, response$predicted, method="spearman")
modelSummary <- paste(modelSummary,
"\n",
sprintf("Number of records = %i\n",
length(response$actual)),
"\n",
sprintf("Variance Explained =
%.1f%%\n", varianceExplained * 100),
sprintf("Mean Squared Error = %f\n",
mse),
sprintf("Root Mean Squared Error = %f\n",
rmse),
"\n",
sprintf("Pearson's r = %.3f\n", r),
sprintf("Kendall's tau = %.3f\n", tau),
sprintf("Spearman's rho = %.3f\n", rho),
sep="")
}
# Print the summary statistics.
message("")
writeLines(modelSummary)
message("")
# If the caller provided a summary file path, write the summary
# statistics to the file.
if (!is.null(outputSummaryFile))
{
f <- file(outputSummaryFile, "wt")
tryCatch(writeLines(modelSummary, f), finally=close(f))
}
}
return(list(predictedResponse, cutoff))
}
R interpreter: The last statement returned R closure function at 0x1ED1AA70
R interpreter: Evaluating statements: exists("rPackage")
R interpreter: The last statement returned False
R interpreter: Evaluating statements: exists("rPackage")
R interpreter: The last statement returned False
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
exists(*args=('bll_blacktip',))...
ArcGIS Geoprocessor object 0x1EAE22A8: exists returned 1
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
describe(*args=('bll_blacktip',))...
ArcGIS Geoprocessor object 0x1EAE22A8: describe returned <geoprocessing
describe data object object at 0x1ED57608>
ArcGIS Describe object 0x1ED57608: Get Fields returned <geoprocessing list
object object at 0x1ED570F8>
ArcGIS Describe object 0x1ED57608: Get Fields returned <geoprocessing list
object object at 0x1ED570F8>
ArcGIS Describe object 0x1ED57608: Get Fields returned <geoprocessing list
object object at 0x1ED570F8>
ArcGIS Fields object 0x1ED570F8: Invoking next(*args=())...
ArcGIS Fields object 0x1ED570F8: next returned <geoprocessing describe field
object object at 0x1ED572D8>
ArcGIS Describe object 0x1ED57608: Get DataType returned 'FeatureLayer'
ArcGIS Describe object 0x1ED57608: Get CatalogPath returned
'Z:\\sharks\\blacktip_bll\\blacktip_bll.gdb\\bll_blacktip'
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
describe(*args=('Z:\\sharks\\blacktip_bll\\blacktip_bll.gdb\\bll_blacktip',))...
ArcGIS Geoprocessor object 0x1EAE22A8: describe returned <geoprocessing
describe data object object at 0x1ED57E18>
ArcGIS Describe object 0x1ED57E18: Get DataType returned 'FeatureClass'
ArcGIS Describe object 0x1ED57608: Get CatalogPath returned
'Z:\\sharks\\blacktip_bll\\blacktip_bll.gdb\\bll_blacktip'
ArcGIS Describe object 0x1ED57608: Get OIDFieldName returned 'OBJECTID'
ArcGIS Describe object 0x1ED57608: Get OIDFieldName returned 'OBJECTID'
ArcGIS Describe object 0x1ED57608: Get OIDFieldName returned 'OBJECTID'
ArcGIS Describe object 0x1ED57608: Get OIDFieldName returned 'OBJECTID'
ArcGIS Describe object 0x1ED57608: Get ShapeFieldName returned 'Shape'
ArcGIS Describe object 0x1ED57608: Get ShapeFieldName returned 'Shape'
ArcGIS Describe object 0x1ED57608: Get ShapeFieldName returned 'Shape'
ArcGIS Describe object 0x1ED57608: Get ShapeFieldName returned 'Shape'
ArcGIS Describe object 0x1ED57608: Get ShapeType returned 'Point'
ArcGIS Describe object 0x1ED57608: Get HasZ returned False
ArcGIS Describe object 0x1ED57608: Get HasZ returned False
ArcGIS Describe object 0x1ED57608: Get SpatialReference returned
<geoprocessing spatial reference object object at 0x1ED57E18>
ArcGIS Geoprocessor object 0x1EAE22A8: Invoking
CreateSpatialReference_management(*args=(<geoprocessing spatial reference
object object at 0x1ED57E18>,))...
Executing: CreateSpatialReference
"PROJCS['North_America_Lambert_Conformal_Conic',GEOGCS['GCS_North_American_1983',DATUM['D_North_American_1983',SPHEROID['GRS_1980',6378137.0,298.257222101]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],PROJECTION['Lambert_Conformal_Conic'],PARAMETER['False_Easting',0.0],PARAMETER['False_Northing',0.0],PARAMETER['Central_Meridian',-79.5],PARAMETER['Standard_Parallel_1',27.5],PARAMETER['Standard_Parallel_2',41.5],PARAMETER['Latitude_Of_Origin',20.0],UNIT['Meter',1.0]];-37658500
-26849000 9999.99370426336;-100000 10000;-100000
10000;9.9999998928979E-04;0.001;0.001;IsHighPrecision" # # # # # 0
Start Time: Fri Sep 21 17:17:49 2012
Spatial Reference = North_America_Lambert_Conformal_Conic
XY Domain (XMin,YMin XMax,YMax) = -37658500,-26849000
900682834044,900693643544
Z Domain (Min,Max) = -100000,900719825474.099
M Domain (Min,Max) = -100000,900719825474.099
Succeeded at Fri Sep 21 17:17:49 2012 (Elapsed Time: 0.00 seconds)
ArcGIS Geoprocessor object 0x1EAE22A8: CreateSpatialReference_management
returned
"PROJCS['North_America_Lambert_Conformal_Conic',GEOGCS['GCS_North_American_1983',DATUM['D_North_American_1983',SPHEROID['GRS_1980',6378137.0,298.257222101]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],PROJECTION['Lambert_Conformal_Conic'],PARAMETER['False_Easting',0.0],PARAMETER['False_Northing',0.0],PARAMETER['Central_Meridian',-79.5],PARAMETER['Standard_Parallel_1',27.5],PARAMETER['Standard_Parallel_2',41.5],PARAMETER['Latitude_Of_Origin',20.0],UNIT['Meter',1.0]];-37658500
-26849000 9999.99370426336;-100000 10000;-100000
10000;9.9999998928979E-04;0.001;0.001;IsHighPrecision"
ArcGIS Describe object 0x1ED57608: Get Fields returned <geoprocessing list
object object at 0x1ED57548>
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57950>
ArcGIS Field object 0x1ED57950: Get Type returned 'OID'
ArcGIS Field object 0x1ED57950: Get IsNullable returned 'FALSE'
ArcGIS Field object 0x1ED57950: Get Name returned 'OBJECTID'
ArcGIS Field object 0x1ED57950: Get Length returned 4
ArcGIS Field object 0x1ED57950: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57848>
ArcGIS Field object 0x1ED57848: Get Type returned 'Geometry'
ArcGIS Field object 0x1ED57848: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57848: Get Name returned 'Shape'
ArcGIS Field object 0x1ED57848: Get Name returned 'Shape'
ArcGIS Field object 0x1ED57848: Get Name returned 'Shape'
ArcGIS Field object 0x1ED57848: Get Length returned 0
ArcGIS Field object 0x1ED57848: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57950>
ArcGIS Field object 0x1ED57950: Get Type returned 'Double'
ArcGIS Field object 0x1ED57950: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57950: Get Name returned 'DD_LAT'
ArcGIS Field object 0x1ED57950: Get Name returned 'DD_LAT'
ArcGIS Field object 0x1ED57950: Get Name returned 'DD_LAT'
ArcGIS Field object 0x1ED57950: Get Length returned 8
ArcGIS Field object 0x1ED57950: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED579C8>
ArcGIS Field object 0x1ED579C8: Get Type returned 'Double'
ArcGIS Field object 0x1ED579C8: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED579C8: Get Name returned 'DD_LONG'
ArcGIS Field object 0x1ED579C8: Get Name returned 'DD_LONG'
ArcGIS Field object 0x1ED579C8: Get Name returned 'DD_LONG'
ArcGIS Field object 0x1ED579C8: Get Length returned 8
ArcGIS Field object 0x1ED579C8: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57B00>
ArcGIS Field object 0x1ED57B00: Get Type returned 'Double'
ArcGIS Field object 0x1ED57B00: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57B00: Get Name returned 'Bathy'
ArcGIS Field object 0x1ED57B00: Get Name returned 'Bathy'
ArcGIS Field object 0x1ED57B00: Get Name returned 'Bathy'
ArcGIS Field object 0x1ED57B00: Get Length returned 8
ArcGIS Field object 0x1ED57B00: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57488>
ArcGIS Field object 0x1ED57488: Get Type returned 'Date'
ArcGIS Field object 0x1ED57488: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57488: Get Name returned 'DATE'
ArcGIS Field object 0x1ED57488: Get Name returned 'DATE'
ArcGIS Field object 0x1ED57488: Get Name returned 'DATE'
ArcGIS Field object 0x1ED57488: Get Length returned 8
ArcGIS Field object 0x1ED57488: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED572D8>
ArcGIS Field object 0x1ED572D8: Get Type returned 'Double'
ArcGIS Field object 0x1ED572D8: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED572D8: Get Name returned 'SST_8day'
ArcGIS Field object 0x1ED572D8: Get Name returned 'SST_8day'
ArcGIS Field object 0x1ED572D8: Get Name returned 'SST_8day'
ArcGIS Field object 0x1ED572D8: Get Length returned 8
ArcGIS Field object 0x1ED572D8: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57560>
ArcGIS Field object 0x1ED57560: Get Type returned 'Double'
ArcGIS Field object 0x1ED57560: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57560: Get Name returned 'Chla_8day_SW'
ArcGIS Field object 0x1ED57560: Get Name returned 'Chla_8day_SW'
ArcGIS Field object 0x1ED57560: Get Name returned 'Chla_8day_SW'
ArcGIS Field object 0x1ED57560: Get Length returned 8
ArcGIS Field object 0x1ED57560: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57320>
ArcGIS Field object 0x1ED57320: Get Type returned 'Double'
ArcGIS Field object 0x1ED57320: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57320: Get Name returned 'Slope'
ArcGIS Field object 0x1ED57320: Get Name returned 'Slope'
ArcGIS Field object 0x1ED57320: Get Name returned 'Slope'
ArcGIS Field object 0x1ED57320: Get Length returned 8
ArcGIS Field object 0x1ED57320: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57F38>
ArcGIS Field object 0x1ED57F38: Get Type returned 'Double'
ArcGIS Field object 0x1ED57F38: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57F38: Get Name returned 'Dist2Shr'
ArcGIS Field object 0x1ED57F38: Get Name returned 'Dist2Shr'
ArcGIS Field object 0x1ED57F38: Get Name returned 'Dist2Shr'
ArcGIS Field object 0x1ED57F38: Get Length returned 8
ArcGIS Field object 0x1ED57F38: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED578C0>
ArcGIS Field object 0x1ED578C0: Get Type returned 'Double'
ArcGIS Field object 0x1ED578C0: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED578C0: Get Name returned 'Bathy2'
ArcGIS Field object 0x1ED578C0: Get Name returned 'Bathy2'
ArcGIS Field object 0x1ED578C0: Get Name returned 'Bathy2'
ArcGIS Field object 0x1ED578C0: Get Length returned 8
ArcGIS Field object 0x1ED578C0: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED570F8>
ArcGIS Field object 0x1ED570F8: Get Type returned 'SmallInteger'
ArcGIS Field object 0x1ED570F8: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED570F8: Get Name returned 'month_1'
ArcGIS Field object 0x1ED570F8: Get Name returned 'month_1'
ArcGIS Field object 0x1ED570F8: Get Name returned 'month_1'
ArcGIS Field object 0x1ED570F8: Get Length returned 2
ArcGIS Field object 0x1ED570F8: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED572F0>
ArcGIS Field object 0x1ED572F0: Get Type returned 'SmallInteger'
ArcGIS Field object 0x1ED572F0: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED572F0: Get Name returned 'Blacktip_Pres'
ArcGIS Field object 0x1ED572F0: Get Name returned 'Blacktip_Pres'
ArcGIS Field object 0x1ED572F0: Get Name returned 'Blacktip_Pres'
ArcGIS Field object 0x1ED572F0: Get Length returned 2
ArcGIS Field object 0x1ED572F0: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57CB0>
ArcGIS Field object 0x1ED57CB0: Get Type returned 'SmallInteger'
ArcGIS Field object 0x1ED57CB0: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57CB0: Get Name returned 'isevaldata_1'
ArcGIS Field object 0x1ED57CB0: Get Name returned 'isevaldata_1'
ArcGIS Field object 0x1ED57CB0: Get Name returned 'isevaldata_1'
ArcGIS Field object 0x1ED57CB0: Get Length returned 2
ArcGIS Field object 0x1ED57CB0: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57E78>
ArcGIS Field object 0x1ED57E78: Get Type returned 'SmallInteger'
ArcGIS Field object 0x1ED57E78: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57E78: Get Name returned 'TestData'
ArcGIS Field object 0x1ED57E78: Get Name returned 'TestData'
ArcGIS Field object 0x1ED57E78: Get Name returned 'TestData'
ArcGIS Field object 0x1ED57E78: Get Length returned 2
ArcGIS Field object 0x1ED57E78: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned <geoprocessing describe field
object object at 0x1ED57F80>
ArcGIS Field object 0x1ED57F80: Get Type returned 'Double'
ArcGIS Field object 0x1ED57F80: Get IsNullable returned 'TRUE'
ArcGIS Field object 0x1ED57F80: Get Name returned 'bll_glm'
ArcGIS Field object 0x1ED57F80: Get Name returned 'bll_glm'
ArcGIS Field object 0x1ED57F80: Get Name returned 'bll_glm'
ArcGIS Field object 0x1ED57F80: Get Length returned 8
ArcGIS Field object 0x1ED57F80: Get Precision returned 0
ArcGIS Fields object 0x1ED57548: Invoking next(*args=())...
ArcGIS Fields object 0x1ED57548: next returned None
ArcGISTable 0x1EC4FED0: Retrieved lazy properties of ArcGIS FeatureLayer
"bll_blacktip" of FeatureClass
"Z:\sharks\blacktip_bll\blacktip_bll.gdb\bll_blacktip":
ArcGISDataType=FeatureLayer, ArcGISPhysicalDataType=FeatureClass,
MaxStringLength=None, HasOID=True, OIDFieldName=u'OBJECTID',
GeometryType=u'Point', GeometryFieldName=u'Shape',
SpatialReference=u'PROJCS["North_America_Lambert_Conformal_Conic",GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.017453292519943295]],PROJECTION["Lambert_Conformal_Conic"],PARAMETER["False_Easting",0.0],PARAMETER["False_Northing",0.0],PARAMETER["Central_Meridian",-79.5],PARAMETER["Standard_Parallel_1",27.5],PARAMETER["Standard_Parallel_2",41.5],PARAMETER["Latitude_Of_Origin",20.0],UNIT["Meter",1.0],PARAMETER["scale_factor",1.0]]'.
ArcGISTable 0x1EC4FED0: Field 0: Name=OBJECTID, DataType=oid, Length=4,
Precision=0, IsNullable=False, IsSettable=False.
ArcGISTable 0x1EC4FED0: Field 1: Name=Shape, DataType=geometry, Length=0,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 2: Name=DD_LAT, DataType=float64, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 3: Name=DD_LONG, DataType=float64, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 4: Name=Bathy, DataType=float64, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 5: Name=DATE, DataType=datetime, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 6: Name=SST_8day, DataType=float64, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 7: Name=Chla_8day_SW, DataType=float64,
Length=8, Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 8: Name=Slope, DataType=float64, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 9: Name=Dist2Shr, DataType=float64, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 10: Name=Bathy2, DataType=float64, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 11: Name=month_1, DataType=int16, Length=2,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 12: Name=Blacktip_Pres, DataType=int16,
Length=2, Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 13: Name=isevaldata_1, DataType=int16,
Length=2, Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 14: Name=TestData, DataType=int16, Length=2,
Precision=0, IsNullable=True, IsSettable=True.
ArcGISTable 0x1EC4FED0: Field 15: Name=bll_glm, DataType=float64, Length=8,
Precision=0, IsNullable=True, IsSettable=True.
R interpreter: Evaluating statements: PrepareForPrediction(model, model$data,
NULL)
R interpreter: The last statement returned [u'Blacktip_Pres',
[u'Chla_8day_SW', u'Slope', u'Dist2Shr', u'Bathy2', u'SST_8day', u'Month_1'],
0, 1, True, False, {'Bathy2': -677.0, 'Chla_8day_SW': 0.11578000336885452,
'Dist2Shr': 0.0, 'Month_1': 1, 'SST_8day': 17.475000381469727, 'Slope': 0.0},
{'Bathy2': -2.0, 'Chla_8day_SW': 16.118820190429688, 'Dist2Shr':
157651.484375, 'Month_1': 11, 'SST_8day': 30.075000762939453, 'Slope':
1.4985077381134033}, u'factor(Month_1)', {'factor(Month_1)': [u'1', u'2',
u'3', u'6', u'7', u'8', u'9', u'10', u'11']}, {'factor(Month_1)': False},
{'Bathy2': True, 'Chla_8day_SW': True, 'Dist2Shr': True, 'Month_1': True,
'SST_8day': True, 'Slope': True}, True]
R interpreter: Evaluating statements: if (exists("rastersForPredictors"))
rm("rastersForPredictors")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("model")) rm("model")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("trainingData"))
rm("trainingData")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("rPackage")) rm("rPackage")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("xVar")) rm("xVar")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("yVar")) rm("yVar")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("zVar")) rm("zVar")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("mVar")) rm("mVar")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("coordinateSystem"))
rm("coordinateSystem")
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("f20985982305"))
rm("f20985982305", pos=globalenv())
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("data20985982305"))
rm("data20985982305", pos=globalenv())
R interpreter: The last statement returned None
R interpreter: Evaluating statements: if (exists("fam20985982305"))
rm("fam20985982305", pos=globalenv())
R interpreter: The last statement returned None
ValueError: too many values to unpack
---------- BEGINNING OF DEBUGGING INFORMATION ----------
Traceback (most recent call last):
File
"C:\Python26\ArcGIS10.0\Lib\site-packages\GeoEco\ArcGISToolbox\Scripts\GLMPredictFromArcGISTable.py",
line 5, in <module>
ExecuteMethodFromCommandLineAsArcGISTool('GeoEco.Statistics.Modeling',
'GLM', 'PredictFromArcGISTable')
File "C:\Python26\ArcGIS10.0\lib\site-packages\GeoEco\ArcGISScripts.py",
line 212, in ExecuteMethodFromCommandLineAsArcGISTool
Logger.LogExceptionAsError()
File "<string>", line 1, in <module>
File
"C:\Python26\ArcGIS10.0\lib\site-packages\GeoEco\Statistics\Modeling.py",
line 126, in PredictFromArcGISTable
return _PredictFromArcGISTable('glm', _(u'Fit GLM'), inputModelFile,
inputTable, predictedValuesField, cutoff, where, ignoreOutOfRangeValues,
noDataValue, outputPlotFile, measure1, measure2, colorize, outputSummaryFile,
res, width, height, pointSize, bg, overwriteExisting)
File
"C:\Python26\ArcGIS10.0\lib\site-packages\GeoEco\Statistics\Modeling.py",
line 1332, in _PredictFromArcGISTable
responseVariable, allPredictors, minResponseValue, maxResponseValue,
minFittedPredictorValues, maxFittedPredictorValues, categoricalPredictors,
categoricalPredictorLevels, warnedAboutMissingLevels,
allValuesForPredictorAreNA, allValuesAreNA = r('PrepareForPrediction(model,
%s, %s)' % (trainingData, rPackage))
ValueError: too many values to unpack
Local variables for stack frame: File
"C:\Python26\ArcGIS10.0\lib\site-packages\GeoEco\Statistics\Modeling.py",
line 1480, in _PredictFromArcGISTable:
ArcGISTable = <class 'GeoEco.Datasets.ArcGIS.ArcGISTable'>
bg = u'white'
colorize = True
cutoff = None
dataFrameName = None
fields = []
fitToolName = u'Fit GLM'
gotPointFeatures = True
height = 3.0
ignoreOutOfRangeValues = True
inputModelFile = u'Z:\\sharks\\blacktip_bll\\glm\\blacktip_bll_glm4'
inputTable = u'bll_blacktip'
mColumnName = None
measure1 = u'tpr'
measure2 = u'fpr'
modelType = 'glm'
noDataValue = None
outputPlotFile = u'Z:\\sharks\\blacktip_bll\\glm\\blacktipglm4.emf'
outputSummaryFile = u'Z:\\sharks\\blacktip_bll\\glm\\blacktipglmSS4'
overwriteExisting = True
pointSize = 10.0
predictedValuesField = None
r = <GeoEco.R.RInterpreter object at 0x1EC4FEB0>
rPackage = 'NULL'
res = 1000.0
table = <GeoEco.Datasets.ArcGIS.ArcGISTable object at 0x1EC4FED0>
trainingData = 'model$data'
where = u'"TestData" = 1'
width = 3.0
xColumnName = None
yColumnName = None
zColumnName = None
Global variables for stack frame: File
"C:\Python26\ArcGIS10.0\lib\site-packages\GeoEco\Statistics\Modeling.py",
line 1480, in _PredictFromArcGISTable:
AddArgumentMetadata = <function AddArgumentMetadata at 0x1E17F770>
AddClassMetadata = <function AddClassMetadata at 0x1E17F670>
AddMethodMetadata = <function AddMethodMetadata at 0x1E17F730>
AddModuleMetadata = <function AddModuleMetadata at 0x1E17F630>
AddPropertyMetadata = <function AddPropertyMetadata at 0x1E17F6B0>
AddResultMetadata = <function AddResultMetadata at 0x1E17F7F0>
AnyObjectTypeMetadata = <class 'GeoEco.Types.AnyObjectTypeMetadata'>
ArcGIS91DatabaseConnection = <class
'GeoEco.DatabaseAccess.ArcGIS.ArcGIS91DatabaseConnection'>
ArcGIS91SelectCursor = <class
'GeoEco.DatabaseAccess.ArcGIS.ArcGIS91SelectCursor'>
ArcGISDependency = <class 'GeoEco.ArcGIS.ArcGISDependency'>
ArcGISExtensionDependency = <class
'GeoEco.ArcGIS.ArcGISExtensionDependency'>
ArcGISFeatureClassTypeMetadata = <class
'GeoEco.Types.ArcGISFeatureClassTypeMetadata'>
ArcGISFeatureLayerTypeMetadata = <class
'GeoEco.Types.ArcGISFeatureLayerTypeMetadata'>
ArcGISFieldTypeMetadata = <class 'GeoEco.Types.ArcGISFieldTypeMetadata'>
ArcGISGeoDatasetTypeMetadata = <class
'GeoEco.Types.ArcGISGeoDatasetTypeMetadata'>
ArcGISRaster = <class 'GeoEco.DataManagement.ArcGISRasters.ArcGISRaster'>
ArcGISRasterCatalogTypeMetadata = <class
'GeoEco.Types.ArcGISRasterCatalogTypeMetadata'>
ArcGISRasterLayerTypeMetadata = <class
'GeoEco.Types.ArcGISRasterLayerTypeMetadata'>
ArcGISRasterTypeMetadata = <class 'GeoEco.Types.ArcGISRasterTypeMetadata'>
ArcGISTableTypeMetadata = <class 'GeoEco.Types.ArcGISTableTypeMetadata'>
ArcGISTableViewTypeMetadata = <class
'GeoEco.Types.ArcGISTableViewTypeMetadata'>
ArcGISWorkspaceTypeMetadata = <class
'GeoEco.Types.ArcGISWorkspaceTypeMetadata'>
ArgumentMetadata = <class 'GeoEco.Metadata.ArgumentMetadata'>
BooleanTypeMetadata = <class 'GeoEco.Types.BooleanTypeMetadata'>
ClassInstanceTypeMetadata = <class 'GeoEco.Types.ClassInstanceTypeMetadata'>
ClassMetadata = <class 'GeoEco.Metadata.ClassMetadata'>
ClassOrClassInstanceTypeMetadata = <class
'GeoEco.Types.ClassOrClassInstanceTypeMetadata'>
ClassTypeMetadata = <class 'GeoEco.Types.ClassTypeMetadata'>
CoordinateSystemTypeMetadata = <class
'GeoEco.Types.CoordinateSystemTypeMetadata'>
CopyArgumentMetadata = <function CopyArgumentMetadata at 0x1E17F830>
CopyPropertyMetadata = <function CopyPropertyMetadata at 0x1E17F6F0>
CopyResultMetadata = <function CopyResultMetadata at 0x1E17F870>
DateTimeTypeMetadata = <class 'GeoEco.Types.DateTimeTypeMetadata'>
DictionaryTypeMetadata = <class 'GeoEco.Types.DictionaryTypeMetadata'>
DirectoryTypeMetadata = <class 'GeoEco.Types.DirectoryTypeMetadata'>
DynamicDocString = <class 'GeoEco.DynamicDocString.DynamicDocString'>
EnvelopeTypeMetadata = <class 'GeoEco.Types.EnvelopeTypeMetadata'>
File = <class 'GeoEco.DataManagement.Files.File'>
FileTypeMetadata = <class 'GeoEco.Types.FileTypeMetadata'>
FloatTypeMetadata = <class 'GeoEco.Types.FloatTypeMetadata'>
GAM = <class 'GeoEco.Statistics.Modeling.GAM'>
GLM = <class 'GeoEco.Statistics.Modeling.GLM'>
GeoprocessorManager = <class 'GeoEco.ArcGIS.GeoprocessorManager'>
IntegerTypeMetadata = <class 'GeoEco.Types.IntegerTypeMetadata'>
LinearMixedModel = <class 'GeoEco.Statistics.Modeling.LinearMixedModel'>
LinearUnitTypeMetadata = <class 'GeoEco.Types.LinearUnitTypeMetadata'>
ListTypeMetadata = <class 'GeoEco.Types.ListTypeMetadata'>
Logger = <class 'GeoEco.Logging.Logger'>
MapAlgebraExpressionTypeMetadata = <class
'GeoEco.Types.MapAlgebraExpressionTypeMetadata'>
Metadata = <class 'GeoEco.Metadata.Metadata'>
MethodMetadata = <class 'GeoEco.Metadata.MethodMetadata'>
ModelEvaluation = <class 'GeoEco.Statistics.Modeling.ModelEvaluation'>
ModuleMetadata = <class 'GeoEco.Metadata.ModuleMetadata'>
NoneTypeMetadata = <class 'GeoEco.Types.NoneTypeMetadata'>
NumPyArrayTypeMetadata = <class 'GeoEco.Types.NumPyArrayTypeMetadata'>
PointTypeMetadata = <class 'GeoEco.Types.PointTypeMetadata'>
PropertyMetadata = <class 'GeoEco.Metadata.PropertyMetadata'>
PythonModuleDependency = <class
'GeoEco.Dependencies.PythonModuleDependency'>
R = <class 'GeoEco.R.R'>
RDependency = <class 'GeoEco.R.RDependency'>
RPackageDependency = <class 'GeoEco.R.RPackageDependency'>
RandomForestModel = <class 'GeoEco.Statistics.Modeling.RandomForestModel'>
ResultMetadata = <class 'GeoEco.Metadata.ResultMetadata'>
SQLWhereClauseTypeMetadata = <class
'GeoEco.Types.SQLWhereClauseTypeMetadata'>
SequenceTypeMetadata = <class 'GeoEco.Types.SequenceTypeMetadata'>
ShapefileTypeMetadata = <class 'GeoEco.Types.ShapefileTypeMetadata'>
SpatialReferenceTypeMetadata = <class
'GeoEco.Types.SpatialReferenceTypeMetadata'>
StoredObjectTypeMetadata = <class 'GeoEco.Types.StoredObjectTypeMetadata'>
TableFieldTypeMetadata = <class 'GeoEco.Types.TableFieldTypeMetadata'>
TemporaryDirectory = <class
'GeoEco.DataManagement.Directories.TemporaryDirectory'>
TextFileTypeMetadata = <class 'GeoEco.Types.TextFileTypeMetadata'>
TreeModel = <class 'GeoEco.Statistics.Modeling.TreeModel'>
TupleTypeMetadata = <class 'GeoEco.Types.TupleTypeMetadata'>
TypeMetadata = <class 'GeoEco.Metadata.TypeMetadata'>
UnicodeStringTypeMetadata = <class 'GeoEco.Types.UnicodeStringTypeMetadata'>
_ = <function _gettext at 0x1DE6F770>
_FormulaDescription = u'The formula must be in the format expected by the R
glm function::\n\n response ~ term1 + term2 + ... + termN\n\nresponse is
the table field that will be modeled as the response\nvariable and the terms
are the table fields that will serve as the\npredictor variables. The field
names are case sensitive. If any field\nused in the formula is NULL for a
given row, that row will not be used\nin fitting the model.\n\nFor example,
if you have a field Presence that indicates the presence\nor absence of a
species (1 or 0) and you want to model it in terms of\nsampled environmental
covariates stored in the SST, ChlDensity, and\nDepth fields, you would use
the formula::\n\n Presence ~ SST + ChlDensity + Depth\n\nBy default, all
terms are treated as continuous variables. To indicate\nthat a term should be
treated as a categorical variable, use the\nfactor function. For example, if
SubstrateType is an integer code that\nshould be treated as categorical::\n\n
Presence ~ SST + ChlDensity + Depth + factor(SubstrateType)\n\nThe model
terms may also use these operators:\n\n* The : operator denotes the
interaction of variables a and b. For\n example: a:b.\n\n* The * operator
denotes "crossing". For example, a*b is identical to\n a+b+a:b.\n\n* The ^
operator denotes crossing to the Nth degree. For example,\n (a+b+c)^2 is
identical to (a+b+c)*(a+b+c) which in turn expands to a\n formula containing
the main effects for a, b and c together with\n their second-order
interactions.\n\n* The %in% operator indicates that the terms on its left are
nested\n within those on the right. For example a + b %in% a expands to
the\n formula a + a:b.\n\n* The - operator (minus) removes the specified
terms, so that\n (a+b+c)^2 - a:b is identical to a + b + c + b:c + a:c. It
can also\n used to remove the intercept term: y ~ x - 1 is a line through
the\n origin. A model with no intercept can be also specified as y ~ x + 0\n
or y ~ 0 + x.\n\nWhile formulae usually involve just variable names, they
can also\ninvolve arithmetic expressions. The formula log(y) ~ a + log(x)
is\nquite legal. When such arithmetic expressions involve operators
which\nare also used symbolically in model formulae, there can be
confusion\nbetween arithmetic and symbolic operator use.\n\nTo avoid this
confusion, the function I() can be used to bracket those\nportions of a model
formula where the operators are used in their\narithmetic sense. For example,
in the formula y ~ a + I(b+c), the term\nb+c is to be interpreted as the sum
of b and c.\n\nPlease see the topics "glm" and "formula" in the R
documentation for\nmore information.'
_LoadDataFrameForModelFitting = <function _LoadDataFrameForModelFitting at
0x1EB1B7B0>
_PredictFromArcGISRasters = <function _PredictFromArcGISRasters at
0x1EB37570>
_PredictFromArcGISTable = <function _PredictFromArcGISTable at 0x1EB37530>
_PreparePredictorRasters = <function _PreparePredictorRasters at 0x1EB375B0>
__all__ = ['GLM', 'GAM', 'TreeModel', 'LinearMixedModel',
'RandomForestModel', 'ModelEvaluation']
__doc__ = u'Provides methods for modeling and prediction.\n'
__file__ =
'C:\\Python26\\ArcGIS10.0\\lib\\site-packages\\GeoEco\\Statistics\\Modeling.pyc'
__name__ = 'GeoEco.Statistics.Modeling'
__package__ = 'GeoEco.Statistics'
copy = <module 'copy' from 'C:\Python26\ArcGIS10.0\Lib\copy.pyc'>
glob = <module 'glob' from 'C:\Python26\ArcGIS10.0\Lib\glob.pyc'>
os = <module 'os' from 'C:\Python26\ArcGIS10.0\Lib\os.pyc'>
random = <module 'random' from 'C:\Python26\ArcGIS10.0\Lib\random.pyc'>
sys = <module 'sys' (built-in)>
types = <module 'types' from 'C:\Python26\ArcGIS10.0\Lib\types.pyc'>
Enviornment variables:
AGSDESKTOPJAVA = 'C:\\Program Files\\ArcGIS\\Desktop10.0\\'
ALLUSERSPROFILE = 'C:\\Documents and Settings\\All Users'
APPDATA = 'C:\\Documents and Settings\\ehh10\\Application Data'
ARCHOME = 'C:\\Program Files\\ArcGIS\\Desktop10.0\\'
ARCHOME_USER = 'C:\\Program Files\\ArcGIS\\Desktop10.0\\'
ARCPWD = 'Z:\\sharks\\blacktip_bll'
CLIENTNAME = 'COMPUTERLIZA'
CLUSTERLOG = 'C:\\WINDOWS\\Cluster\\cluster.log'
COMMONPROGRAMFILES = 'C:\\Program Files\\Common Files'
COMPUTERNAME = 'ESTEBAN'
COMSPEC = 'C:\\WINDOWS\\system32\\cmd.exe'
DSETPATH = 'C:\\Program Files\\Dell\\DSET\\'
FP_NO_HOST_CHECK = 'NO'
HOMEDRIVE = 'C:'
HOMEPATH = '\\Documents and Settings\\ehh10'
LOGONSERVER = '\\\\RIDLEY'
NUMBER_OF_PROCESSORS = '4'
OS = 'Windows_NT'
PATH = 'C:\\Program Files\\Windows Resource
Kits\\Tools\\;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\Program
Files\\MATLAB\\MATLAB Component
Runtime\\v77\\runtime\\win32;C:\\Python25\\Scripts;C:\\Program
Files\\TortoiseSVN\\bin;E:\\MATLAB\\R2012a\\runtime\\win32;E:\\MATLAB\\R2012a\\bin;C:\\Program
Files\\Microsoft SQL Server\\90\\Tools\\binn\\;C:\\Program
Files\\R\\R-2.15.1\\bin\\i386;C:\\Program Files\\R\\R-2.15.1\\modules\\i386'
PATHEXT = '.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH'
PROCESSOR_ARCHITECTURE = 'x86'
PROCESSOR_IDENTIFIER = 'x86 Family 6 Model 23 Stepping 6, GenuineIntel'
PROCESSOR_LEVEL = '6'
PROCESSOR_REVISION = '1706'
PROGRAMFILES = 'C:\\Program Files'
RPY_RHOME = 'C:\\Program Files\\R\\R-2.15.1'
RPY_RUSER = 'C:\\Documents and Settings\\ehh10\\My Documents'
RPY_RVER = '2151'
RPY_RVERSION = '2.15.1'
R_HOME = 'C:\\Program Files\\R\\R-2.15.1'
SESSIONNAME = 'RDP-Tcp#10'
SYSTEMDRIVE = 'C:'
SYSTEMROOT = 'C:\\WINDOWS'
TEMP = 'C:\\DOCUME~1\\ehh10\\LOCALS~1\\Temp\\3'
TMP = 'C:\\DOCUME~1\\ehh10\\LOCALS~1\\Temp\\3'
USERDOMAIN = 'MGEL'
USERNAME = 'ehh10'
USERPROFILE = 'C:\\Documents and Settings\\ehh10'
WINDIR = 'C:\\WINDOWS'
Other variables:
GeoEco.__version__ = u'0.8a43'
sys.argv =
['C:\\Python26\\ArcGIS10.0\\Lib\\site-packages\\GeoEco\\ArcGISToolbox\\Scripts\\GLMPredictFromArcGISTable.py',
'Z:\\sharks\\blacktip_bll\\glm\\blacktip_bll_glm4', 'bll_blacktip', '#',
'#', '"TestData" = 1', 'true', '#',
'Z:\\sharks\\blacktip_bll\\glm\\blacktipglm4.emf', 'tpr', 'fpr', 'true',
'Z:\\sharks\\blacktip_bll\\glm\\blacktipglmSS4', '1000', '3000', '3000',
'10', 'white', 'bll_blacktip', '0.491230300279']
sys.version = 2.6.5 (r265:79096, Mar 19 2010, 21:48:26) [MSC v.1500 32 bit
(Intel)]
sys.version_info = (2, 6, 5, 'final', 0)
sys.platform = win32
sys.getwindowsversion() = (5, 2, 3790, 2, 'Service Pack 2')
os.getcwd() = Z:\sharks\blacktip_bll
sys.path =
['C:\\Python26\\ArcGIS10.0\\Lib\\site-packages\\GeoEco\\ArcGISToolbox\\Scripts',
'C:\\WINDOWS\\system32\\python26.zip', u'C:\\Program
Files\\ArcGIS\\Desktop10.0\\arcpy', 'C:\\Python26\\ArcGIS10.0\\Lib',
'C:\\Python26\\ArcGIS10.0\\DLLs', 'C:\\Python26\\ArcGIS10.0\\Lib\\lib-tk',
'C:\\Program Files\\ArcGIS\\Desktop10.0\\ArcToolbox\\Toolboxes', 'C:\\Program
Files\\ArcGIS\\Desktop10.0\\bin', 'C:\\Python26\\ArcGIS10.0',
'C:\\Python26\\ArcGIS10.0\\lib\\site-packages', 'C:\\Program
Files\\ArcGIS\\Desktop10.0\\arcpy', 'C:\\Program
Files\\ArcGIS\\Desktop10.0\\ArcToolbox\\Scripts',
'C:\\Python26\\ArcGIS10.0\\lib\\site-packages\\win32',
'C:\\Python26\\ArcGIS10.0\\lib\\site-packages\\win32\\lib',
'C:\\Python26\\ArcGIS10.0\\lib\\site-packages\\Pythonwin']
Loaded modules: ConfigParser, GeoEco, GeoEco.ArcGIS, GeoEco.ArcGISScripts,
GeoEco.AssimilatedModules, GeoEco.AssimilatedModules.GeoEco,
GeoEco.AssimilatedModules.osgeo, GeoEco.AssimilatedModules.osgeo.imp,
GeoEco.AssimilatedModules.osgeo.os, GeoEco.AssimilatedModules.osgeo.osr,
GeoEco.AssimilatedModules.osgeo.sys, GeoEco.AssimilatedModules.rpy,
GeoEco.AssimilatedModules.rpy.UserDict,
GeoEco.AssimilatedModules.rpy.__future__,
GeoEco.AssimilatedModules.rpy._rpy2151, GeoEco.AssimilatedModules.rpy.atexit,
GeoEco.AssimilatedModules.rpy.codecs, GeoEco.AssimilatedModules.rpy.glob,
GeoEco.AssimilatedModules.rpy.numpy, GeoEco.AssimilatedModules.rpy.os,
GeoEco.AssimilatedModules.rpy.re, GeoEco.AssimilatedModules.rpy.rpy,
GeoEco.AssimilatedModules.rpy.rpy_io,
GeoEco.AssimilatedModules.rpy.rpy_options,
GeoEco.AssimilatedModules.rpy.rpy_tools,
GeoEco.AssimilatedModules.rpy.rpy_version, GeoEco.AssimilatedModules.rpy.sys,
GeoEco.AssimilatedModules.rpy.threading, GeoEco.AssimilatedModules.rpy.time,
GeoEco.AssimilatedModules.rpy.win32api,
GeoEco.AssimilatedModules.rpy.win32com,
GeoEco.AssimilatedModules.rpy.win32con, GeoEco.BatchProcessing,
GeoEco.DataManagement, GeoEco.DataManagement.ArcGISRasters,
GeoEco.DataManagement.BinaryRasters, GeoEco.DataManagement.Directories,
GeoEco.DataManagement.Fields, GeoEco.DataManagement.Files,
GeoEco.DataManagement.GeoEco, GeoEco.DataManagement.Queue,
GeoEco.DataManagement.array, GeoEco.DataManagement.bz2,
GeoEco.DataManagement.datetime, GeoEco.DataManagement.errno,
GeoEco.DataManagement.glob, GeoEco.DataManagement.gzip,
GeoEco.DataManagement.inspect, GeoEco.DataManagement.locale,
GeoEco.DataManagement.math, GeoEco.DataManagement.operator,
GeoEco.DataManagement.os, GeoEco.DataManagement.random,
GeoEco.DataManagement.re, GeoEco.DataManagement.shutil,
GeoEco.DataManagement.sys, GeoEco.DataManagement.tarfile,
GeoEco.DataManagement.tempfile, GeoEco.DataManagement.threading,
GeoEco.DataManagement.time, GeoEco.DataManagement.types,
GeoEco.DataManagement.urlparse, GeoEco.DataManagement.zipfile,
GeoEco.DatabaseAccess, GeoEco.DatabaseAccess.ArcGIS,
GeoEco.DatabaseAccess.GeoEco, GeoEco.DatabaseAccess.InMemory,
GeoEco.DatabaseAccess.datetime, GeoEco.DatabaseAccess.logging,
GeoEco.DatabaseAccess.os, GeoEco.DatabaseAccess.repr,
GeoEco.DatabaseAccess.time, GeoEco.DatabaseAccess.types, GeoEco.Datasets,
GeoEco.Datasets.ArcGIS, GeoEco.Datasets.Collections, GeoEco.Datasets.GDAL,
GeoEco.Datasets.GeoEco, GeoEco.Datasets.atexit, GeoEco.Datasets.bisect,
GeoEco.Datasets.codecs, GeoEco.Datasets.copy, GeoEco.Datasets.datetime,
GeoEco.Datasets.locale, GeoEco.Datasets.logging, GeoEco.Datasets.math,
GeoEco.Datasets.os, GeoEco.Datasets.re, GeoEco.Datasets.shutil,
GeoEco.Datasets.types, GeoEco.Datasets.weakref, GeoEco.Dependencies,
GeoEco.DynamicDocString, GeoEco.Exceptions, GeoEco.GeoArray, GeoEco.GeoEco,
GeoEco.Internationalization, GeoEco.Logging, GeoEco.Metadata,
GeoEco.MetadataUtils, GeoEco.R, GeoEco.Statistics, GeoEco.Statistics.GeoEco,
GeoEco.Statistics.Modeling, GeoEco.Statistics.copy, GeoEco.Statistics.glob,
GeoEco.Statistics.os, GeoEco.Statistics.random, GeoEco.Statistics.sys,
GeoEco.Statistics.types, GeoEco.StringIO, GeoEco.Types,
GeoEco.arcgisscripting, GeoEco.atexit, GeoEco.bisect, GeoEco.codecs,
GeoEco.copy, GeoEco.datetime, GeoEco.gc, GeoEco.gettext, GeoEco.glob,
GeoEco.inspect, GeoEco.locale, GeoEco.logging, GeoEco.math, GeoEco.new,
GeoEco.numpy, GeoEco.operator, GeoEco.os, GeoEco.platform, GeoEco.pywintypes,
GeoEco.random, GeoEco.re, GeoEco.repr, GeoEco.sys, GeoEco.time,
GeoEco.traceback, GeoEco.types, GeoEco.weakref, GeoEco.win32api,
GeoEco.win32con, GeoEco.xml, Queue, SocketServer, StringIO, UserDict,
_GeoEcoArcGISHelper, __builtin__, __future__, __main__, _abcoll, _bisect,
_codecs, _collections, _ctypes, _functools, _gdal, _heapq, _locale, _osr,
_random, _socket, _sre, _ssl, _struct, _warnings, _weakref, _win32sysloader,
abc, arcgis, arcgisscripting, arcpy, arcpy._base, arcpy._ga, arcpy._graph,
arcpy._management, arcpy._mapping, arcpy.analysis, arcpy.arc,
arcpy.arcgisscripting, arcpy.arcobjects, arcpy.arcobjects._base,
arcpy.arcobjects.arcobjectconversion, arcpy.arcobjects.arcobjects,
arcpy.arcobjects.collections, arcpy.arcobjects.functools,
arcpy.arcobjects.geometries, arcpy.arcobjects.itertools,
arcpy.arcobjects.mixins, arcpy.arcobjects.operator, arcpy.arcobjects.os,
arcpy.arcobjects.types, arcpy.arcobjects.weakref, arcpy.arcobjects.xml,
arcpy.arcpy, arcpy.cartography, arcpy.conversion, arcpy.ddd, arcpy.edit,
arcpy.fabric, arcpy.functools, arcpy.ga, arcpy.geocoding,
arcpy.geoprocessing, arcpy.geoprocessing._base,
arcpy.geoprocessing.arcgisscripting, arcpy.geoprocessing.functools,
arcpy.geoprocessing.thread, arcpy.geoprocessing.weakref, arcpy.inspect,
arcpy.interop, arcpy.logging, arcpy.lr, arcpy.management, arcpy.mapping,
arcpy.md, arcpy.na, arcpy.os, arcpy.pprint, arcpy.re, arcpy.sa,
arcpy.sa.CompoundParameter, arcpy.sa.Functions, arcpy.sa.ParameterClasses,
arcpy.sa.Utils, arcpy.sa.arcgisscripting, arcpy.sa.arcpy, arcpy.sa.datetime,
arcpy.sa.types, arcpy.schematics, arcpy.server, arcpy.stats, arcpy.sys,
arcpy.ta, arcpy.toolbox, arcpy.types, arcpy.utils, array, atexit, binascii,
bisect, bz2, cPickle, cStringIO, codecs, collections, copy, copy_reg, ctypes,
ctypes._ctypes, ctypes._endian, ctypes.ctypes, ctypes.os, ctypes.struct,
ctypes.sys, datetime, dis, encodings, encodings.__builtin__,
encodings.aliases, encodings.codecs, encodings.cp1252, encodings.encodings,
encodings.utf_8, errno, exceptions, fnmatch, functools, gc, genericpath,
gettext, glob, gzip, heapq, imp, inspect, itertools, keyword, linecache,
locale, logging, logging.ConfigParser, logging.SocketServer, logging.atexit,
logging.cPickle, logging.cStringIO, logging.codecs, logging.config,
logging.handlers, logging.logging, logging.os, logging.re, logging.socket,
logging.stat, logging.string, logging.struct, logging.sys, logging.thread,
logging.threading, logging.time, logging.traceback, logging.types, math, new,
nt, ntpath, numpy, numpy.__builtin__, numpy.__config__, numpy._import_tools,
numpy.add_newdocs, numpy.compat, numpy.compat._inspect, numpy.compat.types,
numpy.core, numpy.core.__builtin__, numpy.core._dotblas,
numpy.core._internal, numpy.core._sort, numpy.core.arrayprint,
numpy.core.cPickle, numpy.core.ctypes, numpy.core.defchararray,
numpy.core.fromnumeric, numpy.core.function_base, numpy.core.getlimits,
numpy.core.info, numpy.core.machar, numpy.core.memmap, numpy.core.multiarray,
numpy.core.numeric, numpy.core.numerictypes, numpy.core.numpy, numpy.core.os,
numpy.core.re, numpy.core.records, numpy.core.scalarmath,
numpy.core.shape_base, numpy.core.sys, numpy.core.types, numpy.core.umath,
numpy.core.warnings, numpy.ctypes, numpy.ctypeslib, numpy.fft,
numpy.fft.fftpack, numpy.fft.fftpack_lite, numpy.fft.helper, numpy.fft.info,
numpy.fft.numpy, numpy.fft.types, numpy.lib, numpy.lib.__builtin__,
numpy.lib.__future__, numpy.lib._compiled_base, numpy.lib._datasource,
numpy.lib._iotools, numpy.lib.arraysetops, numpy.lib.arrayterator,
numpy.lib.cPickle, numpy.lib.cStringIO, numpy.lib.financial,
numpy.lib.format, numpy.lib.function_base, numpy.lib.index_tricks,
numpy.lib.info, numpy.lib.io, numpy.lib.itertools, numpy.lib.math,
numpy.lib.numpy, numpy.lib.operator, numpy.lib.os, numpy.lib.polynomial,
numpy.lib.re, numpy.lib.scimath, numpy.lib.shape_base, numpy.lib.shutil,
numpy.lib.stride_tricks, numpy.lib.sys, numpy.lib.twodim_base,
numpy.lib.type_check, numpy.lib.types, numpy.lib.ufunclike, numpy.lib.utils,
numpy.lib.warnings, numpy.linalg, numpy.linalg.info,
numpy.linalg.lapack_lite, numpy.linalg.linalg, numpy.linalg.numpy, numpy.ma,
numpy.ma.cPickle, numpy.ma.core, numpy.ma.extras, numpy.ma.itertools,
numpy.ma.numpy, numpy.ma.operator, numpy.ma.warnings, numpy.matrixlib,
numpy.matrixlib.defmatrix, numpy.matrixlib.numpy, numpy.matrixlib.sys,
numpy.numpy, numpy.os, numpy.polynomial, numpy.polynomial.__future__,
numpy.polynomial.chebyshev, numpy.polynomial.exceptions,
numpy.polynomial.numpy, numpy.polynomial.polynomial,
numpy.polynomial.polytemplate, numpy.polynomial.polyutils,
numpy.polynomial.string, numpy.polynomial.warnings, numpy.random,
numpy.random.info, numpy.random.mtrand, numpy.random.numpy, numpy.sys,
numpy.testing, numpy.testing.decorators, numpy.testing.nosetester,
numpy.testing.numpy, numpy.testing.numpytest, numpy.testing.operator,
numpy.testing.os, numpy.testing.re, numpy.testing.sys,
numpy.testing.traceback, numpy.testing.types, numpy.testing.unittest,
numpy.testing.utils, numpy.testing.warnings, numpy.version, opcode, operator,
os, os.path, platform, posixpath, pprint, pythoncom, pythoncom.__univgw,
pywintypes, random, re, repr, select, shutil, site, socket, sre_compile,
sre_constants, sre_parse, stat, string, strop, struct, swig_runtime_data4,
sys, tarfile, tempfile, thread, threading, time, token, tokenize, traceback,
types, unittest, urlparse, warnings, weakref, win32api, win32com,
win32com.gen_py, win32com.gen_py.win32com, win32com.os, win32com.pythoncom,
win32com.shell, win32com.shell.shell, win32com.shell.shellcon,
win32com.shell.win32com, win32com.sys, win32com.win32api, win32com.win32com,
win32con, xml, xml.dom, xml.dom.domreg, xml.dom.minicompat, xml.dom.xml,
xml.etree, xml.etree.ElementPath, xml.etree.ElementTree, xml.etree.re,
xml.etree.string, xml.etree.sys, zipfile, zipimport, zlib
---------- END OF DEBUGGING INFORMATION ----------
GeoEco is releasing its reference to ArcGIS Geoprocessor object 0x1EAE22A8
and will no longer use it for ArcGIS operations.
<type 'exceptions.ValueError'>: too many values to unpack
Failed to execute (Predict GLM From Table).
Failed at Fri Sep 21 17:17:49 2012 (Elapsed Time: 2.00 seconds)
Archives powered by MHonArc.