mget-help - Re: [mget-help] Resampling error-MGET

Subject: Marine Geospatial Ecology Tools (MGET) help

Text archives

Re: [mget-help] Resampling error-MGET

From: Sreenath Ramanathan <>
To:
Cc: Jason Roberts <>
Subject: Re: [mget-help] Resampling error-MGET
Date: Wed, 1 Aug 2018 05:36:03 +0530
Authentication-results: oit.duke.edu; spf=pass

Dear Dr. Jason,

I have made 3 rasters with WGS_1984_World_Mercator Projection and having common cell sizes. I could get an error again in the step 2 and I couldn't understand where I have missed this time. As a trial I have included only 4 patches. I am sending the simulation directory (a bit heavy) in gdrive for your reference.

https://drive.google.com/drive/folders/1ulKm8CL4BEk6-DL9wAr00r8fA84efwDa?usp=sharing

Thank you

Executing: LarvalDispersalLoadHYCOMGLBa0084DEquatorialCurrentsIntoSimulation C:\Users\DELL\Documents\ArcGIS\Simulation5 20-09-2016 24-09-2016 0 # Degrees false CUBIC Del2a 60 120 C:\Users\DELL\Documents\ArcGIS\sim3Cache

Start Time: Wed Aug 01 04:37:41 2018

Running script LarvalDispersalLoadHYCOMGLBa0084DEquatorialCurrentsIntoSimulation...

Querying time and depth slices of u grid of the equatorial region of the HYCOM + NCODA Global 1/12 degree Analysis (GLBa0.08), clipped to indices tMin = 2924, tMax = 2928, zMax = 0.

Query complete: 0:00:00 elapsed, 5 datasets found, 0:00:00.001000 per dataset.

Importing 5 datasets into ArcGIS Folder C:\Users\DELL\Documents\ArcGIS\Simulation5\OriginalHYCOMCurrents with mode "add".

Checking for existing destination datasets.

Finished checking: 0:00:00 elapsed, 5 datasets checked, 0:00:00.000600 per dataset.

0 destination datasets already exist. Importing 5 datasets.

Import in progress: 0:01:12 elapsed, 1 datasets imported, 0:01:12.175272 per dataset, 4 remaining, estimated completion time: unknown; more progress needed.

Import in progress: 0:02:43 elapsed, 3 datasets imported, 0:00:54.535062 per dataset, 2 remaining, estimated completion time: 04:41:22.

Import complete: 0:04:28 elapsed, 5 datasets imported, 0:00:53.791800 per dataset.

Querying time and depth slices of v grid of the equatorial region of the HYCOM + NCODA Global 1/12 degree Analysis (GLBa0.08), clipped to indices tMin = 2924, tMax = 2928, zMax = 0.

Query complete: 0:00:00 elapsed, 5 datasets found, 0:00:00 per dataset.

Importing 5 datasets into ArcGIS Folder C:\Users\DELL\Documents\ArcGIS\Simulation5\OriginalHYCOMCurrents with mode "add".

Checking for existing destination datasets.

Finished checking: 0:00:00 elapsed, 5 datasets checked, 0:00:00 per dataset.

0 destination datasets already exist. Importing 5 datasets.

Import in progress: 0:01:34 elapsed, 2 datasets imported, 0:00:47.096269 per dataset, 3 remaining, estimated completion time: 04:46:21.

Import complete: 0:03:52 elapsed, 5 datasets imported, 0:00:46.444000 per dataset.

Finding ArcGIS rasters and inserting rows into table "work": workspace="C:\Users\DELL\Documents\ArcGIS\Simulation5\OriginalHYCOMCurrents", wildcard="*.img", searchTree=True, rasterType="None"

Finished inserting rows: 0:00:00 elapsed, 20 rows inserted, 0:00:00.017500 per row.

Updating field "outputRaster" in table "work"...

Update complete: 0:00:00 elapsed, 20 rows updated, 0 deleted, 0 unchanged, 0:00:00 per row.

Querying the table "work" to build lists of inputs and outputs...

Query complete: 0:00:00 elapsed, 20 rows retrieved, 0:00:00 per row.

Checking for existing outputs for each input...

Finished checking for existing outputs: 0:00:00 elapsed, 20 inputs checked, 0:00:00.029350 per input.

Processing 20 inputs...

RuntimeError: ??? Error using ==> mldivide

python.exe: RuntimeError: ??? Error using ==> mldivide

python.exe: Out of memory. Type HELP MEMORY for your options.

python.exe:

python.exe: Error in ==> InpaintNaNs>inpaint_nans at 252

python.exe:

python.exe: Error in ==> InpaintNaNs at 29

RuntimeError: python.exe returned exit code 1, indicating failure.

The following consequences resulted from the original error:

Execution of C:\Python27\ArcGIS10.3\python.exe failed.

Failed script LarvalDispersalLoadHYCOMGLBa0084DEquatorialCurrentsIntoSimulation...

Traceback (most recent call last):

File "C:\Program Files\GeoEco\ArcGISToolbox\Scripts\LarvalDispersalLoadHYCOMGLBa0084DEquatorialCurrentsIntoSimulation.py", line 5, in <module>

ExecuteMethodFromCommandLineAsArcGISTool('GeoEco.Connectivity.LarvalDispersal', 'LarvalDispersal', 'LoadHYCOMGLBa0084DEquatorialCurrentsIntoSimulation')

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\ArcGISScripts.py", line 210, in ExecuteMethodFromCommandLineAsArcGISTool

exec sourceCode in globals(), locals()

File "<string>", line 1, in <module>

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Connectivity\LarvalDispersal.py", line 311, in LoadHYCOMGLBa0084DEquatorialCurrentsIntoSimulation

cls._FinishLoadingCurrents(simulationDirectory, scp, thisCurrentsProduct, u'Center', 86400, hycomCurrentsDirectory, resamplingTechnique, interpolationMethod)

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Connectivity\LarvalDispersal.py", line 402, in _FinishLoadingCurrents

skipExisting=True)

File "<string>", line 7, in FindAndProjectRastersToTemplate

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\BatchProcessing.py", line 2137, in ExecuteFindAndProcessMethod

boundProcessTableMethod(*tuple(argList))

File "<string>", line 5, in ProjectToTemplateTable

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\BatchProcessing.py", line 1271, in ExecuteProcessTableMethod

results = boundProcessListMethod(*tuple(argList))

File "<string>", line 5, in ProjectToTemplateList

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\BatchProcessing.py", line 888, in ExecuteProcessListMethod

result = boundMethod(*tuple(argList))

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\DataManagement\ArcGISRasters.py", line 1892, in ProjectToTemplate

Interpolator.InpaintArcGISRaster(projectedRaster, infilledRaster, interpolationMethod, maxHoleSize, minValue=minValue, maxValue=maxValue)

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\SpatialAnalysis\Interpolation.py", line 726, in InpaintArcGISRaster

ArcGISWorkspace(os.path.dirname(outputRaster), ArcGISRaster, pathCreationExpressions=[os.path.basename(outputRaster)]).ImportDatasets([InpaintedGrid(ArcGISRasterBand.ConstructFromArcGISPath(inputRaster), method, maxHoleSize, xEdgesWrap, minValue, maxValue)], {False: u'Add', True: u'Replace'}[overwriteExisting], reportProgress=False)

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\__init__.py", line 820, in ImportDatasets

self._ImportDatasets(datasets, mode.lower(), reportProgress, options)

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\Collections.py", line 695, in _ImportDatasets

self._ImportDatasetsToPath(pathComponentsForPath[path], datasetsForPath[path], mode, progressReporter, options)

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\ArcGIS.py", line 609, in _ImportDatasetsToPath

self.DatasetType._ImportDatasetsToPath(os.path.join(self.Path, *pathComponents), sourceDatasets, mode, progressReporter, options)

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\ArcGIS.py", line 1128, in _ImportDatasetsToPath

GDALDataset._ImportDatasetsToPath(path, sourceDatasets, mode, None, {'useArcGISSpatialReference': True, 'useUnscaledData': useUnscaledData, 'calculateStatistics': False, 'blockSize': blockSize})

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\GDAL.py", line 1073, in _ImportDatasetsToPath

data = "sourceDatasets[i].Data[rowsCopied:rowsCopied+rowsToCopy," :]

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\__init__.py", line 3671, in __getitem__

return getattr(self._Grid(), self._GetMethod)(key)

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\__init__.py", line 3311, in _GetUnscaledDataAsArray

data, actualNoDataValue = self._ReadNumpyArray(reorderedSliceList)

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\Virtual.py", line 3600, in _ReadNumpyArray

maxRunTime=None)

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\DataManagement\Processes.py", line 575, in ExecuteProgram

process.Execute()

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\DataManagement\Processes.py", line 498, in Execute

Logger.RaiseException(RuntimeError(_(u'%(name)s returned exit code %(code)i, indicating failure.') % {u'name' : programName, u'code' : self.ExitCode}))

File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Logging.py", line 104, in RaiseException

raise exception

RuntimeError: python.exe returned exit code 1, indicating failure.

Failed to execute (LarvalDispersalLoadHYCOMGLBa0084DEquatorialCurrentsIntoSimulation).

Failed at Wed Aug 01 04:47:03 2018 (Elapsed Time: 9 minutes 21 seconds)

SREENATH, K. R.
Scientist
Marine Biodiversity Division, ICAR-Central Marine Fisheries Research Institute
p:	(+91)484-2394867 m: (+91)9662212556
f:	(+91) 484 - 2394909
a:	CMFRI, Abraham Madamackal Road, Kochi, Kerala, India - 682 018.
w:	www.cmfri.org.in e:

On Wed, Aug 1, 2018 at 3:19 AM, Sreenath Ramanathan <> wrote:

Dear Dr. Jason,

I could recognize my mistake a little while before you have replied as I was checking the cell size of the images of current in 'u' folder in the sim dir. Perhaps the polygons which I have used to make the patch ID created the trouble. Thanks for your fast reply. I am not sending you my Directory at this point, but will try to clear this error myself and get back to you if I again stumble up on any more errors. Again thanks for spending your valuable time on this.

Kind regards

SREENATH, K. R.
Scientist
Marine Biodiversity Division, ICAR-Central Marine Fisheries Research Institute

p: (+91)484-2394867  m: (+91)9662212556
f: (+91) 484 - 2394909
a: CMFRI, Abraham Madamackal Road, Kochi, Kerala, India - 682 018.
w: www.cmfri.org.in  e:

On Wed, Aug 1, 2018 at 3:09 AM, Jason Roberts <> wrote:

Dear Dr. Sreenath,

What is the coordinate system used by your rasters? And what is the cell size?

In order to use the larval dispersal tool, it is necessary that your rasters be in a projected coordinate system. Such a system will use a linear unit of measure, typically meters. It is necessary to use a projected system so that physical calculations are easy to perform.

When you say you used patches with cell sizes of 0.08 and 0.046, it sounds like those units are in degrees, which is an angular unit. This suggests that the rasters are not in a projected coordinate system. The Create Larval Connectivity Simulation tool is supposed to fail if you provide rasters in an unprojected coordinate system. Did that happen?

Alternatively, it may be that you created rasters in a projected system with cell sizes of 0.08 or 0.046. This seems very unlikely, because those sizes would be in meters, resulting in a raster with a huge number of rows and columns. But if that is indeed what happened, it is not surprising that the Resampling tool used inside MGET would fail.

If you would like, you may send me the Simulation Directory created by the Create Larval Connectivity Simulation tool. I will check it to see if the coordinate system and the cell size seems reasonable for use with the HYCOM GLBa0.08 data.

Best regards,

Jason

From: <> On Behalf Of Sreenath Ramanathan
Sent: Tuesday, July 31, 2018 3:10 PM
To:
Subject: [mget-help] Resampling error-MGET

Dear Dr.Jason,

I got Resampling failed error 000995 when I used patches of both the cellsizes 0.08 and 0.046. I was trying to simulate the coral larvae of eastern part of Arabian Sea using default GLBa0.08. Patch cover, ID and Wmask have same cellsize and row and columns.

Kind regards

SREENATH, K. R.

Scientist

Marine Biodiversity Division, ICAR-Central Marine Fisheries Research Institute

p:

(+91)484-2394867  m: (+91)9662212556

f:

(+91) 484 - 2394909

a:

CMFRI, Abraham Madamackal Road, Kochi, Kerala, India - 682 018.

w:

www.cmfri.org.in  e:

Re: [mget-help] Resampling error-MGET, Sreenath Ramanathan, 07/31/2018
- RE: [mget-help] Resampling error-MGET, Jason Roberts, 08/01/2018

Please Wait...

Text archives

Re: [mget-help] Resampling error-MGET