Subject: Marine Geospatial Ecology Tools (MGET) help
Text archives
| From: | Jason Roberts <> |
|---|---|
| To: | Andrew Gilbert <> |
| Cc: | "" <> |
| Subject: | RE: [mget-help] Canny front problems |
| Date: | Thu, 11 Jun 2015 13:43:34 +0000 |
| Accept-language: | en-US |
| Authentication-results: | briloon.org; dkim=none (message not signed) header.d=none; |
Andrew,
The root problem relates to this:
python.exe: Traceback (most recent call last):
python.exe: File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\CannyEdgesInGrid.py",
line 170, in <module>
python.exe: main()
python.exe: File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\CannyEdgesInGrid.py",
line 25, in main
python.exe: from GeoEco.Internationalization import _
python.exe: ImportError: No module named GeoEco.Internationalization
What's happening is that MGET is creating a child python.exe process to run
code implemented in Matlab, because Matlab and ArcGIS contain conflicting
shared libraries (.DLL files) that cannot be loaded into the same process.
Because of that, MGET cannot execute the Matlab code within the ArcMap.exe or
ArcCatalog.exe process that is running the MGET tool. (Without going into
detail, this is a design flaw in either ArcGIS or Matlab, or both, but
getting ESRI and Mathworks to settle the issue seemed like a hopeless task so
I have not bothered, and just have MGET work around it in this way...)
So, when that child Python process executes and begins running the MGET code
that will execute Matlab, it fails almost immediately because MGET has not
been installed into the Python installation that the child process was
started from. This usually means there is a Python installation problem, or,
more rarely, an MGET installation problem. Could you please:
1. Save the attached file to your desktop.
2. Double-click it. A black window will come up for a little while then go
away.
3. The script will create a text file called GetMoreInformation.txt on your
desktop. Send that to me.
The file is filled with information about how your computer is configured. It
should help diagnose the problem.
I do not recommend uninstalling and reinstalling MGET at this point. I
somehow missed that 10.3.1 had been released and have not modified the
installer to work with it. It is highly likely that MGET 0.8a58 is completely
compatible with 10.3.1, but when ESRI revs the version number, there is
usually something in the MGET installer that must be tweaked. Thus if you
uninstalled MGET right now, the installer might fail on 10.3.1.
This didn't used to be an issue when ESRI released service packs instead of
reving the version number. But their new policy seems to be rev the version
number instead.
In any case, I will download 10.3.1 immediately, test the MGET installer, and
issue an update to MGET ASAP if necessary.
Jason
-----Original Message-----
From: Andrew Gilbert
[mailto:]
Sent: Wednesday, June 10, 2015 4:43 PM
To:
Subject: [mget-help] Canny front problems
Hi all,
I'm trying to run the Canny Fronts estimation utility and I'm running into
errors (see below). I checked the online MGET archives and have seen a couple
of references to this, so I reinstalled MGET to see if it didn't take the
first time and it still doesn't work. A couple things to
note:
ArcGIS 10.3.1 (just updated to this, but never tried the tool prior to)
Python 2.7.8 Matlab Runtime 7.7 MGET 0.8a58
Any other suggestions are appreciated, but I wonder if 10.3.1 is not not to
blame?
Thanks, Andrew Gilbert
Executing: GHRSSTLevel4CannyEdgesAsArcGISRasters
JPL_OUROCEAN-L4UHfnd-GLOB-G1SST E:\OceanFronts Add
%(AreaCode)s;%(RDACCode)s;%(ProductCode)s;%(VariableName)s;%%Y;%%Y%%m%%d-%(RDACCode)s-L4%(ProductType)s-%(AreaCode)s-v%(GDSVersion)02i-fv%(FileVersion)s-%(ProductCode)s-Canny_fronts.img
# # 1.4 4 # "-76 36 -74 39" 4/22/2012 4/22/2014 600 # # true false false
Start Time: Wed Jun 10 16:13:19 2015 Running script
GHRSSTLevel4CannyEdgesAsArcGISRasters...
Querying time slices of Canny edges (high threshold=None, low threshold=None,
sigma=1.4, min size=4) in GHRSST L4 product JPL_OUROCEAN-L4UHfnd-GLOB-G1SST
hosted by NASA PO.DAAC, clipped to indices tMin = 683, tMax = 1413, yMin =
11600, yMax = 11899, xMin = 10400, xMax = 10599.
Query complete: 0:00:01 elapsed, 731 datasets found, 0:00:00.001508 per
dataset.
Importing 731 datasets into ArcGIS Folder E:\OceanFronts with mode "add".
Checking for existing destination datasets.
Finished checking: 0:00:00 elapsed, 731 datasets checked, 0:00:00.000284 per
dataset.
0 destination datasets already exist. Importing 731 datasets.
python.exe: Traceback (most recent call last):
python.exe: File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\CannyEdgesInGrid.py",
line 170, in <module>
python.exe: main()
python.exe: File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\CannyEdgesInGrid.py",
line 25, in main
python.exe: from GeoEco.Internationalization import _
python.exe: ImportError: No module named GeoEco.Internationalization
RuntimeError: python.exe returned exit code 1, indicating failure.
The following consequences resulted from the original error:
Execution of C:\Python27\ArcGISx6410.3\python.exe failed.
Import stopped before all datasets were imported: 0:00:02 elapsed, 0 datasets
imported, 0:00:00 per dataset, 731 datasets not imported.
Failed script GHRSSTLevel4CannyEdgesAsArcGISRasters...
Traceback (most recent call last):
File "C:\Program
Files\GeoEco\ArcGISToolbox\Scripts\GHRSSTLevel4CannyEdgesAsArcGISRasters.py",
line 5, in <module>
ExecuteMethodFromCommandLineAsArcGISTool('GeoEco.DataProducts.NASA.PODAAC',
'GHRSSTLevel4', 'CannyEdgesAsArcGISRasters')
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\ArcGISScripts.py", line 210,
in ExecuteMethodFromCommandLineAsArcGISTool
exec sourceCode in globals(), locals()
File "<string>", line 1, in <module>
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\DataProducts\NASA\PODAAC.py",
line 915, in CannyEdgesAsArcGISRasters
workspace.ImportDatasets(GridSliceCollection(grid,
tQACoordType=u'min').QueryDatasets(), mode,
calculateStatistics=calculateStatistics, buildRAT=buildRAT,
buildPyramids=buildPyramids)
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\__init__.py",
line 820, in ImportDatasets
self._ImportDatasets(datasets, mode.lower(), reportProgress, options)
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\Collections.py",
line 694, in _ImportDatasets
self._ImportDatasetsToPath(pathComponentsForPath[path],
datasetsForPath[path], mode, progressReporter, options)
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\ArcGIS.py",
line 609, in _ImportDatasetsToPath
self.DatasetType._ImportDatasetsToPath(os.path.join(self.Path,
*pathComponents), sourceDatasets, mode, progressReporter, options)
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\ArcGIS.py",
line 1121, in _ImportDatasetsToPath
GDALDataset._ImportDatasetsToPath(path, sourceDatasets, mode, None,
{'useArcGISSpatialReference': True, 'useUnscaledData': useUnscaledData,
'calculateStatistics': False, 'blockSize': blockSize})
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\GDAL.py", line
1073, in _ImportDatasetsToPath
data = sourceDatasets[i].Data[rowsCopied:rowsCopied+rowsToCopy, :]
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\__init__.py",
line 3670, in __getitem__
return getattr(self._Grid(), self._GetMethod)(key)
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\Virtual.py",
line 562, in _GetUnscaledDataAsArray
return
self._Grid._GetUnscaledDataAsArray(self._AddSlicedDimsToKey(key))
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\__init__.py",
line 3310, in _GetUnscaledDataAsArray
data, actualNoDataValue = self._ReadNumpyArray(reorderedSliceList)
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\Virtual.py",
line 3594, in _ReadNumpyArray
maxRunTime=None)
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\DataManagement\Processes.py",
line 575, in ExecuteProgram
process.Execute()
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\DataManagement\Processes.py",
line 498, in Execute
Logger.RaiseException(RuntimeError(_(u'%(name)s returned exit code
%(code)i, indicating failure.') % {u'name' : programName, u'code' :
self.ExitCode}))
File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Logging.py",
line 104, in RaiseException
raise exception
RuntimeError: python.exe returned exit code 1, indicating failure.
Failed to execute (GHRSSTLevel4CannyEdgesAsArcGISRasters).
Failed at Wed Jun 10 16:13:33 2015 (Elapsed Time: 14.08 seconds)
--
Andrew Gilbert
Data Management Director
Biodiversity Research Institute
276 Canco Rd.
Portland, ME 04103
207-839-7600
www.briloon.org
The root problem relates to this:
python.exe: Traceback (most recent call last):
python.exe: File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\CannyEdgesInGrid.py",
line 170, in <module>
python.exe: main()
python.exe: File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\CannyEdgesInGrid.py",
line 25, in main
python.exe: from GeoEco.Internationalization import _
python.exe: ImportError: No module named GeoEco.Internationalization
What's happening is that MGET is creating a child python.exe process to run
code implemented in Matlab, because Matlab and ArcGIS contain conflicting
shared libraries (.DLL files) that cannot be loaded into the same process.
Because of that, MGET cannot execute the Matlab code within the ArcMap.exe or
ArcCatalog.exe process that is running the MGET tool. (Without going into
detail, this is a design flaw in either ArcGIS or Matlab, or both, but
getting ESRI and Mathworks to settle the issue seemed like a hopeless task so
I have not bothered, and just have MGET work around it in this way...)
So, when that child Python process executes and begins running the MGET code
that will execute Matlab, it fails almost immediately because MGET has not
been installed into the Python installation that the child process was
started from. This usually means there is a Python installation problem, or,
more rarely, an MGET installation problem. Could you please:
1. Save the attached file to your desktop.
2. Double-click it. A black window will come up for a little while then go
away.
3. The script will create a text file called GetMoreInformation.txt on your
desktop. Send that to me.
The file is filled with information about how your computer is configured. It
should help diagnose the problem.
I do not recommend uninstalling and reinstalling MGET at this point. I
somehow missed that 10.3.1 had been released and have not modified the
installer to work with it. It is highly likely that MGET 0.8a58 is completely
compatible with 10.3.1, but when ESRI revs the version number, there is
usually something in the MGET installer that must be tweaked. Thus if you
uninstalled MGET right now, the installer might fail on 10.3.1.
This didn't used to be an issue when ESRI released service packs instead of
reving the version number. But their new policy seems to be rev the version
number instead.
In any case, I will download 10.3.1 immediately, test the MGET installer, and
issue an update to MGET ASAP if necessary.
Jason
-----Original Message-----
From: Andrew Gilbert
[mailto:]
Sent: Wednesday, June 10, 2015 4:43 PM
To:
Subject: [mget-help] Canny front problems
Hi all,
I'm trying to run the Canny Fronts estimation utility and I'm running into
errors (see below). I checked the online MGET archives and have seen a couple
of references to this, so I reinstalled MGET to see if it didn't take the
first time and it still doesn't work. A couple things to
note:
ArcGIS 10.3.1 (just updated to this, but never tried the tool prior to)
Python 2.7.8 Matlab Runtime 7.7 MGET 0.8a58
Any other suggestions are appreciated, but I wonder if 10.3.1 is not not to
blame?
Thanks, Andrew Gilbert
Executing: GHRSSTLevel4CannyEdgesAsArcGISRasters
JPL_OUROCEAN-L4UHfnd-GLOB-G1SST E:\OceanFronts Add
%(AreaCode)s;%(RDACCode)s;%(ProductCode)s;%(VariableName)s;%%Y;%%Y%%m%%d-%(RDACCode)s-L4%(ProductType)s-%(AreaCode)s-v%(GDSVersion)02i-fv%(FileVersion)s-%(ProductCode)s-Canny_fronts.img
# # 1.4 4 # "-76 36 -74 39" 4/22/2012 4/22/2014 600 # # true false false
Start Time: Wed Jun 10 16:13:19 2015 Running script
GHRSSTLevel4CannyEdgesAsArcGISRasters...
Querying time slices of Canny edges (high threshold=None, low threshold=None,
sigma=1.4, min size=4) in GHRSST L4 product JPL_OUROCEAN-L4UHfnd-GLOB-G1SST
hosted by NASA PO.DAAC, clipped to indices tMin = 683, tMax = 1413, yMin =
11600, yMax = 11899, xMin = 10400, xMax = 10599.
Query complete: 0:00:01 elapsed, 731 datasets found, 0:00:00.001508 per
dataset.
Importing 731 datasets into ArcGIS Folder E:\OceanFronts with mode "add".
Checking for existing destination datasets.
Finished checking: 0:00:00 elapsed, 731 datasets checked, 0:00:00.000284 per
dataset.
0 destination datasets already exist. Importing 731 datasets.
python.exe: Traceback (most recent call last):
python.exe: File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\CannyEdgesInGrid.py",
line 170, in <module>
python.exe: main()
python.exe: File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\CannyEdgesInGrid.py",
line 25, in main
python.exe: from GeoEco.Internationalization import _
python.exe: ImportError: No module named GeoEco.Internationalization
RuntimeError: python.exe returned exit code 1, indicating failure.
The following consequences resulted from the original error:
Execution of C:\Python27\ArcGISx6410.3\python.exe failed.
Import stopped before all datasets were imported: 0:00:02 elapsed, 0 datasets
imported, 0:00:00 per dataset, 731 datasets not imported.
Failed script GHRSSTLevel4CannyEdgesAsArcGISRasters...
Traceback (most recent call last):
File "C:\Program
Files\GeoEco\ArcGISToolbox\Scripts\GHRSSTLevel4CannyEdgesAsArcGISRasters.py",
line 5, in <module>
ExecuteMethodFromCommandLineAsArcGISTool('GeoEco.DataProducts.NASA.PODAAC',
'GHRSSTLevel4', 'CannyEdgesAsArcGISRasters')
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\ArcGISScripts.py", line 210,
in ExecuteMethodFromCommandLineAsArcGISTool
exec sourceCode in globals(), locals()
File "<string>", line 1, in <module>
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\DataProducts\NASA\PODAAC.py",
line 915, in CannyEdgesAsArcGISRasters
workspace.ImportDatasets(GridSliceCollection(grid,
tQACoordType=u'min').QueryDatasets(), mode,
calculateStatistics=calculateStatistics, buildRAT=buildRAT,
buildPyramids=buildPyramids)
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\__init__.py",
line 820, in ImportDatasets
self._ImportDatasets(datasets, mode.lower(), reportProgress, options)
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\Collections.py",
line 694, in _ImportDatasets
self._ImportDatasetsToPath(pathComponentsForPath[path],
datasetsForPath[path], mode, progressReporter, options)
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\ArcGIS.py",
line 609, in _ImportDatasetsToPath
self.DatasetType._ImportDatasetsToPath(os.path.join(self.Path,
*pathComponents), sourceDatasets, mode, progressReporter, options)
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\ArcGIS.py",
line 1121, in _ImportDatasetsToPath
GDALDataset._ImportDatasetsToPath(path, sourceDatasets, mode, None,
{'useArcGISSpatialReference': True, 'useUnscaledData': useUnscaledData,
'calculateStatistics': False, 'blockSize': blockSize})
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\GDAL.py", line
1073, in _ImportDatasetsToPath
data = sourceDatasets[i].Data[rowsCopied:rowsCopied+rowsToCopy, :]
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\__init__.py",
line 3670, in __getitem__
return getattr(self._Grid(), self._GetMethod)(key)
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\Virtual.py",
line 562, in _GetUnscaledDataAsArray
return
self._Grid._GetUnscaledDataAsArray(self._AddSlicedDimsToKey(key))
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\__init__.py",
line 3310, in _GetUnscaledDataAsArray
data, actualNoDataValue = self._ReadNumpyArray(reorderedSliceList)
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Datasets\Virtual.py",
line 3594, in _ReadNumpyArray
maxRunTime=None)
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\DataManagement\Processes.py",
line 575, in ExecuteProgram
process.Execute()
File
"C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\DataManagement\Processes.py",
line 498, in Execute
Logger.RaiseException(RuntimeError(_(u'%(name)s returned exit code
%(code)i, indicating failure.') % {u'name' : programName, u'code' :
self.ExitCode}))
File "C:\Python27\ArcGIS10.3\lib\site-packages\GeoEco\Logging.py",
line 104, in RaiseException
raise exception
RuntimeError: python.exe returned exit code 1, indicating failure.
Failed to execute (GHRSSTLevel4CannyEdgesAsArcGISRasters).
Failed at Wed Jun 10 16:13:33 2015 (Elapsed Time: 14.08 seconds)
--
Andrew Gilbert
Data Management Director
Biodiversity Research Institute
276 Canco Rd.
Portland, ME 04103
207-839-7600
www.briloon.org
Attachment:
GetMoreInformation.py
Description: GetMoreInformation.py
Archives powered by MHonArc.