Skip to Content.

mget-help - Re: [mget-help] HYCOM variable download failure

Please Wait...

Subject: Marine Geospatial Ecology Tools (MGET) help

Text archives


Re: [mget-help] HYCOM variable download failure


Chronological Thread 
    < Chronological >      < Thread >
  • From: Jason Roberts <>
  • To: Julia Stepanuk <>,
  • Subject: Re: [mget-help] HYCOM variable download failure
  • Date: Mon, 11 Mar 2024 15:54:17 -0400
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=duke.edu; dmarc=pass action=none header.from=duke.edu; dkim=pass header.d=duke.edu; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dgbP2mVlC/AE2qFX3bwfkwTz20N2ECsg9H5LZX2H5nQ=; b=VDPejqBSOI/uFZkAYZRrRZOQ3pyiLkAvbHFCgKBuwPiHqvjhow02g4hoqWZRqO6dMd/iD3SoiPiBMd2zm6vY7e5YjAgz6EFy1/4QTsovBVO83gfYdK/t7Q/N8kD/W0hADeW3FlNyf3g/ciuqw/ytXlzGeae0xhH05vD3LutKuCZdhMZoZcjCsKQrv3VV4eCB8XZf1seSjrvBmC0pIalpTqprBY9mySwp8e2A+nlInVGl4v4QJ/Y+I5+4sR+qbdeFsVuCfvQjLlmkzTD5IQQCoWTZIEhnbjXE1ldWcr4HRYwde6rRiplIWELzYkEIrP0axFJr8srgLQYVT8d2ogevqw==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BrXZ7kuBSa7VUhDjspg+bkBdBq7sVgVcRqEpGHimKZwu2a6W/OzrCW35oG4WYHfcOrqJcQv/TfpmtNSYDx8YMpS4P3j4PQL7ndaWpnSTMVDAImZGcIX3RLHuyus5Y1uP3RLAFRxBbrIIAxs1l9KttAo6qJTfedp1Y8DSr+txlYNVkeZzjgfZjtQ0QVXJxP4t5gD53/bvS9vXqzN3RH049XnQeB4No3SNpby6uJsXJoQlDdDGuSivwT2I2EaIL1253oAKrGn/LcbiFRf+KTXycafPWTCoXmamVG/yMo5BBIyzo6/BQEza/rWBEiCd+CmwQNdTUgSzzLHLDlNDwIBn5g==
Hi Julia,

Thank you for reported this. I was able to reproduce and fix it. It is a problem relating to MGET's internal copy of the pydap package, which we updated recently to enable compatibility with the NASA Earthdata servers. It turned out that part of the updated pydap was not compatible with a certain part of the numpy package, at least with the versions of numpy included with ArcGIS. I did not catch this in testing, so thanks again for letting me know about it.

I will privately email you a patch of the pydap source file that fixes the problem. I'm going to avoid attaching here with mget-help CC'd, as our email system as become more finicky about allowing .py files as attachments.

Best,
Jason

On 3/7/24 14:05, Julia Stepanuk wrote:
Hello MGET team!

Thank you for such a great product. I'm attempting to download 3D raster variables from the HYCOM dataset using MGET 0.8a79 and ArcMap 10.8.2 (though I have tried multiple arc versions and have tried on multiple machines). I'm running into an error where two arguments are given for an array that only takes one argument. I've copied the text below and am wondering if you can lend any assistance.

Thank you!
Julia Stepanuk

Executing: HYCOMGLBv0083DCreateArcGISRasters water_temp_bottom C:\Users\Julia.Stepanuk\Documents\envr Replace %(VariableName)s;%%Y;%(VariableName)s_%%Y%%m%%d_%%H%%M%%S.img # # "-75 34 -55 47" # # 60 600 # true false
Start Time: Thu Mar  7 14:02:19 2024
Running script HYCOMGLBv0083DCreateArcGISRasters...
Querying time slices of water_temp_bottom grid of the GOFS 3.1 HYCOM + NCODA Global 1/12 degree datasets (GLBv0.08), clipped to indices yMin = 1425, yMax = 1587, xMin = 1313, xMax = 1562.
Query complete: 0:00:32 elapsed, 88184 datasets found, 0:00:00.000366 per dataset.
Importing 88184 datasets into ArcGIS Folder C:\Users\Julia.Stepanuk\Documents\envr with mode "replace".
Import stopped before all datasets were imported: 0:00:06 elapsed, 0 datasets imported, 0:00:00 per dataset, 88184 datasets not imported.
TypeError: __array__() takes exactly 1 argument (2 given)
Failed script HYCOMGLBv0083DCreateArcGISRasters...

Traceback (most recent call last):
  File "C:\Program Files\GeoEco\ArcGISToolbox\Scripts\HYCOMGLBv0083DCreateArcGISRasters.py", line 5, in <module>
    ExecuteMethodFromCommandLineAsArcGISTool('GeoEco.DataProducts.HYCOM.GLBv008', 'HYCOMGLBv0083D', 'CreateArcGISRasters')
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\ArcGISScripts.py", line 210, in ExecuteMethodFromCommandLineAsArcGISTool
    exec sourceCode in globals(), locals()
  File "<string>", line 1, in <module>
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\DataProducts\HYCOM\GLBv008.py", line 551, in CreateArcGISRasters
    calculateStatistics=calculateStatistics, buildPyramids=buildPyramids)
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\DataProducts\HYCOM\_HYCOM.py", line 48, in _CreateArcGISRasters
    calculateStatistics=calculateStatistics, buildPyramids=buildPyramids)
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\DataProducts\HYCOM\_HYCOM.py", line 218, in _CreateArcGISRasters
    workspace.ImportDatasets(grids, mode, calculateStatistics=calculateStatistics, buildPyramids=buildPyramids)
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\Datasets\__init__.py", line 822, in ImportDatasets
    self._ImportDatasets(datasets, mode.lower(), reportProgress, options)
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\Datasets\Collections.py", line 695, in _ImportDatasets
    self._ImportDatasetsToPath(pathComponentsForPath[path], datasetsForPath[path], mode, progressReporter, options)
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\Datasets\ArcGIS.py", line 625, in _ImportDatasetsToPath
    self.DatasetType._ImportDatasetsToPath(os.path.join(self.Path, *pathComponents), sourceDatasets, mode, progressReporter, options)
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\Datasets\ArcGIS.py", line 1149, in _ImportDatasetsToPath
    GDALDataset._ImportDatasetsToPath(path, sourceDatasets, mode, None, {'useArcGISSpatialReference': True, 'useUnscaledData': useUnscaledData, 'calculateStatistics': False, 'blockSize': blockSize})
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\Datasets\GDAL.py", line 1073, in _ImportDatasetsToPath
    data = "sourceDatasets[i].Data[rowsCopied:rowsCopied+rowsToCopy," :]
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\Datasets\__init__.py", line 3678, in __getitem__
    return getattr(self._Grid(), self._GetMethod)(key)
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\Datasets\Virtual.py", line 582, in _GetUnscaledDataAsArray
    return self._Grid._GetUnscaledDataAsArray(self._AddSlicedDimsToKey(key))
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\Datasets\__init__.py", line 3318, in _GetUnscaledDataAsArray
    data, actualNoDataValue = self._ReadNumpyArray(reorderedSliceList)
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\Datasets\Virtual.py", line 1082, in _ReadNumpyArray
    return self._Grid._ReadNumpyArray(slicesToRead)
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\DataProducts\HYCOM\GLBv008.py", line 479, in _ReadNumpyArray
    data = "self._OpenGrid.Data.__getitem__((slice(gridTStart," gridTStop), sliceList[1], sliceList[2]))          # We are getting scaled data here
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\Datasets\__init__.py", line 3678, in __getitem__
    return getattr(self._Grid(), self._GetMethod)(key)
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\Datasets\__init__.py", line 3258, in _GetScaledDataAsArray
    unscaledData = self._GetUnscaledDataAsArray(key)
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\Datasets\__init__.py", line 3318, in _GetUnscaledDataAsArray
    data, actualNoDataValue = self._ReadNumpyArray(reorderedSliceList)
  File "C:\Python27\ArcGIS10.8\lib\site-packages\GeoEco\DataProducts\HYCOM\GLBv008.py", line 171, in _ReadNumpyArray
    result.__setitem__([slice(None), slice(copyToStart[i], copyToStop[i]), slice(None)], data)
TypeError: __array__() takes exactly 1 argument (2 given)

Failed to execute (HYCOMGLBv0083DCreateArcGISRasters).
Failed at Thu Mar  7 14:04:02 2024 (Elapsed Time: 1 minutes 42 seconds)

Biodiversity Research Institute

Biodiversity Research
Institute

Julia Stepanuk, PhD (she/her)

Quantitative Ecologist

 207-839-7600
 
 https://briwildlife.org

   

BRI is a nonprofit ecological research group based in Portland, Maine, conducting innovative wildlife science worldwide.
This email may contain confidential or proprietary materials for the sole use of the intended recipient.
If you are not the intended recipient, any review, use, distribution, or disclosure is strictly prohibited.


To unsubscribe from this list, visit: https://lists.nicholas.duke.edu/sympa/ca/initiate_unsubscribe/mget-help/jason.roberts%40duke.edu


Archive powered by MHonArc 2.6.19.

Top of Page