Dear MGET users,

 

Recently we have received many reports of trouble accessing NASA ocean color data (e.g. chlorophyll concentration) with MGET tools such as Create Rasters for NASA OceanColor L3 SMI Product. The common thread to all of these reports was that MGET was run from ArcGIS 10.2.2 or earlier and the tool would fail with message similar to this:

 

RuntimeError: Failed to query the NASA OceanColor server at https://oceandata.sci.gsfc.nasa.gov/search/file_search.cgi. The query was retried for 300 seconds without success. Check that the server is operating properly and that your computer can connect to it. If necessary, contact the server's operator for assistance. If the server and network are operating properly, this problem could be a programming error in this tool. If you suspect one, contact the author of this tool for assistance. Error details: error: [Errno 10054] An existing connection was forcibly closed by the remote host

 

The root cause of this problem is that the NASA server recently began requiring clients to issue requests over https and for clients to support relatively recent improvements to the https protocol. If a client does not meet these requirements, the server refuses to communicate with it.

 

MGET is implemented in the Python programming language, which is installed with ArcGIS by the ArcGIS installer. For https communication, MGET relies on code built into Python itself, which in turn relies on the widely-used OpenSSL programming library. Important problems have been discovered in OpenSSL in the past few years, such as the famous “heartbleed” bug. The Python team has kept up with the problems and continued to release new versions of Python as they are discovered. But ArcGIS does not automatically update Python when new Python versions are released.

 

The usual way that ArcGIS users update Python is by installing a new version of ArcGIS. So if you have been updating your ArcGIS, you’ve been keeping up with the bug fixes made to Python. But if you’ve stayed with an old version of ArcGIS, you’re still running an old version of Python that includes the OpenSSL bugs, unless you updated Python manually yourself.

 

From what I can guess, the NASA server is likely to refuse to communicate with clients that are running Python 2.7.7 or earlier. ArcGIS 10.2.x came with Python 2.7.5 and 10.3 came with 2.7.8. Therefore if you have ArcGIS 10.2.2 or earlier, the NASA server will refuse the connection, but if you have 10.3 or later, it is likely to work.

 

How to solve this problem:

 

The best way is to upgrade ArcGIS to 10.3 or later, which will update Python for you. Before upgrading ArcGIS, uninstall your existing MGET and the Python pywin32 package. After upgrading ArcGIS, install pywin32 and MGET again.

 

If you can’t upgrade ArcGIS but have 10.1, 10.2, 10.2.1, or 10.2.2, you can manually update Python 2.7:

 

1.    You must have ArcGIS 10.1, 10.2, 10.2.1, or 10.2.2. If you have ArcGIS 10.0 or earlier, there is no solution. ArcGIS 10.0 runs Python 2.6; ArcGIS 9.3 runs Python 2.5. The Python team never patched these. You must upgrade to ArcGIS 10.1 or later so that you have Python 2.7. If that is not possible for you, stop here. You will have to find some other way to obtain data from NASA, such as manually downloading and converting netCDF files.

 

2.    Once you have ArcGIS 10.1 – 10.2.2, log on to your computer as an administrator.

 

3.    Make a backup copy of the entire C:\Python27 directory, e.g. to C:\Python27_Backup. You can use this to restore your original Python version if something goes wrong.

 

4.    Go to the Python website here and download the Python 2.7.13 “Windows x86 MSI Installer”. (Note: the Duke email system will rewrite the “here” link to something long and scary. It is ok. The Duke system is trying to protect us from malware and I have no way to disable this protection.)

 

5.    Run the downloaded MSI file. When you get to the Select Destination Directory step, be sure to select the folder that says C:\Python27\ArcGIS10.1 or C:\Python27\ArcGIS10.2, depending on your version of ArcGIS. It should look like the screenshot below my signature (below) but say 10.1 or 10.2 instead of 10.5. Then accept the rest of the default options.

 

Several users tested this procedure with ArcGIS 10.2.x and reported success.

 

Finally, if you do not have MGET 0.8a67 or later, you should get the latest version of MGET here (uninstall your old version from the Windows control panel first). MGET 0.8a67 updated MGET’s root certificates to allow communication with certain NOAA and NASA servers.

 

If you have any questions, feel free to contact me directly.

 

All the best,

 

Jason

 

You are receiving this message because you have subscribed to the mget-announce or mget-help email lists. Click those links to unsubscribe.