mget-help - RE: [mget-help] Error message using MGET tool

Subject: Marine Geospatial Ecology Tools (MGET) help

Text archives

RE: [mget-help] Error message using MGET tool

From: Jason Roberts <>
To: Fabio Ronchetti <>
Cc: "" <>
Subject: RE: [mget-help] Error message using MGET tool
Date: Mon, 5 Apr 2021 20:01:10 +0000
Accept-language: en-US
Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=duke.edu; dmarc=pass action=none header.from=duke.edu; dkim=pass header.d=duke.edu; arc=none
Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C7QPR6N9696leDFyFDM1ygB+WxeePuC4URVPvqC/D5g=; b=M6Ck23C3WAfb3AyukBBHgzae+mHJqQ2oa+pSefFlIl3FXcuIQlUFWda/MEsKdxECO54u57iFz+mrkA20dB0S6gqIUTGQPkNJXaOLETKtvdJXhkqw69WyUdkr10e5jc+5CccruaFSEU6Q7iJf37VAtDkzNLL2yDFA4SRslNYkM6I7PKfrg+mPTmY6N++l8u/BBNFfUWXH3zUIsBhYBsCTghTg9oUp2FL5QVDBmtWzmYqiqhkljVehcUi15Jc0xVykjuTv9Yi2vCadCLTWAr0aINVAMjOlbFGu/vFVywrlx0IZcC3vGBn7AvMDoPUdCK7VrYDmjXfQ0Y3I7ZtUY9dmnw==
Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=U/SzfRL67mIEXT2rOek3cOLbs92M0B05IKrdLY76nuezNZh2wb0spaGa6/7d1SoEZcLJvIqB5HjYFJQZLjIEThkmYDsB4b+XqfXTA0f+9h4UACVs3fJz82pJS07F0o5nQ/WY5TwFx2lvLzyEc4HkH2XjrCH/hPVkorxfRWSo8tpnuGRWdistFcvbTH66mGIjMrfpNOUTd/7Ifs5sO+5+e8n5KTdEiavzLxtBSlhVdQHshSz6yqT33Z9UtpB5KB7qaxNeJWvCWlKpia9toO69livF8uIyjW7qA208a5G5jFOPpdWEpNj7L556jBOsnOCKbaxb5DN4TLleT4eUJEO67g==
Authentication-results: gmail.com; dkim=none (message not signed) header.d=none;gmail.com; dmarc=none action=none header.from=duke.edu;

Dear Fabio,

Thanks for reporting this. I can reproduce the problem with ArcGIS 10.8 and the latest version of MGET.

It looks like NASA PO.DAAC reissued their server’s SSL certificate on 31 March. I suspect this is why it was working up until very recently. We ourselves were using this tool as recently as three weeks ago with no problem.

Internally, MGET uses a Python library called httplib2 to interact with web servers. httplib2 maintains a list of Certificate Authorities it trusts; in MGET’s copy of httplib2 these appear in C:\Python27\ArcGIS10.8\Lib\site-packages\GeoEco\httplib2\cacerts.txt. When this problem has happened in the past (SSLHandshakeError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed), it was because the server MGET was accessing began using a certificate that was ultimately tracible to a root CA that was not listed in httplib2’s cacerts.txt file. To fix it, we had to update the cacerts.txt file with an entry to the new root CA.

I’m going to investigate this further to see if that is the correct problem and solution here. If it is, I will probably take the opportunity to switch MGET over to use Mozilla’s store of root CAs. You can find a convenient and fairly authoritative copy of it rendered in the necessary format at https://curl.se/docs/caextract.html. If this is the right solution (which I’m not sure of yet) then you can probably patch your own installation by doing this:

Close all ArcGIS programs
Make a backup copy of C:\Python27\ArcGIS10.8\Lib\site-packages\GeoEco\httplib2\cacerts.txt
Download the cacert.pem file referenced at the top of the curl.se link I mentioned above. Overwrite C:\Python27\ArcGIS10.8\Lib\site-packages\GeoEco\httplib2\cacerts.txt with that file (changing the name from cacert.pem to cacerts.txt).
Start ArcGIS and try the MGET tool again. MGET’s copy of httplib2 will now use the new list of root certificates. If PO.DAAC’s new certificate descends from one of those, it will now be trusted.

I’ll let you know after I have a chance to look into this more.

Best,

Jason

From: <> On Behalf Of Fabio Ronchetti
Sent: Friday, April 2, 2021 7:33 AM
To:
Subject: [mget-help] Error message using MGET tool

Dear Jason and MGET team,

I am writing to request support in the use of the MGET tools for ArcGIS.

I'm running the Data Products tools to create a climatological raster for studying bottlenose dolphins. I'm using AVHRR_OI-NCEI-L4-GLOB-v2.0 to download SST. Everything was working fine until 3 days ago when I started to receive an error. Independently by the product that I'm going to select this is the message:

Failed to download THREDDS catalog https://opendap.jpl.nasa.gov/opendap/allData/ghrsst/data/GDS2/L4/catalog.xml. The HTTP request failed with SSLHandshakeError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727).

I use Window 10 Home version 20H2, ArcGIS 10.8 and the last version of your tool (wonderful tool!). I've tried to manage this problem in my OS changing some settings about SSL/TSL with no success. I've tried also to cancel the update of the OS of these last days.

I am attaching txt file with the complete message. Could you provide some help? Any assistance is really appreciated because I do not have any idea how to get around this issue.

Best regards,

Fabio