Skip to Content.

mget-help - RE: [mget-help] MGet error messages

Please Wait...

Subject: Marine Geospatial Ecology Tools (MGET) help

Text archives


From: "Jason Roberts" <>
To: "'Burbank, Renane Carla. \(SSC-UA10\)[SSC Education Office]'" <>
Cc: <>
Subject: RE: [mget-help] MGet error messages
Date: Wed, 29 Jun 2011 12:52:55 -0400

Dear Renane,

 

I’m sorry you’re hitting some problems. We should be able to get to the bottom of them. Let’s take them one at a time.

 

Error aqua mean 2008.docx

 

In this, there are several warnings and an error at the end:

 

There is no data in MODIS Aqua L3 mapped daily 4km thermal IR NSST from NASA PO.DAAC for the time slice that starts on 2002-07-30 12:00:00.

 

This one is pretty self explanatory. There is simply no MODIS data available for that date, due to problems with the satellite, or whatever. The raster for that date will be blank.

 

Failed to open URL 'http://opendap.jpl.nasa.gov/opendap/allData/modis/L3/aqua/11um/4km/daily/2006/196/A2006196.L3m_DAY_NSST_4.bz2' with OPeNDAP due to ServerError: 'Server error 1001: "Unable to lock the cache /opendap_cache/hyrax/cache/seadap"'. Retrying...

Retry successful.

 

Messages like this represent temporary problems accessing the NASA PO.DAAC server. The tool retries the download until it is successful or it times out. In this case, the retry was successful. You can safely ignore it, but if it happens a lot, it may indicate that the NASA server is becoming overloaded and you can expect that it might go down, ultimately causing the tool to time out and stop.

 

AttributeError: 'list' object has no attribute 'next'

<type 'exceptions.AttributeError'>: 'list' object has no attribute 'next'

Failed to execute (MODISL3SSTTimeSeriesCreateArcGISRasters).

 

This looks like a bug in MGET. It may be caused by something going wrong with the download, but MGET should either retry it or fail gracefully with a more intelligible message. Unfortunately I cannot determine the root cause or how to fix it just from that message.

 

Could you please run the tool again with the exact same parameters? It will detect the existing rasters and not download them again. Instead it will pick up where it left off. Hopefully that will cause it to fail with the same message almost immediately. If we can reliably reproduce it, I can tell you how to turn on more logging messages and we can identify the exact reason for the failure, which will allow me to fix the problem.

 

Error message aqua mean 2010.docx

 

No data is available to create the January climatological sum of the MODIS Aqua L3 mapped daily 4km thermal IR NSST from NASA PO.DAAC, masked where quality level > 1, clipped to indices tMin = 2769, tMax = 3102, yMin = 2568, yMax = 2927, xMin = 1968, xMax = 2735.

 

This occurs because the date range you specified started 2/1/2010. The tool is just saying that it cannot create a January climatology because there was no data for January. That is expected in your situation so you can ignore it.

 

Failed to open URL 'http://opendap.jpl.nasa.gov/opendap/allData/modis/L3/aqua/11um/4km/daily/2010/032/A2010032.L3m_DAY_NSST_4.bz2' with OPeNDAP due to ServerError: 'Server error 1001: "Unable to lock the cache /opendap_cache/hyrax/cache/seadap"'. Retrying...

 

It is failing to access the NASA PO.DAAC server again. But rather than succeeding, it continues to fail and then times out after the limit of 300 seconds is reached with this message:

 

RuntimeError: Failed to open URL http://opendap.jpl.nasa.gov/opendap/allData/modis/L3/aqua/11um/4km/daily/2010/032/A2010032.L3m_DAY_NSST_4.bz2 with OPeNDAP. The operation was retried for 300 seconds without success. Verify that the URL is correct. If it is, check that the server is operating properly and that your computer can connect to it. If necessary, contact the server's operator for assistance. If the server and network are operating properly, this problem could be a programming error in this tool. If you suspect one, contact the author of this tool for assistance. Error details: ServerError: 'Server error 1001: "Unable to lock the cache /opendap_cache/hyrax/cache/seadap"'

 

It appears that the NASA PO.DAAC server experienced a problem that lasted longer than the 300 seconds. I checked it when I started composing my email message and it was still down. But I just checked it right now and it was back up again. They must have detected the problem and fixed it. If you try the tool again, it should get around this problem.

 

But there is another problem you should be aware of. Due to a problem with the software that NASA uses for their OPeNDAP server, it is not possible to access MODIS data for 2010 day 265 or later dates. Please see the attached message for more information. This means that any MGET tool you use to access MODIS will ultimately fail unless you restrict it to not go beyond day 264 of 2010. I’m sorry for this inconvenience, but there is nothing we can do until PO.DAAC can fix the server. I will send them another email about it, CC to you.

 

Let me know if you have any more questions…

 

Best regards,

 

Jason

 

From: Burbank, Renane Carla. (SSC-UA10)[SSC Education Office] [mailto:]
Sent: Wednesday, June 29, 2011 11:52 AM
To:
Subject: [mget-help] MGet error messages

 

Hello,

 

I am an intern student with the NASA DEVELOP National Program.  We are using the MGet tool  to pull data sets from the NASA JPL. PO. DAAC site.

 

Tool used:

Data Products > NASA JPL. PO. DAAC site. > MODIS Global Level 3 Mapped > Create Climatological Raster’s for PO.DAAC MODIS L3 SST

 

Parameters used:

Satellite – Aqua

Temporal resolution – Daily

Spatial resolution  - 4km

Geophysical parameter – NSST

Dataset variable – l3m_data

Statistic – Mean

Climatology bin type – monthly

Masking options – lvl 1

Spatiotemporal extent

Y max 32.0, Y min 17.0, X Min -98.0, X Max -66.0

Dates:

1/1/2008 – 12/31/2008,

1/1/2009-12/31/2009,

1/1/2010-12/31-2010

 

I have attached the error messages for 2008 and 2010.  I have checked the website, the server and everything on our end.  Could you please help me out on this.   

 

 

 

C. Renane Burbank

Summer Intern

DEVELOP National Program
Building 1105, Room F605B
Stennis Space Center, MS 36529
228.688.2214 voice
228.688.2764 fax

 

--- Begin Message ---
From: "Henderson, Matthew L \(388C\)" <>
To: "Jason Roberts" <>
Cc: <>
Subject: Re: OPeNDAP
Date: Wed, 18 May 2011 01:52:42 -0400
Jason,

Unfortunately we are still waiting on a fix from the opendap group.  The latest version was supposed to correct this issue, but after installing and testing the latest release internally we still see the same issue here.  We will notify you as soon as the situation changes.

Matt

From: Jason Roberts <>
Date: Mon, 9 May 2011 09:53:52 -0700
To: Matt Henderson <>
Cc: "" <>
Subject: RE: OPeNDAP

Hi Matt,

 

Do you have an update on this problem? I have another user that is interested in MODIS SST data within the problematic time period.

 

Thanks for your help,

 

Jason

 

From: Henderson, Matthew L (388C) []
Sent: Wednesday, April 20, 2011 4:50 PM
To: Jason Roberts
Cc:
Subject: Re: OPeNDAP

 

Jason,

 

I got a response back from the OPeNDAP folks.  This is confirmed as a bug within the OPeNDAP server that has been fixed in a newer release.

I'm trying to find out from them now if they can patch the versions we have as well, or if I can apply the patches myself.

I'll let you know what the verdict is there.

 

Unfortunately the latest version is not available yet in a form that we can install, so we don't have the option currently of installing a new version.

 

Matt

 

From: Jason Roberts <>
Date: Mon, 18 Apr 2011 17:55:01 -0700
To: Matt Henderson <>
Cc: "" <>
Subject: RE: OPeNDAP

 

Hi Matt,

 

Thanks for the reply and the detailed information. I’m surprised nobody else has encountered it yet, but perhaps they are just like us and don’t have many experiments running that require data close to the present date. I hope you are able to resolve it quickly. Please keep me apprised…

 

Jason

 

From: Henderson, Matthew L (388C) []
Sent: Monday, April 18, 2011 8:24 PM
To:
Cc:
Subject: re: OPeNDAP

 

Hi Jason,

 

Just letting you know I'm looking into this.

 

Actually the problem is a little worse than your description.  It turns out that files starting from 2010/265 onward for the L3 are not being served properly via Hyrax, our OPeNDAP server.  I haven't gotten to the bottom of it yet, but the problem seems to be related to either a new hdf library used to produce the files, or new software used to create the GHRSST data.  I'm able to pull the files via ftp and read them here, so it seems like it's not an hdf problem, but a Hyrax problem.  I'll see what I can do here, but we may have to get some clarity from the Hyrax developers to resolve the problem.

 

I'm going to get in touch with the Hyrax developers anyway to see if this is a known problem.  I'll be sure to follow up with you as soon as I know more.

 

Thanks for bringing this to our attention.  We'll get this resolved ASAP.

 

Matt

 

 

 

 

MODIS 2011 files do not work on PO.DAAC OPeNDAP server


Greetings NASA folks,

We are encountering some difficulty downloading MODIS 2011 data from the
PO.DAAC OPeNDAP server. None of the 2011 datasets we've tried will work, on
either the existing OPeNDAP server or the Beta server that is supposed to go
live this month.

For example, the following URL is supposed to open the OPeNDAP Server
Dataset Access Form, but it returns a blank page. 

http://dods.jpl.nasa.gov/opendap/hyrax/sea_surface_temperature/modis/data/aq
ua/L3_mapped/sst4/daily/04km/2011/001/A2011001.L3m_DAY_SST4_4.bz2.html


Compare that to the 2010 URL, which does work:

http://dods.jpl.nasa.gov/opendap/hyrax/sea_surface_temperature/modis/data/aq
ua/L3_mapped/sst4/daily/04km/2010/001/A2010001.L3m_DAY_SST4_4.bz2.html


Here is the broken 2011 URL on the new server:

http://opendap.jpl.nasa.gov/opendap/allData/modis/L3/aqua/4um/4km/daily/2011
/001/A2011001.L3m_DAY_SST4_4.bz2.html

That results in the 'OUCH! Something Bad Happened On This Server' page, with
the error message 'libdap exception building response: error_code = 1001:
Could not read Vdata records.'

Here is a 2010 URL on the new server, which works:

http://opendap.jpl.nasa.gov/opendap/allData/modis/L3/aqua/4um/4km/daily/2010
/001/A2010001.L3m_DAY_SST4_4.bz2.htm

Could you please take a look at this? If possible, could you fix it on both the old server and the new server? I know you're about to promote the new server out of beta. We have some existing tools that operate against the old server and will not be able to update them to use the new server until later this month.

Note that the problem does not seem to be restricted to Aqua daily SST4. I
tried 8Day, I tried SST and NSST, I tried Terra. Nothing would work from
2011.

Thanks for your help,
Jason

 


--- End Message ---
Archives powered by MHonArc.
Top of Page