Hey Jason, thanks for the quick response. Let me know if you ever make it up this way. Are you thinking of going to the ESRI Ocean Summit Nov. 7-8 in Redlands. Notice just came across my desk.

 

On to the problem. The good news is that I can replicate it, every time.. That is also the bad news L   Attached is the point file we are using. Location is outer WA coast.  I think the dialog box below give you all the info you need to try and run it on your end. I tried several different sensors and Product Codes.

 

Give me a call if you have any questions. And I will be real mad if it works for you ;-}

 

I assume that is it SOP to have to hit a couple of different sensors for the same parameter (SST) to fill in missing data. You can see that in the Terra and Aqua SST columns. Any wisdom on the best order of sats/sensors to do this, and thoughts on the comparability of these different sensors.

 

Andy

 

 

 

Andy,

 

Nice to see you in Oakland. Sorry I was only in town for 21 hours. We’ll have to make a more concerted effort to catch up next time…

 

A couple of people reported a similar problem recently. Can you reliably reproduce it, and does it seem to happen on the same file?

 

Behind the scenes, the tool downloads 2D OceanColor HDFs, decompresses them from bz2 format, and reads them with the pyhdf library. The error occurs at the bz2-decompression step. Python’s built-in bz2 module seems to complain that the files are corrupt. But whenever someone sends in an error report, I try to repro it on the problem file and it always succeeds.

 

My working hypothesis is that a transient problem with the NASA server causes files to be incompletely downloaded. They made some changes over the summer that seemed to be coincident with this problem starting up. But since I’ve never reproed it myself, I haven’t been able to investigate.

 

If you can reliably repro it—even if it doesn’t happen on the same file—I can send some debug code that will help us get to the bottom of the issue. Let me know how you fare. I’d like to get it figure out.

 

Thanks,

 

Jason

 

 

Hi Jason,

 

Been using your fantastic GeoEco tools, particularly your automated Interpolate satellite images at points tools, but am running into a problem with Ocean Color products.

 

Successfully got L3 SST from from Terra and Aqua, but the GSFC ocean color is bombing.  Also no luck with SeaWifs or Terra (period of record 5/4/2011 – 8/11/2011).

 

And, BTW, good to see you at SCB in Oakland. Sorry to have missed you at SCGIS.

 

Andy

 

Andrew D. Weiss

Fish Program GIS Lead

WA Dept. of Fish and Wildlife

600 Capitol Way N   MS: 43150

Olympia, WA 98501-1091

Tel: 360-902-2487

Email:

 

 

 

Executing: OceanColorLevel3SMITimeSeriesInterpolateAtArcGISPoints Aqua 8day 4km CHL_chlor_a Samples_2011_assignments_MAC GSFC_OCeanColor_L3_ChloroA_8D_4k GPS_Date Nearest # # # 60 300 # 256 16 16 3

Start Time: Mon Sep 24 10:26:26 2012

Running script OceanColorLevel3SMITimeSeriesInterpolateAtArcGISPoints...

Interpolating values for 737 points of ArcGIS FeatureLayer "Samples_2011_assignments_MAC" of FeatureClass "C:\data\data\Andy\0Projects\OceanGSI\OceanGSI_2011\OceanGSI_2011.gdb\Samples_2011_assignments_MAC_UTM10".

EOFError: compressed file ended before the logical end-of-stream was detected

The following consequences resulted from the original error:

Could not decompress file c:\users\weissadw\appdata\local\temp\GeoEco_OceanColorLevel3SMIFileSearcher_Temp_3fe5a_\A20111212011128.L3m_8D_CHL_chlor_a_4km.bz2 to directory c:\users\weissadw\appdata\local\temp\GeoEco_OceanColorLevel3SMIFileSearcher_Temp_3fe5a_

Update stopped before all points were processed: 0:02:54 elapsed, 0 points updated, 0 deleted, 0 unchanged, 0:00:00 per point, 737 points not processed.

Update in progress: 0:02:54 elapsed, 0 points updated, 0 deleted, 1 unchanged, 0:02:54.250000 per point, 736 remaining, estimated completion time: unknown; more progress needed.

<type 'exceptions.EOFError'>: compressed file ended before the logical end-of-stream was detected

Failed to execute (OceanColorLevel3SMITimeSeriesInterpolateAtArcGISPoints).

Failed at Mon Sep 24 10:29:24 2012 (Elapsed Time: 2 minutes 58 seconds)