Subject: Marine Geospatial Ecology Tools (MGET) help
Text archives
| From: | Jason Roberts <> |
|---|---|
| To: | Jana Jeglinski <> |
| Cc: | "" <> |
| Subject: | [mget-help] RE: Problem with MGET "interpolate no data cells" tool |
| Date: | Tue, 20 May 2014 19:17:10 +0000 |
| Accept-language: | en-US |
| Authentication-results: | spf=none (sender IP is ) ; |
Hello Jana,
Thanks for your interest in MGET. You have an interesting problem.
The core code used by the MGET tool you mention is implemented in Matlab, by
a function called inpaint_nans written by John D'Errico. See
http://www.mathworks.com/matlabcentral/fileexchange/4551-inpaint-nans. There ;
is a lot of good discussion on that page and documentation in the code that
you might be interested in, depending on how deep you want to dive into this
problem.
As you noticed, there are two basic kinds of interpolators available: a
Laplacian interpolator (the "Del2" methods, and the related "Del4" method)
and a "spring" interpolator. In general, both methods should not produce
negative values for empty regions that are completely surrounded by non-empty
cells. The place where trouble is most possible is along coastlines, where an
empty region extends to the edge of a raster, providing no boundary points
for the interpolator to work with. In that situation, both methods will
extrapolate. Speaking very roughly, they will "continue the trend" that
occurred right at the edge of the region that had data. If the region with
data showed a downward trend or gradient in the direction of the adjacent
region that was empty, the interpolators will fill in that region with values
that continue to descend as they progress across it, away from the area that
had the data that established the trend.
Unfortunately, the code as currently written has no notion of a minimum
possible value. If you trust the interpolator's guess that chlorophyll
concentration should continue to decrease in the problematic empty area, my
recommendation is to implement a post-interpolation step with Spatial
Analyst, in which you set all negative values to something appropriately
small, such as 0.01 mg/m^3, or whatever you think is "really close to zero".
It's kind of a kludge, but I'm not immediately sure what would be better,
assuming that leaving the area empty is not acceptable.
Another thing to beware of is that extrapolation problems can occur in the
opposite direction. Often chlorophyll values rise exponentially approaching
shore, and as you noted remote sensing often masks values very close to shore
(due to inability to estimate chlorophyll concentration accurately in turbid
waters, etc.) It is hard to know whether an extrapolator is guessing accurate
values for these shoreline pixels. If your animal model depends strongly on
the accuracy of those pixels, this could be problematic.
I hope this helps! Sorry I could not provide a perfect solution.
Best,
Jason
-----Original Message-----
From: Jana Jeglinski
[mailto:]
Sent: Tuesday, May 20, 2014 4:48 AM
To:
Subject: [mget-help] Problem with MGET "interpolate no data cells" tool
Hello,
I came across an interesting situation using the fill NaNs in rasters tool
and it would be fantastic if you could point me to some help.
The purpose for using the tool is to fill empty cells in rasters of
environmental covariates such as chlorophyll concentration (Aqua Modis NPP
0.025 degrees acquired from coastwatch) with the general aim to extract
values of these covariates for GPS tracking data from sea lions. I am
particularly interested in filling empty cells along the coastline, because
the animals I study forage very coastally and the quite coarse resolution of
the covariate datatsets uses an equally coarse landmask, which leaves gaps of
at least one cell to the true coastline.
I had no problems running the tool but discovered that in some of my
covariate chlorophyll rasters negative values are produced for previously
empty cells. This of course is a major problem because an animal that foraged
in this cell would "prefer" a negative chlorophyll concentration. Do yo have
an idea how this can happen during the extrapolation process? Would you be
able to help me how to avoid this situation, while being able to fill in
empty cells? I have been playing around with different settings and methods -
reducing the number of connected NaNs to 6 and in some case to 2 helped, but
of course does not do the job of extrapolating the missing data along the
coastline. I also used spring, which was more successful (out of 12 rasters
where negative values were produced only one ended up with negatives), but
not 100%.
I am using MGET 0.8a50 with Arc GIS desktop 10.1, have installed Matlab and
have a spatial analyst license.
I hope I managed to explain my problem in enough detail. Thank you very much
for your response.
Kind regards,
Jana Jeglinski
Dr. Jana W E Jeglinski
University of Glasgow
Institute of Biodiversity, Animal Health and Comparative Medicine Graham Kerr
Building, Rm 510
G12 8QQ, Scotland
Tel: +44 141 330 3560
University of Bielefeld
Galapagos Sea Lion Project
Department of Animal Behaviour
Morgenbreede 45
33615 Bielefeld
Germany
Thanks for your interest in MGET. You have an interesting problem.
The core code used by the MGET tool you mention is implemented in Matlab, by
a function called inpaint_nans written by John D'Errico. See
http://www.mathworks.com/matlabcentral/fileexchange/4551-inpaint-nans. There ;
is a lot of good discussion on that page and documentation in the code that
you might be interested in, depending on how deep you want to dive into this
problem.
As you noticed, there are two basic kinds of interpolators available: a
Laplacian interpolator (the "Del2" methods, and the related "Del4" method)
and a "spring" interpolator. In general, both methods should not produce
negative values for empty regions that are completely surrounded by non-empty
cells. The place where trouble is most possible is along coastlines, where an
empty region extends to the edge of a raster, providing no boundary points
for the interpolator to work with. In that situation, both methods will
extrapolate. Speaking very roughly, they will "continue the trend" that
occurred right at the edge of the region that had data. If the region with
data showed a downward trend or gradient in the direction of the adjacent
region that was empty, the interpolators will fill in that region with values
that continue to descend as they progress across it, away from the area that
had the data that established the trend.
Unfortunately, the code as currently written has no notion of a minimum
possible value. If you trust the interpolator's guess that chlorophyll
concentration should continue to decrease in the problematic empty area, my
recommendation is to implement a post-interpolation step with Spatial
Analyst, in which you set all negative values to something appropriately
small, such as 0.01 mg/m^3, or whatever you think is "really close to zero".
It's kind of a kludge, but I'm not immediately sure what would be better,
assuming that leaving the area empty is not acceptable.
Another thing to beware of is that extrapolation problems can occur in the
opposite direction. Often chlorophyll values rise exponentially approaching
shore, and as you noted remote sensing often masks values very close to shore
(due to inability to estimate chlorophyll concentration accurately in turbid
waters, etc.) It is hard to know whether an extrapolator is guessing accurate
values for these shoreline pixels. If your animal model depends strongly on
the accuracy of those pixels, this could be problematic.
I hope this helps! Sorry I could not provide a perfect solution.
Best,
Jason
-----Original Message-----
From: Jana Jeglinski
[mailto:]
Sent: Tuesday, May 20, 2014 4:48 AM
To:
Subject: [mget-help] Problem with MGET "interpolate no data cells" tool
Hello,
I came across an interesting situation using the fill NaNs in rasters tool
and it would be fantastic if you could point me to some help.
The purpose for using the tool is to fill empty cells in rasters of
environmental covariates such as chlorophyll concentration (Aqua Modis NPP
0.025 degrees acquired from coastwatch) with the general aim to extract
values of these covariates for GPS tracking data from sea lions. I am
particularly interested in filling empty cells along the coastline, because
the animals I study forage very coastally and the quite coarse resolution of
the covariate datatsets uses an equally coarse landmask, which leaves gaps of
at least one cell to the true coastline.
I had no problems running the tool but discovered that in some of my
covariate chlorophyll rasters negative values are produced for previously
empty cells. This of course is a major problem because an animal that foraged
in this cell would "prefer" a negative chlorophyll concentration. Do yo have
an idea how this can happen during the extrapolation process? Would you be
able to help me how to avoid this situation, while being able to fill in
empty cells? I have been playing around with different settings and methods -
reducing the number of connected NaNs to 6 and in some case to 2 helped, but
of course does not do the job of extrapolating the missing data along the
coastline. I also used spring, which was more successful (out of 12 rasters
where negative values were produced only one ended up with negatives), but
not 100%.
I am using MGET 0.8a50 with Arc GIS desktop 10.1, have installed Matlab and
have a spatial analyst license.
I hope I managed to explain my problem in enough detail. Thank you very much
for your response.
Kind regards,
Jana Jeglinski
Dr. Jana W E Jeglinski
University of Glasgow
Institute of Biodiversity, Animal Health and Comparative Medicine Graham Kerr
Building, Rm 510
G12 8QQ, Scotland
Tel: +44 141 330 3560
University of Bielefeld
Galapagos Sea Lion Project
Department of Animal Behaviour
Morgenbreede 45
33615 Bielefeld
Germany
Archives powered by MHonArc.