tag:blogger.com,1999:blog-7729093380675162051.post1774001535709127487..comments2021-10-13T19:43:03.069+11:00Comments on moyhu: On "Mannian" SmoothingNick Stokeshttp://www.blogger.com/profile/06377413236983002873noreply@blogger.comBlogger12125tag:blogger.com,1999:blog-7729093380675162051.post-36222637766635631592011-03-17T16:06:07.320+11:002011-03-17T16:06:07.320+11:00NIck, I agree with you here too: The Butterworth ...NIck, I agree with you here too: The Butterworth is not the most natural choice for smoothing data that contain secular trends in them. <br /><br />The transfer function is the magnitude squared of the "single-pass" version of the filter so the roll-off is twice the rate of the single-pass...this is something one needs to keep in mind when using forward-backward filtering. (On the other hand, it is free of phase distortion, and has zero delay.)Carrickhttps://www.blogger.com/profile/03476050886656768837noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-88148044698768206872011-03-16T19:59:17.099+11:002011-03-16T19:59:17.099+11:00Carrick,
I agree acausality doesn't matter, so...Carrick,<br />I agree acausality doesn't matter, so I'm wondering why the Butterworth is used at all. Maybe your two-pass version has the flat top and smooth roll-off that a 1-pass does (I expect it would), but it's not obvious that that is optimal for a time series smoothing filter. leaving, say, a low order polynomial invariant, as a Henderson filter does, seems more natural.Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-7611858070489772962011-03-16T17:50:59.549+11:002011-03-16T17:50:59.549+11:00Nick, I meant to comment on this earlier... You ca...Nick, I meant to comment on this earlier... You can implement a symmetric version of the Butterworth filter...it's "just" acausal. For analyses of this types, the acausality doesn't matter.<br /><br />(Matlab's "filtfilt" function does something like this:<br /><br /><i>y = filtfilt(b,a,x) performs zero-phase digital filtering by processing the input data, x, in both the forward and reverse directions [...]. </i><br /><br />I implement my "symmetric" butterworth filter (I call it cleverly my "butterworth2" filter) in the same way, run the data forward and backwards through the same butterworth filter.<br /><br />My preferred way of treating endpoints is truncating the series when the various assumptions for the end points "begin to matter". At the moment, there is no way to know the current long-term slope, because it depends on future climate change. (You could also set bounds on the error over time as you approach the endpoint of the data using a Monte Carlo approach, but I digress...)Carrickhttps://www.blogger.com/profile/03476050886656768837noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-5696375526818685212011-01-24T12:05:45.200+11:002011-01-24T12:05:45.200+11:00Anon,
As I understand, Mannian smoothing is regard...Anon,<br />As I understand, Mannian smoothing is regarded as the extrapolation method as I described and implemented. You can use it with any digital filter. Mann in his 2004 paper chose a Butterworth filter - I'm not sure why. Symmetric filters make more sense in smoothing time series where you have data on both sides of the smoothing point.<br /><br />I chose the symmetric filter used in the AR4 because it has been discussed at CA in the context of mannian smoothing. My first alternative choice would have been the Henderson filter. But I don't think that makes much difference.<br /><br />It's possible that the unpinning of the endpoint achieved with a Butterworth filter improves the performance of the Mann process. I'm travelling at the moment and can't test that, but maybe when I get back.Nick Stokesnoreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-28442227649802447882011-01-23T01:20:00.680+11:002011-01-23T01:20:00.680+11:00Can you show what the actual Mann filter will do i...Can you show what the actual Mann filter will do in your comparison? Or add a figure pointing to the differences associated with different choices for the filter?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-37990847584878428442011-01-21T16:37:38.592+11:002011-01-21T16:37:38.592+11:00Anon,
I think I see it. It's only pinned if th...Anon,<br />I think I see it. It's only pinned if the filter coefficients are symmetric. The Butterworth filter isn't. I don't know why Mann chose a Butterworth here - I can't see that causality is an issue,Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-49895648792394902082011-01-21T15:46:35.878+11:002011-01-21T15:46:35.878+11:00try octave - it an OS matlab emulatortry octave - it an OS matlab emulatorAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-23681693905104392732011-01-21T14:22:35.775+11:002011-01-21T14:22:35.775+11:00Anon,
I checked lowpass.m and it seems to do exact...Anon,<br />I checked lowpass.m and it seems to do exactly the same thing that I did, and is as described in GRL04. I don't have Matlab, so I can't implement it.Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-4961716585293548722011-01-21T14:05:27.678+11:002011-01-21T14:05:27.678+11:00Anon,
I followed the method in GRL04:
Finally, to ...Anon,<br />I followed the method in <a href="http://www.meteo.psu.edu/~mann/shared/articles/MannGRL04.pdf" rel="nofollow">GRL04</a>:<br><br><i><br />Finally, to approximate the ‘minimum roughness’ constraint, one pads the series with the values within one filter width of the boundary reflected about the time boundary, and reflected vertically (i.e., about the ‘‘y’’ axis) relative to the final value.<br></i><br />It seems to me that that reflection means that the data becomes an odd function about the final point treated as the t- and y-origin, so any smoothing approx would have to be odd too, and pass through that origin.<br><br />However, it's true that if you look very closely at his graphs, the MRC smooth passes close too butnot exactly on the final point. I'm not very familiar with Matlab, but I'll try to see what is going on.Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-23315107762524163272011-01-21T12:38:48.290+11:002011-01-21T12:38:48.290+11:00are you sure that you have emulated Mann's rou...are you sure that you have emulated Mann's routine properly? I did a quick test with his matlab lowpass routine and it does not peg the final point when using the min roughness criteria.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-46343300801257687192011-01-19T06:33:08.101+11:002011-01-19T06:33:08.101+11:00Thanks, Anon - fixed.Thanks, Anon - fixed.Nick Stokeshttps://www.blogger.com/profile/06377413236983002873noreply@blogger.comtag:blogger.com,1999:blog-7729093380675162051.post-74321814668027410072011-01-19T00:01:58.468+11:002011-01-19T00:01:58.468+11:00I think there is a copy-paste error in the section...I think there is a copy-paste error in the section 'Mann's paper and methods', as #3 point is identical in both numbered lists. And the #2 in the second list seems to end mid-sentence...Anonymousnoreply@blogger.com