From jzweizig@ligo.caltech.edu Mon Feb 11 19:05:42 2008
Date: Mon, 11 Feb 2008 16:04:03 -0800 (PST)
From: John Zweizig 
To: Gabriela Gonzalez 
Cc: Glitch Working Group , Valera Frolov ,  
Subject: Re: BS oplev problem

Hi Gaby,

Thanks for the info. You can see the plots I used at

http://www.ligo.caltech.edu/~jzweizig/S5_Data_Quality/l-bs_optlev_sigma-87M.pdf

The channel name is at the top of each plot, but I calculate the sigma from
the minute trends from sqrt(^2 - ^2). The broken red line shows
the science mode segments, and I used a threshold of 10 on the
sigma for L1:SUS-BS_OPLEV_POUT for the segments I sent you.
Best regards,

John

On Sun, 10 Feb 2008, Gabriela Gonzalez wrote:

> Hi John,
>
> thanks for following up on this! Do you have a plot showing the trends and
> the thresholds you used?
>
> I talked a little bit more with Valera, who was one of the people
> diagnosing and fixing the problem, and I think I understand the problem a
> bt better.
> It seems that on 9/21, measurements showed that the gain in the loop was
> high enough that the phase margin at the unity gain frequency near 3.6 Hz
> was small, and the servo actually amplified rather than suppressed noise
> at 3.6 Hz (this is what we call "gain peaking"). Of course, the actual
> amplitude of the motion at 3.6 Hz depends on the amplitude of the source
> noise, not just on the loop gain.
>
> The optical lever loop gains were tuned once in a while during S5, in
> order to minimize the "gain peaking", the loops are marginal enough that
> small changes in gain can make the phase margin very small. Usually gains
> went down as lasers were dying, and if they go down too much the phase
> margin again gets small, so the gains have little margin for change.
> However, what was not known until Sept 21 is that if the tuning was done
> different conditions than the ones used in science mode (namely, with SUS
> dewhitening filters on), the loop gain will be higher in science mode than
> the tuned setting, and thus not be optimal. So I suspect that the gain on
> 8/23 was tuned in the wrong state, and set too high. Valera thinks that
> the gain may have been marginal for most of S5, though.
>
> There are still a few questions in my mind, like noticing that the yaw
> gain went down on 8/23, not up, so the problem may not have started there,
> although some elog entries suggest the problem was in pitch rather than
> yaw. Also, a search in the elog for "wobble" for a month before Sep 21
> shows entries on Sep 5, 8, 13, and 21 (by Dan Hoak on all days, Sany on
> the 13th, and Valera on the 21st), most of them mentioning the problem at
> night, when the seismic noise at 3.6 Hz should have been smaller, not
> larger.
>
> In any case, it looks like we have to look at the specific frequency in
> the BS oplev signals  to better identify times when there is a problem for
> data quality. Cristina is looking into this, hopefully she may be able to
> tell us something more definite soon.
>
> Gaby.
> ------------------------------------------------------------
> Gabriela Gonzalez
> Associate Professor
>
> Department of Physics and Astronomy
> Louisiana State University
> 202 Nicholson Hall
> Tower Drive
> Baton Rouge,LA 70803
>
> Phone: (225) 578 0468
> Email: gonzalez@lsu.edu
> http://www.phys.lsu.edu/faculty/gonzalez/
>
>
>
>
>
>
> Begin forwarded message:
>
> > From: John Zweizig 
> > Date: February 8, 2008 12:18:49 PM CST
> > To: Gabriela Gonzalez 
> > Cc: Glitch Working Group 
> > Subject: Re: BS oplev problem and equinox candidate
> > 
> > Hi Gaby,
> > 
> > I have looked at various sigma distributions reconstructed from
> > BS_OPTLEV minute trends. The POUT trends show a noticeable increase in
> > scatter at 
> > ~1430UTC on August 23, 2007 which then reverts to "normal" at ~2300UTC
> > ~Sep 
> > 21, 2007. Looking at Optlev gain channels in conlog shows:
> > 
> > Date             8/01 0000    8/23 1436    9/21 2240
> > BS_OLPIT_GAIN      121.7        134.5        106.4
> > BS_OLYAW_GAIN      307.7        296.5        245.8
> > 
> > So, assuming that the gain changes were responsible for the
> > oscillations, the time period would have been 8/23 to 9/21. However,
> > looking at the trends, I can see the oscillations you noted on 9/21, but
> > not much that is comparable in the rest of the period.
> > 
> > I also ran an automatic scan on the trends to tag segments with large
> > sigma optlev_pout sigmas. The resulting list is attached to this email.
> > The majority of such segment are probably indiuuced by sesmic activity
> > rather than the oscillations seen on 9/21, although some seem to be due
> > to oscilaations ans are clearly correlated with dips in the L1 inspiral
> > range (e.g. on 9/13 and 9/21).
> > 
> > Best regards,
> > 
> > John
> > 
> > 
> > On Mon, 4 Feb 2008, Gabriela Gonzalez wrote:
> > 
> > > Hello,
> > > 
> > > following up on the comments I made today in the glitch call, I wanted
> > > to clarify the significance  of the BS optical lever problem for the
> > > equinox candidate. My observations arise from following the large
> > > standard deviation of SenseMon in the segment L1-6313, ranked #3 in
> > > worse std for all of S5, as shown in the list here:
> > > 
> > > http://www.phys.lsu.edu/faculty/gonzalez/S5/L1BadSegs_AllS5/
> > > 
> > > (See also segment ranked #38).  According to the elog on Friday Sep
> > > 21, the "problem" that had been observed for at least a couple of days
> > > was diagnosed and fixed by Brian and Valera:
> > > 
> > > "We found that the BS oplev servo was gain peaking at 3.6 Hz in the
> > > low noise.
> > > The reason for that was the digital compensation of the dewhitening
> > > gain.
> > > It is off by ~2 dB with dewhitening on. We set the BS oplev pitch and
> > > yaw ugfs
> > > in run state with dewhitening on."
> > > 
> > > According to the elog's time and the operator's entry, the fix was
> > > done between segments L1-6323 and L1-6324, so L1-6324 and later ones
> > > should not be affected.
> > > The equinox candidate, on Sep 22 3:05 UTC, happens during L1-6325, and
> > > thus cannot have been caused by the optical lever problem.
> > > 
> > > The segments L1-6312 to L1-6315 are very obviously affected; the ones
> > > that follow (L1-6315-6318) do not show obvious problems in SenseMon
> > > but there maybe problems in the data quality, the problem appears
> > > again very badly in L1-6323. I don't know when the problem started,
> > > the elog on Sep 08 mentions similar oscillations, so it may have been
> > > there for some time.
> > > 
> > > Although the problem does not affect the segment with the equinox
> > > candidate, it may affect estimates of false alarm that use times near
> > > the candidate, if they include the noisy segments.
> > > 
> > > The sequence I reconstruct is then:
> > > 
> > > Segments most of which show a problem in range due to the BS
> > > oscillations:
> > > L1-6312   4491 s    874369184- 874373675   2007 09/21 00:19:30 - 09/21
> > > 01:34:21 utc
> > > L1-6313   5115 s    874374723- 874379838   2007 09/21 01:51:49 - 09/21
> > > 03:17:04 utc
> > > L1-6314   2745 s    874381600- 874384345   2007 09/21 03:46:26 - 09/21
> > > 04:32:11 utc
> > > L1-6315   9760 s    874385837- 874395597   2007 09/21 04:57:03 - 09/21
> > > 07:39:43 utc
> > > L1-6316   3526 s    874397966- 874401492   2007 09/21 08:19:12 - 09/21
> > > 09:17:58 utc
> > > L1-6317  19288 s    874402396- 874421684   2007 09/21 09:33:02 - 09/21
> > > 14:54:30 utc
> > > L1-6318    345 s    874421938- 874422283   2007 09/21 14:58:44 - 09/21
> > > 15:04:29 utc
> > > L1-6319   3611 s    874429308- 874432919   2007 09/21 17:01:34 - 09/21
> > > 18:01:45 utc
> > > L1-6322   2025 s    874434243- 874436268   2007 09/21 18:23:49 - 09/21
> > > 18:57:34 utc
> > > L1-6323   6618 s    874436696- 874443314   2007 09/21 19:04:42 - 09/21
> > > 20:55:00 utc
> > > 
> > > Segment that follows the fix, which should not have problems due to
> > > the BS oplev servo:
> > > L1-6324   2007 s    874450664- 874452671   2007 09/21 22:57:30 - 09/21
> > > 23:30:57 utc
> > > 
> > > Segment that contains the equinox candidate:
> > > L1-6325  35320 s    874452908- 874488228   2007 09/21 23:34:54 - 09/22
> > > 09:23:34 utc
> > > 
> > > I hope this is useful!
> > > 
> > > Gaby.
> > > 
> > > ------------------------------------------------------------
> > > Gabriela Gonzalez
> > > Associate Professor
> > > 
> > > Department of Physics and Astronomy
> > > Louisiana State University
> > > 202 Nicholson Hall
> > > Tower Drive
> > > Baton Rouge,LA 70803
> > > 
> > > Phone: (225) 578 0468
> > > Email: gonzalez@lsu.edu
> > > http://www.phys.lsu.edu/faculty/gonzalez/
> > > 
> > > 
>