From shourov@ligo.caltech.edu Sat Oct  7 18:20:42 2006
Date: Sat, 7 Oct 2006 18:19:16 -0400
From: Shourov Keith Chatterji 
To: Keith Riles 
Cc:  ,  , Greg Mendell , Ben Johnson 
Subject: Re: missing data flags


Hi Keith,

Xavi can say for sure, but to me it looks like h(t) is not available
for the first and last 6 seconds of a lock stretch.  Ideally it is
produced for all locked data, so the 6 seconds at the start should not
affect science mode segments.  The 6 seconds at the end will have a
minor affect.  Ideally they should not affect analyses that use one of
the PRE_LOCKLOSS flags.  So I think the only concern should be the 12
additional seconds around missing data.  From my discussion with Xavi,
there are a few short periods of missing data that prohibited the
production of h(t).

However, I have also found a few segments of data with unknown state.
This stems from my follow-up of missing h(t) data in the S5 Q Pipeline
analysis.  One such segment occurs in H1 from gps time 817494090 to
817494136.  I looked at the daily dumps, LSCsegFind, conlog, the
detector log, the injection log, and the available h(t) data.  My
detailed notes on this particular event are posted at the following
URL.

http://ldas-jobs.ligo.caltech.edu/~shourov/reports/20061007_missingdata/

From what I can tell, H1 was removed from science mode by the
operator, but remained in lock.  But, for some reason, no type
argument to LSCsegFind indicates that H1 is in lock.  As a result,
h(t) was not generated.  At the same time, no PRE_LOCKLOSS data
quality flag was generated, and my analysis reported an error due to
missing data.  It is also strange that the detector returns first to
injection mode before science mode.  It looks like a scheduled
injection just before leaving science mode puts the detector in a
state that is not easily queried using LSCsegFind.  Is there an
"impending injection bit" in the state word?

In my list of problem times, I found a few such strange segments.  The
only other problems were missing h(t) data due to missing raw data, an
erroneous injection segment in the daily dumps, and also some missing
h(t) data that should be available but isn't.  I talked to Xavi about
the latter, and we decided that it is a small enough quantity of data
that we should wait until the V3 calibration is available and then
reproduce everything.

The erroneous injection segment is the first one in the daily dump,

http://ldas-cit.ligo.caltech.edu/segments/S5/H1/injection_segments.txt

and is not reported when directly querying LSCsegFind.  I already
sent an email to Duncan and Igor about this one.

From what I can tell, these are minor concerns.  The total amount of
missing data from my analysis totaled around 2 hours out of the full
H1H2 data set from the beginning of S5 through April 3.  Nonetheless,
it would be nice to correct any problems if at all possible.

Shourov