S5 Segment Data Quality Repository
This page has links to files containing information about science-mode
segments in each detector during the S5 run. Science mode segment
numbers for each interferometer were incremented each time a control room
operator pushed the "GO" button to indicate start of science mode conditions.
As in the past, the segment numbers will be permanent,
and many e-log entries refer to them by IFO and segment index (e.g., H1-27).
This repository provides data quality summary information on individual
segments, based on the work of many investigators. Data quality flags are
listed for time intervals, and links to more detailed information are provided,
where appropriate. Links
to segment manipulation tools (e.g, segwizard) are also included. An
extensive set of data quality investigations are described in detail on
John Zweizig's data
quality investigation page. Note that segwizard (Peter Shawhan)
derives its segment
information from a daily automated dump from the database, not from
the files in this repository.
In contrast to previous science runs, the definitive source of
segment lists is NOT this repository, but is instead the segment
database set up and maintained by Duncan Brown. Kept here
are
- A nearly up-to-date distillation of the database (updated once
per day).
- Detailed information (or links) concerning the investigations
that have led to inserted data quality flags.
The flags in the database are inserted in two different ways:
- Automatic insertion using dmtdq_seg_insert based on minute trends produced by online DMT monitors (John Zweizig)
- Manual insertion using LSCdqInsert based on offline analysis by
detector characterization investigators (Keith Riles)
Based on the information stored in the database, the following files
are generated once per day (08:00 Eastern Time):
- For each detector, there is an ASCII "work" file ( S5H1_seginfo_work.txt
, S5H2_seginfo_work.txt , S5L1_seginfo_work.txt
)
which lists for each segment all of the subintervals for which DQ flags
are defined. The files for LIGO detectors are derived from the database,
and for other detectors, are manually updated, as information becomes
available.
- S5_seginfo.html : a copy of the seginfo work file with some HTML mark-up.
- S5_segs.html : a fixed-format list, consisting of:
- Segment index (hyperlinked to the seginfo file)
- Start time
- End time
- Duration
- Binary value representing the logical OR of any bad-flags that apply to this time interval
- List of bad-flag names, if any, which apply to this time interval
Note that there may be more than one line for a given segment index, if
the set of applicable bad-flags changes during
the segment. The user is expected to decide which bad-flags should be heeded,
omit those time intervals from the list, and stitch back together any contiguous
intervals which were not omitted.
-
S5_segs.txt : like S5_segs.html
, but without the hyperlinks.
Note that any given segment will retain the same numerical index in all
versions of the lists, but it may acquire bad-flags indicating that it
is bad or potentially bad.
The goal of this effort is to determine what data is suitable for searching
for gravitational waves, and to provide this information in a machine-readable
form. The success of this effort will depend on individuals examining the
data, identifying effects which might cause the data to be bad,
and then evaluating which effects are serious enough to disqualify
data. Contributions are eagerly awaited; Keith Riles (kriles@umich.edu
) will collect this information and update the database accordingly.
Caution: Please do not apply DQ flags in your analysis indiscriminately.
The severity of DQ flags varies considerably. Some flags (e.g., OUT_OF_LOCK or AS_TRIGGER)
should be applied by every analysis, but other flags (e.g., LIGHTDIP_02_PERCENT) may be
unimportant to some analyses. Also, for analyses requiring large contiguous blocks of good data,
e.g., for spectral noise estimation, it may be wiser to apply short-lived DQ flags
(e.g., LIGHTDIP_5PERCENT) as post-processing
vetoes rather than to fragment the data prior to processing.
Other cautions
Segment files
Copies of science & injection segment files dumped each day from the database: (last updated January 29, 2008)
Current data quality segment files (which undergo frequent updating) are
available for
H1,
H2, and
L1.